HTTP Message basics, and POSTing form data in Graphical HTTP Client

“The POST body isn’t working correctly! I add in something like name=spencer&age=29, but the server doesn’t get the parameters. Please fix your program!”

After a few support requests and 1 one-star review related to this issue, I figured it was at the very least worth a deeper explanation. If you’ve googled this and are just looking for the answer here it is:

You need to set the Content-Type header to ‘application/x-www-form-urlencoded’.

That said, I think it is worth spending a few minutes to learn about HTTP messages and why this is required. Removing levels of abstraction and learning more about what is going on underneath makes you a better developer and is usually a lot of fun, too. This is a big part of the reason I’m giving up my evenings and weekends to go back to school for a CS degree.

Anyway, here is a quick primer on HTTP messages and why you have to add the header above…

The HTTP message

When you make an HTTP request, you’re connecting to an HTTP server at a given IP on a certain port and sending it a message. It then processes that message and gives you some sort of response.

What does that message look like? It consists of 3 parts:

Request Line. This tells the server what resource you’re looking for and what HTTP method (ie, one of GET, POST, PUT, DELETE, etc..) you are using. It also specifies the version of the HTTP protocol to use.
Headers. These give metadata about the message, such as the Content-Type, the Host, what type of content you’ll accept, and a whole bunch of other things.
[CRLF]
Optional message body.

So, for example, if you came to the homepage of my blog, your browser sent a message to my server that looked something like this:

GET / HTTP/1.1
Host: www.spenceruresk.com
[CRLF]

Pretty simple, right? Let’s look at a slightly more complex example. Let’s say I was hosting a Twitter-like service that allowed you to post status updates with simple HTTP POSTs. A request you’d send might look something like this:

POST /twitterClone/updateStatus HTTP/1.1
Host: www.spenceruresk.com
Content-Type: text/plain
[CRLF]
Check out this crappy cellphone pic I took at a concert!

In this case, we’re POSTing some data and telling the server what kind of content it is – in this case, it is just plain text. Pretty simple, right?

Content-Type and message bodies

One header in particular – Content-Type – is relevant to the issue and is worth talking about. When you send a message body, it can represent any number of types of data. For example, it could be plain text, a GIF, form data, or a JSON document. In order for the server to properly decode the body, you have to tell it what it is you’re sending it. The Content-Type header is how you do this. For a list of common MIME types, this Wikipedia article is useful.

Submitting forms

So, what HTTP message is generated and sent to the server when a user submits a form in their browser? The answer is that it depends. If the form is submitted via GET request, the message would look something like this:

GET /signupform?name=Spencer&age=29 HTTP/1.1
Host: www.spenceruresk.com
[CRLF]

Your web framework will then take the key/value pairs in the query string and make them available as request parameters.

What if it is a POST?

POST /signupform HTTP/1.1
Host: www.spenceruresk.com
Content-Type: application/x-www-form-urlencoded
[CRLF]
name=Spencer&age=29

The Content-Type line is critical – it tells the server “I’m sending you URL-encoded form data in the message body.” Most/All web frameworks see that, then decode the body and make each key/value pair available as request parameters.

If you don’t tell it that you are sending URL-encoded form data, it doesn’t decode the body and make the parameters available.

Conclusion

Hope that helps make it at least a little bit clearer. In the next version of the app, I’m thinking about adding some sort of Content-Type autodetection (that you can turn off) to help avoid people getting confused and frustrated in situations like this.

Leave a Comment


NOTE - You can use these HTML tags and attributes:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> <pre lang="" line="" escaped="">