Categories
Posts

XMLHttpRequest (XHR) Uses Multiple Packets for HTTP POST?

A recent Think Vitamin article, The Definitive Guide to GET vs POST, mentioned something that I hadn’t seen before about XMLHttpRequest (XHR). Their Rule #4 states:

When using XMLHttpRequest, browsers implement POST as a two-step process (sending the headers first and then the data). This means that GET requests are more responsive – something you need in AJAX environments.

The claim is that even the smallest XHR will be sent using two packets if the request is done over HTTP POST instead of HTTP GET. I don’t remember ever having heard this claim before.

Let me first say that performance issues for POST vs. GET probably shouldn’t be your top factor for deciding which one to use. Make sure that you understand the implications of each and pick the right method for your request. For most people I suspect the biggest factor will involve caching, not performance. I was going to leave a comment on the article about this, but Simon beat me to it.

I wasn’t the only one who wanted to find out more about XHR POST using multiple packets. Fortunately someone else already asked that question and the author replied:

2. My claim is based on research done by Iain Lamb, cofounder of the Oddpost webmail startup that was acquired by Yahoo! and eventually became the basis for the all-new Yahoo! Mail.

His research showed “rather baffling finding: POST requests, made via the XMLHTTP object, send header and body data in separate tcp/ip packets [and therefore,] xmlhttp GET performs better when sending small amounts of data than an xmlhttp POST.”

That is why Yahoo includes the use of GET instead of POST as one of their high performance speed optimisation rules.

Simon Willison did some looking around and found more links for this. It was mentioned here and here, so it looks like Iain Lamb did do this research, even though I couldn’t find a first person account of it. This was enough information to make me curious, but not enough to answer all of my questions. It was time to run some tests of my own.

So I updated my install of Wireshark on Windows XP, turned off all of the packet reassembly options for HTTP decoding and started testing browsers. My very simple XHR POST test page looked like this:

[sourcecode lang=”html”]
<button type="button" onclick="$.post(‘hello.txt’, {name: ‘Joseph’})">XHR POST</button>
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js"></script>
[/sourcecode]

When the button is clicked an XHR POST request is made to hello.txt with the name=Joseph for a tiny amount of data. The domain I tested on sent along some cookies as well, but still left enough room for the tiny POST payload to fit in a single TCP packet.

Here are the results of the tests that I ran:

  • IE 6 – 2 packets
  • IE 7 – 2 packets
  • IE 8 – 2 packets
  • Firefox 3.0.13 – 1 packet
  • Firefox 3.5.2 – 1 packet
  • Opera 9.27 – 2 packets
  • Safari 4.0.3 – 2 packets
  • Chrome 2.0.172.43 – 2 packets

The short version of this is pretty easy to see, all of the browsers except for Firefox will use at least 2 packets for an XHR done over HTTP POST. When I saw that Safari sent 2 packets I figured that Chrome would as well, but I tested it anyway just to make sure.

I looked at the data size of each packet in IE 6; the first packet had 575 bytes of data and the second packet had 11 bytes of data. This lined up with the POST request which indicated that the content length was 11 bytes. The second packet consisted only of the POST data. Because Firefox sent less data in the user-agent string I increased the POST data so that it would exceed the combined total of the two IE packets to make sure I wasn’t running into any odd packet fragmentation. The second packet in Opera, Safari and Chrome was also only the 11 bytes of POST data.

If this were Myth Busters I’d call this myth confirmed. While it is true that not ALL browsers will always use two packets, it appears that the two packet process is the rule, not the exception. And with IE still the most widely used browser it’s very likely that a large portion of your users fall into the two packet category. If on the other hand 95% of your users happen to be using Firefox, then sure, you can skip thinking about this.

27 replies on “XMLHttpRequest (XHR) Uses Multiple Packets for HTTP POST?”

I suspect this is a case of premature optimisation.

The browser implementer probably decided that as POST requests will likely have a large body and the server will need all of that data before it processes the request that it is better to try and send it all in a single TCP packet.

It also allows for the server to do all the processing it might want to do on the header before the POST data arrives too.

But it should really only trigger for large amounts of POST data 🙂

Thanks for taking the time to run the tests.

I had scheduled them into my timetable, but never got around to it as I’m so busy. Will point here next time?

No problem. I thought it such an odd thing that I wanted to see them for myself. I think westi is right, this was probably done as optimization with the thought that most POST requests wouldn’t fit in a single packet anyway.

I would have expected a (minimum) 2 packet POST to be the norm.

RFC2616 HTTP/1.1 introduces the “Expect: 100-continue” mechanism which is meant to be used for POSTs, where the client sends the HTTP request headers first including “Expect: 100-continue” and then waits for the server (so it can setup whatever resources it needs to handle the POST body) to send either a 100 (Continue) or 417 (Expectation Failed) response before sending the POST body.

Out of curiosity: when you tested Firefox were you using a proxy server? It’s likely that Firefox was sending a HTTP/1.0 request through the proxy, as a single packet (single HTTP/1.0 didn’t define Expect: 100-continue headers), as I don’t think Firefox sends HTTP/1.1 through proxies by default.

Another possibility… does your Firefox config have network.http.pipelining and/or network.http.proxy.pipelining options set to true? If so, try setting them to false and repeating your tests.

Both network.http.pipelining and network.http.proxy.pipelining (though I did not use a proxy) are set to false. For the most part the Firefox installs are fairly stock, with just a couple of plugins installed.

Thanks for taking the time out to test this – it’s an interesting fact, albeit perhaps not a hugely important one.

If you’re using XHR to replace conventional postbacks, the odds are that you will be sending a lot less post-data, because you’ll be doing fragmented transactions (however you want to call them) – so it’s probably a net gain overall in most situations!

Determining if this is hugely important depends on what you are using the XHRs for. For basic things like submitting a form without reloading the page, sure, not a big deal. If you have a web app that is doing lots of updates (chat, live stream, etc.) then it could become enormously important.

Hi, First of all thank you very much for such a good article. Do you have any idea on what happens if second packet gets lost in the post operation? I am running out of one problem just trying to correlate my problem with this article.

Thanks & regards,
chandru

Thank you very much! We have such a web app that is doing lots of updates and to us this is enormously important. After doing some wiresharking and emulating some loss, jitter, low bandwidth and reordering (using tc) and reading your post I decided to change all the ajax post requests to ajax get requests hoping it had something to do with the 2 packet issue. And it did: after that no more strange 404’s on the ajax calls appeared in the apache access log. I still didn’t figure out why exactly, but emulating these conditions is no easy task either, so I may never find out.

After pulling my hair out for four days while working on an HTTP server implementation using .NET sockets, I was able to determine that i may actually get a minimum of one packet but usually about 5 packets for a single HTTP post on most mobile browsers. This for an AJAX call with a content length of about 11 Kb.

In case you’re curious about why this happens in IE– the headers are sent by WinINET, but the calling pattern of XHR through URLMon to WinINET results in going async, which means that the POST body bytes aren’t typically supplied to WinINET until after the headers have been sent. This is something that we looked at changing in IE10 but ran out of time. In practice, the performance impact is pretty small.

To chandru’s question: The issue here is that the POST body can get lost if the browsing process is torn down in the window between the headers being sent to WinINET and the body being sent down. As a consequence, while a window is closing, you could end up with the headers and no body. (Fiddler will warn if it sees this happen).

October 2016 and I am experiencing this problem with Safari 9.1.3 (10601.7.8) on Mac OS and also on iOS 10.0.2.
It is sending the HTTP headers in one packet and the body in a second packet, even for a very small request.

I am implementing a tiny IPv6 HTTP server on Arduino, which can’t currently cope with requests spanning multiple packets. So very frustrating that it works with Chrome but not Safari.

The good news however it that Chrome 53.0.2785.143 (also on Mac) is sending a single packet, containing both the headers and body.

Leave a Reply

Your email address will not be published. Required fields are marked *