Firefox vs. Chrome vs. IE8: multiple connections to one HTTP/SCGI/WSGI app

I am currently evaluating ways to run a persistent Python WSGI Webapp using Apache2 or Lighttpd. The idea is to have a daemon running on it’s own doing all the work, that also has an interface towards the webserver.

So I set up Lighttpd as SCGI client and used flup to wrap a simple WSGI app in a threaded SCGI server. The configuration is pretty straight forward …

server.modules   += ( "mod_scgi" )
scgi.debug = 1

scgi.server = (
    "/" =>
    (
        "127.0.0.1" =>
        (
        "host" => "127.0.0.1",
        "port" => 4000,
        "check-local" => "disable",
        "disable-time" => 3
        )
    )
)

… and an example app using flup is also available online. I added a few lines to log the time the request arrived, a random time.sleep() and a log for after the sleep to simulate longer requests.

msg = "%s: %s: sleeping %s seconds, request %s" % (time.time(), thread.get_ident(), local, request)
time.sleep(random.randint(0,10))
msg = "%s\n%s: %s: slept for %s seconds!" % (msg, time.time(), thread.get_ident(), local)

The weirdness started when I actually tried to load the app in multiple tabs in Firefox at once. They were executed strictly one after the other. The start time stamp of the second tab was always about the finish time of the first and so on. Same effect with Chrome!

There is a forking and a threaded version of the flup SCGI server. I tried both. The thread.get_ident() output from my debug lines clearly showed that flup was multithreading. The serverside log of the SCGI/WSGI application however showed that the “GET /” requests did not come in all at once!

So I also set up Apache2 for the same SCGI server to see if Lighttpd was not working right. The configuration looked like this:

LoadModule scgi_module /usr/lib/apache2/modules/mod_scgi.so
SCGIMount / 127.0.0.1:4000

Again, the requests were processed strictly sequential.

Ok, so maybe SCGI just sucks. So I tried mod_wsgi with Apache2, but of course that showed the same behavior.

Finally, in a moment of desperation, I tested the app using IE8… and everything worked perfectly. All three start-timestamps were about the same. Now… what…?!

Firefox’s “network.http.max-connections” is set to it’s default value of 8. Don’t know about Chrome, but a Wireshark trace clearly showed that Firefox and Chrome send the “GET /” requests only after the previous one finished (using the same source port), while IE8 fires all of them at once (opening a new socket for each).

What is this!? Some weird “only one connection per server IP” rule? If anybody can shed some light on this, please comment!