Author v+python
Recipients amaury.forgeotdarc, barry, eric.araujo, erob, flox, ggenellina, gvanrossum, oopos, pebbe, quentel, r.david.murray, tcourbon, tercero12, tobias, v+python
Date 2011-01-03.20:45:27
SpamBayes Score 0.0
Marked as misclassified No
Message-id <>
David, Starting from a working (but hacked to work) version of http.server and using 3.2a1 (I should upgrade to the Beta, but I doubt it makes a difference at the moment), I modified

        # if hasattr( sys.stdin, 'buffer'):
        #     sys.stdin = sys.stdin.buffer
        sys.stdin = sys.stdin.detach()

and it all kept working.

Then I took out the

    try: # Windows needs stdio set for binary mode.
        import msvcrt
        msvcrt.setmode (0, os.O_BINARY) # stdin  = 0
        msvcrt.setmode (1, os.O_BINARY) # stdout = 1
        msvcrt.setmode (2, os.O_BINARY) # stderr = 2
    except ImportError:

and it quit working.  Seems that \r\r\n\r\r\n is not recognized by Firefox as the "end of the headers" delimiter.

Whether this is a bug in IO or not, I can't say for sure.  It does seem, though, that

1) If Python is fully replacing the IO layers, which in 3.x it seems to claim to, then it should fully replace them, building on a binary byte stream, not a "binary byte stream with replacement of \n by \r\n".  The Windows hack above replaces, for stdin, stdout, and stderr, a "binary byte stream with replacement of \n by \r\n" with a binary byte stream.  Seems like Python should do that, on Windows, so that it has a chance of actually knowing/controlling what gets generated.  Perhaps it does, if started with "-u", but starting with "-u" should not be a requirement for a properly functioning program. Alternately, the IO streams could understand, and toggle the os.O_BINARY flag, but that seems like it would require more platform-specific code than simply opening all Windows files (and adjusting preopened Windows files) during initialization.

2) The weird CGI processing that exists in the released version of http.server seems to cover up this problem, partly because it isn't very functional, claims "alternate semantics" (read: non-standard semantics), and invokes Python with -u when it does do so.  It is so non-standard that it isn't clear what should or should not be happening.  But the CGI scripts I am running, that pass or fail as above, also run on Windows 2.6, and particularly, Unix 2.6, in an Apache environment.  So I have been trying to minimize the differences to startup code, rather than add platform-specific tweaks throughout the CGI scripts.

That said, it clearly could be my environment, but I've debugged enough different versions of things to think that the Windows hack above is required on both 2.x and 3.x to ensure proper bytestreams.... and others must think so too, because I found the code by searching on Google, not because I learned enough Python internals to figure it out on my own.  The question I'm attempting to address here, is only that 3.x still needs the same hack that 2.x needs, on Windows, to create bytestreams.
Date User Action Args
2011-01-03 20:45:29v+pythonsetrecipients: + v+python, gvanrossum, barry, amaury.forgeotdarc, ggenellina, eric.araujo, r.david.murray, oopos, tercero12, tcourbon, tobias, flox, pebbe, quentel, erob
2011-01-03 20:45:28v+pythonsetmessageid: <>
2011-01-03 20:45:27v+pythonlinkissue4953 messages
2011-01-03 20:45:27v+pythoncreate