classification
Title: http.client header debug output format
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.4, Python 3.3
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Kim.Gräsman, terry.reedy
Priority: normal Keywords:

Created on 2013-02-19 06:52 by Kim.Gräsman, last changed 2013-02-22 20:05 by Kim.Gräsman.

Files
File name Uploaded Description Edit
httpclient.debuglevel.txt Kim.Gräsman, 2013-02-19 06:52 Example of httpclient log output
Messages (3)
msg182367 - (view) Author: Kim Gräsman (Kim.Gräsman) * Date: 2013-02-19 06:52
Python 3.2.3 on 64-bit Windows 7.

When I set debuglevel on HTTPConnection to 1, the output seems jumbled, and I'm having trouble interpreting it.

Attached is a full, anonymized log from a conversation I was troubleshooting.

Here's an excerpt:

 send: b'GET /a HTTP/1.1\r\nHost: localhost:55380\r\nAccept-Encoding: identity\r\n\r\n'
 reply: 'HTTP/1.1 503 Service unavailable\r\n'
 header: Connection header: Content-Length header: Content-Type header: Date send: b'GET /a HTTP/1.1\r\nHost: localhost:55380\r\nAccept-Encoding: identity\r\n\r\n'
 reply: 'HTTP/1.1 503 Service unavailable\r\n'

- Does line 3, starting with header:, show headers for the request or response? I'm guessing response, but it didn't occur to me until just now, after a full day of looking at it.
- It would be nice if the header dump showed both header key and value
- There's a trailing "send:" at the end of the header: line, shouldn't that be on its own line?

Overall, not printing a newline after these debug statements makes it really hard to combine with other print debugging, as unrelated prints show up at the end of every line of httpclient's debug output.

I find "header" a bit confusing as a prefix, as it doesn't say whether it's request or response headers. Maybe change the prefix to "response-headers:" and also add "request-headers:", even if those are visible in the "send:" lines? Alternatively, why not show the response headers in the "reply:" dump?

Would you consider patches to address these concerns?

Thank you!
msg182689 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-02-22 19:15
The doc is vague on debug output (intentionally so, I suspect).
" Any value greater than 0 will cause all currently defined debug output to be printed to stdout."

I agree that send: and reply: should be formatted the same, so the reply: line should include the headers *with* the values. The current header line seems to be truncated after Date, without even its ':', and it is definitely missing a newline. Are you claiming that there are other missing newlines? I don't understand the lines with
send: bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00')

Patches are good, but you might wait a few days for other comments as I am definitely not an http expert.
msg182694 - (view) Author: Kim Gräsman (Kim.Gräsman) * Date: 2013-02-22 20:05
Thanks for your response!

> I agree that send: and reply: should be formatted the same, 
> so the reply: line should include the headers *with* the
> values.

OK, yeah, that would avoid having to specify request/response for headers as well, I think.

> The current header line seems to be truncated after Date,
> without even its ':',

It doesn't look like there are any trailing colons on header names. Rather, all header names have a "header: " prefix. As you can see, it's pretty hard to parse :-)

> and it is definitely missing a newline.

Glad we agree!

> Are you claiming that there are other missing newlines?

It's not visible in the attached log, but if you add print() statements in a script using http.client, the output usually ends up at the end of one of http.client's log lines. So I think every log output is missing a newline.

> I don't understand the lines with
> send: bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00')

We're sending binary data in our HTTP body, so this is the actual content. For some reason, though, the fact that it's binary seems to force it onto its own line.

> Patches are good, but you might wait a few days
> for other comments as I am definitely not an http expert.

OK, thanks again!
History
Date User Action Args
2013-02-22 20:05:35Kim.Gräsmansetmessages: + msg182694
2013-02-22 19:15:59terry.reedysetnosy: + terry.reedy

messages: + msg182689
versions: + Python 3.3, Python 3.4, - Python 3.2
2013-02-19 06:52:54Kim.Gräsmancreate