This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: http.server.BaseHTTPRequestHandler end_header() fails
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: andrei.avk, endian, grumblor, maggyero
Priority: normal Keywords:

Created on 2021-03-11 11:09 by grumblor, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (5)
msg388498 - (view) Author: grumblor (grumblor) Date: 2021-03-11 11:09
Python Version 3.8
http.server version 0.6
This is current install in new xubuntu 20.04 LTS, no idea if this is fixed in other version but appears to be present on github 
https://github.com/python/cpython/blob/3.9/Lib/http/server.py
at line 525

http.server.BaseHTTPRequestHandler end_headers() can reference _header_buffer array before it is assigned.

Should this be updated to something like the following? This fixes the problem of end_headers() failing for me:


    def end_headers(self):
        if not hasattr(self, '_headers_buffer'):
            self._headers_buffer = []
        """Send the blank line ending the MIME headers."""
        if self.request_version != 'HTTP/0.9':
            self._headers_buffer.append(b"\r\n")
            self.flush_headers()


This is my first issue, apologies for any mistakes I might have made.
msg388500 - (view) Author: grumblor (grumblor) Date: 2021-03-11 11:31
perhaps simply returning the method if _headers_buffer isn't defined is better since there is probably nothing to flush.

    def end_headers(self):
        if not hasattr(self, '_headers_buffer'):
            return 1 
        """Send the blank line ending the MIME headers."""
        if self.request_version != 'HTTP/0.9':
            self._headers_buffer.append(b"\r\n")
            self.flush_headers()
msg396936 - (view) Author: Andrei Kulakov (andrei.avk) * (Python triager) Date: 2021-07-04 02:57
Why do you need to `end_headers()` before first doing `send_header()`?

I guess the normal use should be:

if headers:
  for h in headers: send_header(h)
  end_headers()
msg407693 - (view) Author: Andrei Kulakov (andrei.avk) * (Python triager) Date: 2021-12-05 00:22
It seems like sending zero headers is not supported, because:

- if using http/1.0
https://www.rfc-editor.org/rfc/rfc7230#section-6.1  -- Connection: close must be sent

- if using http/1.1 -- https://docs.python.org/3.11/library/http.server.html#http.server.BaseHTTPRequestHandler.protocol_version

Content-Length must be included.

Therefore closing as not a bug.
msg411017 - (view) Author: Géry (maggyero) * Date: 2022-01-20 10:36
> http.server.BaseHTTPRequestHandler end_headers() can reference _header_buffer array before it is assigned.

@grumblor I was about to open the same bug after reading the implementation of http.server this morning and noticing that the attribute _headers_buffer of BaseHTTPRequestHandler is used in 4 methods:

- send_response_only;
- send_header;
- end_headers;
- flush_headers

but its existence is not checked only in end_headers.

> It seems like sending zero headers is not supported

@andrei.avk It is actually supported by the syntax of HTTP/1.1 messages, cf. RFC 7230, § 3:

    HTTP-message   = start-line
                          *( header-field CRLF )
                          CRLF
                          [ message-body ]

For instance the method handle_expect_100 does not send any header:

    def handle_expect_100(self):
        self.send_response_only(HTTPStatus.CONTINUE)
        self.end_headers()
        return True

It only writes a start line (which includes \r\n) followed by an empty line (\r\n) as a response:

    HTTP/1.1 100 Continue\r\n\r\n

But self.end_headers() does not raise an AttributeError here like one might expect from its implementation:

    def end_headers(self):
        if self.request_version != 'HTTP/0.9':
            self._headers_buffer.append(b"\r\n")
            self.flush_headers()

because, contrary to what its name suggests, self._headers_buffer does not only include the response headers but also the response start line, which is appended to the buffer before by self.send_response_only(HTTPStatus.CONTINUE):

    def send_response_only(self, code, message=None):
        """Send the response header only."""
        if self.request_version != 'HTTP/0.9':
            if message is None:
                if code in self.responses:
                    message = self.responses[code][0]
                else:
                    message = ''
            if not hasattr(self, '_headers_buffer'):
                self._headers_buffer = []
            self._headers_buffer.append(("%s %d %s\r\n" %
                    (self.protocol_version, code, message)).encode(
                        'latin-1', 'strict'))

So I am not sure it is a bug if we consider that send_response_only (which appends a start line to the buffer) is a precondition to end_headers (which appends an empty line to the buffer and flushes it). But then flush_headers should also have this precondition instead of preventing the AttributeError like this:

    def flush_headers(self):
        if hasattr(self, '_headers_buffer'):
            self.wfile.write(b"".join(self._headers_buffer))
            self._headers_buffer = []

Let’s ask Andrew Schaaf (@endian) who introduced flush_headers in Python 3.3 (cf. https://bugs.python.org/issue3709) why he implemented end_headers by contract and flush_headers defensively.
History
Date User Action Args
2022-04-11 14:59:42adminsetgithub: 87640
2022-01-20 10:36:25maggyerosetnosy: + maggyero, endian
messages: + msg411017
2021-12-05 00:22:33andrei.avksetstatus: open -> closed
versions: + Python 3.9, Python 3.10, Python 3.11, - Python 3.8
type: behavior
messages: + msg407693

resolution: not a bug
stage: resolved
2021-07-04 02:57:21andrei.avksetnosy: + andrei.avk
messages: + msg396936
2021-03-11 11:31:51grumblorsetmessages: + msg388500
2021-03-11 11:09:59grumblorcreate