This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: In Lib/urllib/parse.py quote_from_bytes, exception is thrown if bs = None
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.9
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: asicscwcs, eric.smith, iritkatriel, orsenthil, serhiy.storchaka
Priority: normal Keywords:

Created on 2021-06-28 08:25 by asicscwcs, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 26887 asicscwcs, 2021-06-28 08:25
Messages (16)
msg396617 - (view) Author: Олег Масло (asicscwcs) Date: 2021-06-28 08:25
need moved the check parameter to the top of the isnstance check, because if bs = None then an exception is thrown
msg396618 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-06-28 08:36
Crash is typically a segfault, not an exception.
msg396620 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-06-28 08:45
Please describe the problem you would like to see fixed, rather than the code change you would like to make.
msg396625 - (view) Author: Олег Масло (asicscwcs) Date: 2021-06-28 09:44
Irit, the problem is that if you pass the bs parameter equal to None to the quote_from_bytes function, then the bs parameter will not be checked for existence, because a TypeError exception will be raised, and an empty string needs to be returned in this case. The check "if not bs" should be earlier than "if not isinstance (bs, (bytes, bytearray))", because "if bs = None", then the code will not reach this check
msg396626 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-06-28 09:52
Right, so the question is whether it is correct that "an empty string needs to be returned in this case". Adding orsenthil to answer that as the module's expert.

Also, your patch would need a unit test.
msg396631 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2021-06-28 13:02
Clarifying the title.
msg396636 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-06-28 14:06
If you want to interpret None as an empty string, you can just write quote_from_bytes(data or b'').
msg396660 - (view) Author: Олег Масло (asicscwcs) Date: 2021-06-28 19:51
That is, all the libraries already created need to follow your advice? :)
msg396661 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-06-28 19:53
I don’t know if it needs to change, but if anything I would map None to None and not to ‘’.
msg396848 - (view) Author: Олег Масло (asicscwcs) Date: 2021-07-02 07:50
What are the next actions? Do I need to do something or are we waiting for something?
msg396849 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-07-02 08:55
As I wrote on the PR, and again on this issue, a PR that makes a behavior change (like this one) is not complete without a unit test (which fails before the change and passes after the change, and can show what the bug being fixed is).  You could add such a test.

In this case, it is also not clear that the current behavior is a bug at all, and if it is then what the fix should be (you propose to map None to ''. But why not 'None'? Why not 'Mary had a little lamb'?  I suggested to return None rather than some arbitrary string). You could explain why you think it's a bug and why you think '' is the correct return value.

Once you do write the test and there is consensus that it is a bug and we agree what the fix should be, it should be some core dev's top priority to review and merge the PR (as opposed to reviewing and merging another PR). So you could push it forward by explaining why this bug is important.
msg396850 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-07-02 09:34
The documentation states that this function accepts bytes:

https://docs.python.org/3/library/urllib.parse.html#urllib.parse.quote_from_bytes

None is not of type bytes, so raising a TypeError is not unreasonable. It would certainly be wrong to return any string. It could have returned None but that doesn't make a massive usability difference and it's not worth changing now.

I propose to close this as 'not a bug'.
msg396855 - (view) Author: Олег Масло (asicscwcs) Date: 2021-07-02 11:34
If you pass None to the quote_from_bytes function, then there is no point in the "if not bs" check, because it won't even reach it. 

This function is not with dynamic behavior, which violates python concepts. If you pass a string instead of bytes, it will throw a TypeError exception, it's ok. But if for some reason you need to pass None, and this happens, then the function does not behave as expected.

Why even check that bs is not None, if this can never be? And if it does, there will always be a TypeError exception.
msg396857 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-07-02 11:42
There is still a point in the "if not bs:" check, it will be true for bs which is an empty string.
msg396859 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-07-02 11:47
I meant:

... it will be true for bs which is an empty bytes().


You are thinking of b'' and None as if they are the same thing. They are not.  If this was a check for None it would be "if bs is None" and not "if not bs".
msg396860 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2021-07-02 11:54
I agree this should be closed (and I'm closing it). I don't see any reason why this function should do something other than raise TypeError if given bs=None. If you want that behavior, write a small wrapper function.

The "if not bs" check appears to be an optimization for the case of zero-length input. Hopefully the code would continue to work without that test (or instead testing for len(bs)==0), but in my opinion it's not worth the risk of removing or changing it.
History
Date User Action Args
2022-04-11 14:59:47adminsetgithub: 88686
2021-07-02 11:54:38eric.smithsetstatus: pending -> closed

messages: + msg396860
stage: resolved
2021-07-02 11:47:41iritkatrielsetstatus: open -> pending
2021-07-02 11:47:37iritkatrielsetstatus: pending -> open

messages: + msg396859
2021-07-02 11:44:52iritkatrielsetstatus: open -> pending
2021-07-02 11:42:31iritkatrielsetmessages: + msg396857
2021-07-02 11:34:53asicscwcssetstatus: pending -> open

messages: + msg396855
2021-07-02 09:34:25iritkatrielsetstatus: open -> pending
resolution: not a bug
messages: + msg396850
2021-07-02 08:55:39iritkatrielsetmessages: + msg396849
2021-07-02 07:50:18asicscwcssetmessages: + msg396848
2021-06-28 19:53:16iritkatrielsetmessages: + msg396661
2021-06-28 19:51:14asicscwcssetmessages: + msg396660
2021-06-28 14:06:09serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg396636
2021-06-28 13:02:58eric.smithsetnosy: + eric.smith

messages: + msg396631
title: exception is thrown if bs = None -> In Lib/urllib/parse.py quote_from_bytes, exception is thrown if bs = None
2021-06-28 09:52:56iritkatrielsetnosy: + orsenthil
messages: + msg396626
2021-06-28 09:44:13asicscwcssetmessages: + msg396625
2021-06-28 08:45:45iritkatrielsetmessages: + msg396620
2021-06-28 08:36:56iritkatrielsettype: crash -> behavior

messages: + msg396618
nosy: + iritkatriel
2021-06-28 08:25:44asicscwcscreate