Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adopt binacii.a2b_base64's strict mode in base64.b64decode #88856

Closed
idan22moral mannequin opened this issue Jul 20, 2021 · 3 comments
Closed

Adopt binacii.a2b_base64's strict mode in base64.b64decode #88856

idan22moral mannequin opened this issue Jul 20, 2021 · 3 comments
Labels
3.11 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@idan22moral
Copy link
Mannequin

idan22moral mannequin commented Jul 20, 2021

BPO 44690
Nosy @gpshead, @idan22moral
PRs
  • bpo-44690: Adopt binacii.a2b_base64's strict mode in base64.b64decode #27272
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2021-08-23.23:45:16.798>
created_at = <Date 2021-07-20.21:57:36.437>
labels = ['type-bug', 'library', '3.11']
title = "Adopt binacii.a2b_base64's strict mode in base64.b64decode"
updated_at = <Date 2021-08-23.23:45:16.798>
user = 'https://github.com/idan22moral'

    bugs.python.org fields:

    activity = <Date 2021-08-23.23:45:16.798>
actor = 'gregory.p.smith'
assignee = 'none'
closed = True
closed_date = <Date 2021-08-23.23:45:16.798>
closer = 'gregory.p.smith'
components = ['Library (Lib)']
creation = <Date 2021-07-20.21:57:36.437>
creator = 'idan22moral'
dependencies = []
files = []
hgrepos = []
issue_num = 44690
keywords = ['patch']
message_count = 3.0
messages = ['397917', '400184', '400185']
nosy_count = 2.0
nosy_names = ['gregory.p.smith', 'idan22moral']
pr_nums = ['27272']
priority = 'normal'
resolution = 'fixed'
stage = 'commit review'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue44690'
versions = ['Python 3.11']
    @idan22moral
    Copy link
    Mannequin Author

    idan22moral mannequin commented Jul 20, 2021

    This is a follow-up PR to #68590.

    Currently, *base64.b64decode* uses a generic regex to validate *s* (when *validate* is true),
    which sometimes results in unexpected behavior and exception messages.

    Example:

    (1) base64.b64decode('ab==', validate=True) # b'i'
    (2) base64.b64decode('ab3==', validate=True) # b'i\xbd'
    (3) base64.b64decode('ab=3=', validate=True) # raises binascii.Error: Non-base64 digit found
    (4) base64.b64decode('ab==3', validate=True) # raises binascii.Error: Non-base64 digit found
    (5) base64.b64decode('ab===', validate=True) # raises binascii.Error: Non-base64 digit found
    (6) base64.b64decode('=ab==', validate=True) # raises binascii.Error: Non-base64 digit found

    The only strict-base64 valid example here is (1).
    (2), (4) and (5) should raise 'Excess data after padding',
    (3) should raise 'Discontinuous padding not allowed',
    and (6) should raise 'Leading padding not allowed'.

    To get this behavior, we can use the new (at the time of creating this PR) *binascii.a2b_base64* functionality of strict mode.

    I have one (not so big) concern - efficiency.
    I'm not that experienced with how fast regex-es are (in Python or in general) compared to the implementation of *binascii.a2b_base64* in C.
    So, I've no idea what would be the impact of migrating from regex pre-validation to input parsing.
    Let me know if you find it inefficient.

    -----

    Referenced issue (GH-24402): https://bugs.python.org/issue43086

    @idan22moral idan22moral mannequin added 3.11 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Jul 20, 2021
    @gpshead
    Copy link
    Member

    gpshead commented Aug 23, 2021

    I'm not worried about the regex vs binascii C implementation performance at all.

    @gpshead
    Copy link
    Member

    gpshead commented Aug 23, 2021

    New changeset fa6304a by Idan Moral in branch 'main':
    bpo-44690: Adopt binacii.a2b_base64's strict mode in base64.b64decode (GH-27272)
    fa6304a

    @gpshead gpshead closed this as completed Aug 23, 2021
    @gpshead gpshead closed this as completed Aug 23, 2021
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.11 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    1 participant
    @gpshead