Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test suite needs adjustments for Expat >=2.4.5 #90967

Closed
hartwork mannequin opened this issue Feb 20, 2022 · 10 comments
Closed

Test suite needs adjustments for Expat >=2.4.5 #90967

hartwork mannequin opened this issue Feb 20, 2022 · 10 comments
Labels
3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes 3.10 only security fixes 3.11 only security fixes topic-XML

Comments

@hartwork
Copy link
Mannequin

hartwork mannequin commented Feb 20, 2022

BPO 46811
Nosy @ned-deily, @ambv, @mgorny, @hartwork, @miss-islington
PRs
  • bpo-46811: Make test suite support Expat >=2.4.5 #31453
  • [3.9] bpo-46811: Make test suite support Expat >=2.4.5 (GH-31453) #31469
  • [3.8] bpo-46811: Make test suite support Expat >=2.4.5 (GH-31453) #31470
  • [3.7] bpo-46811: Make test suite support Expat >=2.4.5 (GH-31453) #31471
  • [3.10] bpo-46811: Make test suite support Expat >=2.4.5 (GH-31453) #31472
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2022-02-20.19:54:05.845>
labels = ['3.8', '3.9', '3.10', '3.11', 'expert-XML', '3.7']
title = 'Test suite needs adjustments for Expat >=2.4.5'
updated_at = <Date 2022-02-22.20:58:09.013>
user = 'https://github.com/hartwork'

    bugs.python.org fields:

    activity = <Date 2022-02-22.20:58:09.013>
actor = 'lukasz.langa'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['XML']
creation = <Date 2022-02-20.19:54:05.845>
creator = 'sping'
dependencies = []
files = []
hgrepos = []
issue_num = 46811
keywords = ['patch']
message_count = 6.0
messages = ['413605', '413654', '413658', '413659', '413669', '413743']
nosy_count = 5.0
nosy_names = ['ned.deily', 'lukasz.langa', 'mgorny', 'sping', 'miss-islington']
pr_nums = ['31453', '31469', '31470', '31471', '31472']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue46811'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10', 'Python 3.11']
    @hartwork
    Copy link
    Mannequin Author

    hartwork mannequin commented Feb 20, 2022

    It has been reported at https://bugs.python.org/issue46794#msg413587 that the current CPython test suite needs some adjustments for Expat >=2.4.5. Since that is somewhat separate from updating the bundled copy of Expat to >=2.4.6, I am creating this dedicated ticket. I pull request for discussion will follow shortly.

    @hartwork hartwork mannequin added 3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes 3.10 only security fixes 3.11 only security fixes topic-XML labels Feb 20, 2022
    @ambv
    Copy link
    Contributor

    ambv commented Feb 21, 2022

    New changeset 2cae938 by Sebastian Pipping in branch 'main':
    bpo-46811: Make test suite support Expat >=2.4.5 (GH-31453)
    2cae938

    @ambv
    Copy link
    Contributor

    ambv commented Feb 21, 2022

    New changeset 7da97f6 by Miss Islington (bot) in branch '3.10':
    bpo-46811: Make test suite support Expat >=2.4.5 (GH-31453) (GH-31472)
    7da97f6

    @ambv
    Copy link
    Contributor

    ambv commented Feb 21, 2022

    New changeset 336a916 by Miss Islington (bot) in branch '3.9':
    bpo-46811: Make test suite support Expat >=2.4.5 (GH-31453) (GH-31469)
    336a916

    @ned-deily
    Copy link
    Member

    New changeset d4f5bb9 by Miss Islington (bot) in branch '3.7':
    bpo-46811: Make test suite support Expat >=2.4.5 (GH-31453) (GH-31471)
    d4f5bb9

    @ambv
    Copy link
    Contributor

    ambv commented Feb 22, 2022

    New changeset fdfd7a9 by Łukasz Langa (Sebastian Pipping) in branch '3.8':
    bpo-46811: Make test suite support Expat >=2.4.5 (GH-31453)
    fdfd7a9

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @vfazio
    Copy link
    Contributor

    vfazio commented Apr 18, 2022

    If python is built to use the system libexpat, the version check used in the listed PRs will not necessarily be effective if the distribution ports patches back:

    # dpkg -l libexpat1
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name            Version         Architecture Description
+++-===============-===============-============-===============================
ii  libexpat1:arm64 2.2.6-2+deb10u4 arm64        XML parsing C library - runtime


    # python3 -c "import pyexpat; print(pyexpat.version_info); from xml.dom.minidom import parseString; parseString('<element xmlns:abc=\"http:abc.com/de f g/hi/j k\"><abc:foo /></element>')"
(2, 2, 6)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.10/xml/dom/minidom.py", line 1998, in parseString
    return expatbuilder.parseString(string)
  File "/usr/local/lib/python3.10/xml/dom/expatbuilder.py", line 925, in parseString
    return builder.parseString(string)
  File "/usr/local/lib/python3.10/xml/dom/expatbuilder.py", line 223, in parseString
    parser.Parse(string, True)
xml.parsers.expat.ExpatError: syntax error: line 1, column 0


    # dpkg -i libexpat1_2.2.6-2+deb10u1_arm64.deb
dpkg: warning: downgrading libexpat1:arm64 from 2.2.6-2+deb10u4 to 2.2.6-2+deb10u1
(Reading database ... 22507 files and directories currently installed.)
Preparing to unpack libexpat1_2.2.6-2+deb10u1_arm64.deb ...
Unpacking libexpat1:arm64 (2.2.6-2+deb10u1) over (2.2.6-2+deb10u4) ...
Setting up libexpat1:arm64 (2.2.6-2+deb10u1) ...
Processing triggers for libc-bin (2.28-10+deb10u1) ...


# dpkg -l libexpat1
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name            Version         Architecture Description
+++-===============-===============-============-===============================
ii  libexpat1:arm64 2.2.6-2+deb10u1 arm64        XML parsing C library - runtime

# python3 -c "import pyexpat; print(pyexpat.version_info); from xml.dom.minidom import parseString; parseString('<element xmlns:abc=\"http:abc.com/de f g/hi/j k\"><abc:foo /></element>')"
(2, 2, 6)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.10/xml/dom/minidom.py", line 1998, in parseString
    return expatbuilder.parseString(string)
  File "/usr/local/lib/python3.10/xml/dom/expatbuilder.py", line 925, in parseString
    return builder.parseString(string)
  File "/usr/local/lib/python3.10/xml/dom/expatbuilder.py", line 223, in parseString
    parser.Parse(string, True)
  File "/build/python/3.10.2/python-3.10.2/Modules/pyexpat.c", line 416, in StartElement
  File "/usr/local/lib/python3.10/xml/dom/expatbuilder.py", line 738, in start_element_handler
    uri, localname, prefix, qname = _parse_ns_name(self, name)
  File "/usr/local/lib/python3.10/xml/dom/expatbuilder.py", line 129, in _parse_ns_name
    raise ValueError("Unsupported syntax: spaces in URIs not supported: %r" % name)
ValueError: Unsupported syntax: spaces in URIs not supported: 'http:abc.com/de f g/hi/j k foo abc'

    @ambv
    Copy link
    Contributor

    ambv commented Apr 19, 2022

    @vfazio, system expat which isn't truly any upstream version but a series of cherry-picked patches on an old version, is not something upstream Python is able to support. In this case you should be using your distribution's Python to get an internally consistent state.

    @vfazio
    Copy link
    Contributor

    vfazio commented Apr 19, 2022

    @vfazio, system expat which isn't truly any upstream version but a series of cherry-picked patches on an old version, is not something upstream Python is able to support. In this case you should be using your distribution's Python to get an internally consistent state.

    @ambv Using a distribution's package isn't always feasible, especially if there's no port for the latest upstream version (Debian Bullseye does not provide 3.10.x) so many people will compile their own. A number of places document configuring --with-system-expat to take advantage of the system provided library to receive security updates vs using CPython's in-tree version.

    I'm just calling out that people may see these tests fail if they built Python to leverage the external libraries

    @hartwork
    Copy link
    Contributor

    With all the linked pull requests merged, should this issue be closed as fixed?

    @ambv ambv closed this as completed Feb 13, 2024
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes 3.10 only security fixes 3.11 only security fixes topic-XML
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    4 participants
    @ambv @hartwork @vfazio @ned-deily