When running pyperformance html5lib test, the application crashes with "ImportError: cannot import name 'Mapping' from 'collections' (/py3buildpath/lib/python3.9/collections/__init__.py)"

To reproduce:

1 - Build Python from source. I produced with git commit befa032d8869e0fab4732d910f3887642879d644 from cpython GitHub.

2 - Run pyperformance with: /py3buildpath/bin/pyperformance run --python=/py3buildpath/bin/python3 --venv venvpath -b html5lib -o output.json

3 - Immediate crash is seen with message:

#!/bin/sh
  File "/usr/local/src/pyperf-tot-no-venv/lib/python3.9/site-packages/html5lib/_tokenizer.py", line 16, in <module>
    from ._trie import Trie
  File "/usr/local/src/pyperf-tot-no-venv/lib/python3.9/site-packages/html5lib/_trie/__init__.py", line 3, in <module>
    from .py import Trie as PyTrie
  File "/usr/local/src/pyperf-tot-no-venv/lib/python3.9/site-packages/html5lib/_trie/py.py", line 6, in <module>
    from ._base import Trie as ABCTrie
  File "/usr/local/src/pyperf-tot-no-venv/lib/python3.9/site-packages/html5lib/_trie/_base.py", line 3, in <module>
    from collections import Mapping
ImportError: cannot import name 'Mapping' from 'collections' (/root/py3-tot-no/lib/python3.9/collections/__init__.py)
ERROR: Benchmark html5lib failed: Benchmark died
Traceback (most recent call last):
  File "/usr/local/src/pyperf-tot-no-venv/lib/python3.9/site-packages/pyperformance/run.py", line 132, in run_benchmarks
    bench = func(cmd_prefix, options)
  File "/usr/local/src/pyperf-tot-no-venv/lib/python3.9/site-packages/pyperformance/benchmarks/__init__.py", line 244, in BM_html5lib
    return run_perf_script(python, options, "html5lib")
  File "/usr/local/src/pyperf-tot-no-venv/lib/python3.9/site-packages/pyperformance/run.py", line 98, in run_perf_script
    run_command(cmd, hide_stderr=not options.verbose)
  File "/usr/local/src/pyperf-tot-no-venv/lib/python3.9/site-packages/pyperformance/run.py", line 66, in run_command
    raise RuntimeError("Benchmark died")
RuntimeError: Benchmark died

Forgot to include that this was built on an aarch64 (Ampere eMAG) system.

collections removed ABC access in 3.9. It seems you are using an older version of pip. Please update to the latest version which has the vendored html5lib package fixed.

Sorry, I misread the traceback itself as if it's from pip. It's from html5lib being incompatible with 3.9 as per the latest release on 1.0.1 (Dec 7, 2017) . There is an issue to request new release with the fix at https://github.com/html5lib/html5lib-python/issues/419 . Maybe you can try building html5lib from their master branch to be used in pyperformance. I would propose closing it as third party.

I fixed the issue by disabling temporarily the benchmark, until html5lib is fixed:
https://github.com/python/pyperformance/commit/8043c159215836fa733ee0815feecdd359e8852a

--

That's Python bug tracker. The pyperformance bug tracker is at:
https://github.com/python/pyperformance/issues

Your issue is that "import html5lib" fails on Python 3.9 because there is no html5lib release including the fix:
https://github.com/html5lib/html5lib-python/issues/419

The bug has already been fixed in html5lib.

One workaround would be to disable htmllib benchmark on Python 3.9 until https://github.com/html5lib/html5lib-python/issues/419 is fixed.

--

> collections removed ABC access in 3.9.

That's bpo-25988 with commit ef092fe9905f61ca27889092ca1248a11aa74498.

--

pip backported the html5lib patch:
https://github.com/pypa/pip/commit/ef7ca1472c1fdd085cffb8183b7ce8abbe9e2800#diff-2496ad1eedee846e323ed2916d6c2d24

The fix is included in pip since pip 19.3.1.

Note: The ensurepip module of the master branch of Python uses a bundled copy of pip 19.2.3.

Really appreciate all the commentary and references here. Thanks for that.

The resolution of disabling html5lib in pyperformance is good. I'll assume no html5lib benchmarking in pyperformance (master) until a version of html5lib is released compatible with 3.9.

Thanks again.

FYI: this affects tornado_http and django_template as well.

> FYI: this affects tornado_http and django_template as well.

Please report pyperformance issues at:
https://github.com/python/pyperformance/issues