There appears to have been a regression introduced in the compileall library back with Python 3.5 as a result of commit f1a8df0ac98 .

The issues appears when compiling a hierarchy of directories with the 'ddir' parameter specified (either in the compile_dir() function or via the '-d' command line parameter).

Examining the code before the f1a8df0ac98 commit suggests the original intent for the 'ddir' parameter, when recursing into subdirectories, was to alter the root of the path displayed in exception traces. That means that the hierarchical directory structure of the reported source path in each compiled file was maintained, and only the root directory being compiled was replaced with ddir.

More recently two new parameters, 'stripdir' and 'prependdir', have been added with commit 8e7bb991de7 which seem to largely reimplement the missing behaviour in a different way.

The current behaviour does not seem very useful or expected. I think it would be helpful to restore the original behaviour and would be happy to contribute a PR to do so. Additionally it may be worth consulting the submitter of bpo-38112 whether the alternative arguments are still necessary once the regression is fixed, though this is optional and not needed to fix the core issue.



Here's a simple script to demonstrate the above:

mkdir -p library/a/a

cat <<'EOF' > library/__init__.py
def go():
  raise AssertionError('dummy')
EOF
cp library/__init__.py library/a/__init__.py
cp library/__init__.py library/a/a/__init__.py

python3 -m compileall -b -d /usr/lib library
rm library/**/*.py

python3 -c 'from library import go; go()'
python3 -c 'from library.a import go; go()'
python3 -c 'from library.a.a import go; go()'



And the resulting output which shows that each module is claiming to be raised from the same location:

Listing 'library'...
Compiling 'library/__init__.py'...
Listing 'library/a'...
Compiling 'library/a/__init__.py'...
Listing 'library/a/a'...
Compiling 'library/a/a/__init__.py'...

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/__init__.py", line 2, in go
AssertionError: dummy

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/__init__.py", line 2, in go
AssertionError: dummy

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/__init__.py", line 2, in go
AssertionError: dummy

Can this be closed? I can't reproduce.

$ python -c 'from library.a.a import go; go()'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/a/a/__init__.py", line 2, in go
    raise AssertionError('dummy')
AssertionError: dummy
$ python -c 'from library.a import go; go()'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/a/__init__.py", line 2, in go
    raise AssertionError('dummy')
AssertionError: dummy
$ python -c 'from library import go; go()'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/__init__.py", line 2, in go
    raise AssertionError('dummy')
AssertionError: dummy

Nevermind! I ran it with my system python binary, which is patched to against this.