I'm trying to run python from UID which doesn't have entry in passwd database. This fails with:

 Traceback (most recent call last):
File "/usr/share/python2.7/site.py", line 567, in <module>
File "/usr/share/python2.7/site.py", line 549, in main
File "/usr/share/python2.7/site.py", line 278, in addusersitepackages
File "/usr/share/python2.7/site.py", line 253, in getusersitepackages
File "/usr/share/python2.7/site.py", line 243, in getuserbase
File "/usr/share/python2.7/sysconfig.py", line 522, in get_config_var
File "/usr/share/python2.7/sysconfig.py", line 426, in get_config_vars
File "/usr/share/python2.7/sysconfig.py", line 184, in _getuserbase
File "/usr/share/python2.7/sysconfig.py", line 171, in joinuser
File "/usr/share/python2.7/posixpath.py", line 260, in expanduser
KeyError: 'getpwuid(): uid not found: 51'

This looks similar to a sporadic failure on the debian bigmem buildbot:

http://www.python.org/dev/buildbot/all/builders/AMD64%20debian%20bigmem%203.x/builds/168/steps/test/logs/stdio

    ERROR: test_expanduser (test.test_posixpath.PosixPathTest)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/var/tmpfs/martin.vonloewis/3.x.loewis-parallel2/build/Lib/test/test_posixpath.py", line 305, in test_expanduser
        home = pwd.getpwuid(os.getuid()).pw_dir
    KeyError: 'getpwuid(): uid not found: 5025'

I agree -- there should be a guard here that just doesn't add user site directories if the lookup fails.

> there should be a guard here that just doesn't add user site directories if the lookup fails.
Agreed too.  site is easy to fix (I can do it or review a patch), but we also need to fix sysconfig; I think the right thing to do would be not to have a config var named 'userbase' when run from a UID without passwd entry.  This implies that the functions listed under http://docs.python.org/library/sysconfig#installation-paths would also need to omit *user schemes.

I have started to work on this.

This patch fixes the issue but needs review. It is my first patch ever so be nice :)

Thanks you for your contribution!  I hope you’ll find fun bugs to make patches for.

Your patch looks too simplistic; in the discussion, we agreed that if the lookup fails, the site module should not add user site directories (see http://docs.python.org/library/site#site.USER_SITE or PEP 370), and your patch does not seem to do that.

Moreover, this bug has repercussions on the sysconfig module (see my previous message), which needs edits too.

BTW, I don’t know how we can test a patch for this bug; arekm, how do you run Python from a UID without passwd entry?

$  LC_ALL=C sudo -u '#9999' -g '#9999' /bin/sh -c 'unset HOME ; python -c ""'
Traceback (most recent call last):
  File "/usr/share/python2.7/site.py", line 567, in <module>
    main()
  File "/usr/share/python2.7/site.py", line 549, in main
    known_paths = addusersitepackages(known_paths)
  File "/usr/share/python2.7/site.py", line 278, in addusersitepackages
    user_site = getusersitepackages()
  File "/usr/share/python2.7/site.py", line 253, in getusersitepackages
    user_base = getuserbase() # this will also set USER_BASE
  File "/usr/share/python2.7/site.py", line 243, in getuserbase
    USER_BASE = get_config_var('userbase')
  File "/usr/share/python2.7/sysconfig.py", line 522, in get_config_var
    return get_config_vars().get(name)
  File "/usr/share/python2.7/sysconfig.py", line 426, in get_config_vars
    _CONFIG_VARS['userbase'] = _getuserbase()
  File "/usr/share/python2.7/sysconfig.py", line 184, in _getuserbase
    return env_base if env_base else joinuser("~", ".local")
  File "/usr/share/python2.7/sysconfig.py", line 171, in joinuser
    return os.path.expanduser(os.path.join(*args))
  File "/usr/share/python2.7/posixpath.py", line 260, in expanduser
    userhome = pwd.getpwuid(os.getuid()).pw_dir
KeyError: 'getpwuid(): uid not found: 9999'

See also the issue #10496.

I would suggest that this issue is the same as http://bugs.python.org/issue10496 like Victor pointed out and would say close it because it is a duplicate.

Indeed.

> This patch fixes the issue but needs review.
> It is my first patch ever so be nice :)

Python should not decide what is the default home directory. The issue is in the site module: I proposed nonexistent_user.patch in the issue #10496 to fix the issue in the site module instead.