Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

os.path.exists(os.devnull) regression on windows #45652

Closed
dkagedal mannequin opened this issue Oct 22, 2007 · 14 comments
Closed

os.path.exists(os.devnull) regression on windows #45652

dkagedal mannequin opened this issue Oct 22, 2007 · 14 comments
Labels
3.8 only security fixes 3.9 only security fixes stdlib Python modules in the Lib dir

Comments

@dkagedal
Copy link
Mannequin

dkagedal mannequin commented Oct 22, 2007

BPO 1311
Nosy @loewis, @facundobatista, @kdwyer, @zooba
PRs
  • [3.8] bpo-37834: Normalise handling of reparse points on Windows (GH-15231) #15370
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2008-02-23.21:19:36.246>
    created_at = <Date 2007-10-22.12:04:18.402>
    labels = ['3.8', 'library', '3.9']
    title = 'os.path.exists(os.devnull) regression on windows'
    updated_at = <Date 2019-08-21.22:52:45.584>
    user = 'https://bugs.python.org/dkagedal'

    bugs.python.org fields:

    activity = <Date 2019-08-21.22:52:45.584>
    actor = 'steve.dower'
    assignee = 'none'
    closed = True
    closed_date = <Date 2008-02-23.21:19:36.246>
    closer = 'akuchling'
    components = ['Library (Lib)']
    creation = <Date 2007-10-22.12:04:18.402>
    creator = 'd_kagedal'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 1311
    keywords = []
    message_count = 14.0
    messages = ['56644', '56645', '56652', '56674', '56684', '56692', '56733', '56734', '56735', '56795', '56873', '350122', '350124', '350126']
    nosy_count = 6.0
    nosy_names = ['loewis', 'facundobatista', 'ggenellina', 'd_kagedal', 'kdwyer', 'steve.dower']
    pr_nums = ['15370']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue1311'
    versions = ['Python 3.8', 'Python 3.9']

    @dkagedal
    Copy link
    Mannequin Author

    dkagedal mannequin commented Oct 22, 2007

    When moving from Python 2.4 to Python 2.5, my program stopped working on
    win32 because of a change in os.path.exists. I couldn't find any
    description of the change, so I can only assume it's a bug.

    The os.devnul variable contains the name of the "null file", which is
    "/dev/null" on posix, and "nul" on win32. As I understand it, "NUL" is a
    file that exists in every directory on win32.

    Opening the "nul" file with open("nul", "r") works fine, but
    os.path.exists("nul") returns False. In Python 2.4, it returns True.

    @dkagedal dkagedal mannequin added the stdlib Python modules in the Lib dir label Oct 22, 2007
    @dkagedal
    Copy link
    Mannequin Author

    dkagedal mannequin commented Oct 22, 2007

    It's called os.devnull, and nothing else.

    @dkagedal dkagedal mannequin changed the title os.path.exists(os.devnul) regression on windows os.path.exists(os.devnull) regression on windows Oct 22, 2007
    @facundobatista
    Copy link
    Member

    You migrated only of Python version, or also of windows installation?

    I checked Py25 and Py23 in my Win 2k, and in both I have the same behaviour:

    C:\Python23>python
    Python 2.3.5 (#62, Feb  8 2005, 16:23:02) [MSC v.1200 32 bit (Intel)] on
    win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import os
    >>> os.stat("nul")
    Traceback (most recent call last):
      File "<stdin>", line 1, in ?
    OSError: [Errno 22] Invalid argument: 'nul'
    >>> os.path.exists("nul")
    False
    >>>

    I checked the os.stat() function because the documentation says that if
    it gives an error, exists() will return False (and that this is how it's
    implemented).

    BTW, note that if you call to the GetFileAttributesEx() function of
    kernel32.dll by yourself, it will give error for the "NUL" file, showing
    that it actually does not exist.

    Because of this, I'd say that current behaviour of os.exists() is ok,
    but I want to know the answer of my question regarding your change
    before closing this as not-a-bug.

    Thanks!

    @dkagedal
    Copy link
    Mannequin Author

    dkagedal mannequin commented Oct 23, 2007

    I tried it on two different machines, and got two different answers:

    conan$ /cygdrive/c/Python25/python -c 'import os.path; print
    os.path.exists("nul")'
    False
    conan$ /cygdrive/c/Python24/python -c 'import os.path; print
    os.path.exists("nul")'
    False

    conan$ /cygdrive/c/Python24/python -c 'import os; print os.stat("nul")'
    Traceback (most recent call last):
      File "<string>", line 1, in ?
    OSError: [Errno 22] Invalid argument: 'nul'
    conan$ /cygdrive/c/Python25/python -c 'import os; print os.stat("nul")'
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    WindowsError: [Error 87] The parameter is incorrect: 'nul'

    titti$ /cygdrive/c/Python24/python -c 'import os.path; print
    os.path.exists("nul")'
    True
    titti$ /cygdrive/c/Python25/python -c 'import os.path; print
    os.path.exists("nul")'
    False

    titti$ /cygdrive/c/Python24/python -c 'import os; print os.stat("nul")'
    (33206, 0L, 3, 1, 0, 0, 0L, -1, -1, -1)
    titti$ /cygdrive/c/Python25/python -c 'import os; print
    os.stat("nul")'Traceback (most recent call last):
      File "<string>", line 1, in <module>
    WindowsError: [Error 87] The parameter is incorrect: 'nul'

    I ran it from a cygwin prompt, but the pythons are native.

    So you are correct that it doesn't work as I expected in Python 2.4
    either on the "conan" host.. On "titti", there is a difference in how
    os.path.exist works between 2.4 and 2.5. On "conan" there is actually
    also a difference, but only in the exception raised by os.stat.

    I don't know what the differences between these installation are.

    @kdwyer
    Copy link
    Mannequin

    kdwyer mannequin commented Oct 23, 2007

    I tried this under Python 2.3.3, 2.5.1 (native) and 2.3.4 (cygwin). The
    operating system is Windows 2000 SP4.

    C:\Python23>python
    Python 2.3.3 (#51, Dec 18 2003, 20:22:39) [MSC v.1200 32 bit (Intel)] on
    win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import os.path
    >>> print os.path.exists('nul')
    True
    
    C:\Python25>python
    Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit
    (Intel)] on
    win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import os.path
    >>> print os.path.exists('nul')
    False
    
    $ python
    Python 2.3.4 (#1, Jun 13 2004, 11:21:03)
    [GCC 3.3.1 (cygming special)] on cygwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import os.path
    >>> print os.path.exists('nul')
    True

    So there does seem to be a change in behaviour at 2.5.

    @kdwyer
    Copy link
    Mannequin

    kdwyer mannequin commented Oct 23, 2007

    Ok, it seems that Python 2.5 implements two new functions
    Py_GetFileAttributesExA and Py_GetFileAttributesExW in posixmodule.c
    within the #ifdef MS_WINDOWS block that may return
    ERROR_INVALID_PARAMETER to os.stat, which will percolate WindowsError up
    to os.exists():

    In both functions we find:

    static BOOL WINAPI
    Py_GetFileAttributesExA(LPCSTR pszFile, 
    		       GET_FILEEX_INFO_LEVELS level,
                           LPVOID pv)
    {
    	BOOL result;
    	LPWIN32_FILE_ATTRIBUTE_DATA pfad = pv;
    	/* First try to use the system's implementation, if that is
    	   available and either succeeds to gives an error other than
    	   that it isn't implemented. */
    	check_gfax();
    	if (gfaxa) {
    		result = gfaxa(pszFile, level, pv);
    		if (result || GetLastError() != ERROR_CALL_NOT_IMPLEMENTED)
    			return result;
    	}
    	/* It's either not present, or not implemented.
    	   Emulate using FindFirstFile. */
    	if (level != GetFileExInfoStandard) {
    		SetLastError(ERROR_INVALID_PARAMETER);
    		return FALSE;
    ...
    
    
    static BOOL WINAPI
    Py_GetFileAttributesExW(LPCWSTR pszFile, 
    		       GET_FILEEX_INFO_LEVELS level,
                           LPVOID pv)
    {
    	BOOL result;
    	LPWIN32_FILE_ATTRIBUTE_DATA pfad = pv;
    	/* First try to use the system's implementation, if that is
    	   available and either succeeds to gives an error other than
    	   that it isn't implemented. */
    	check_gfax();
    	if (gfaxa) {
    		result = gfaxw(pszFile, level, pv);
    		if (result || GetLastError() != ERROR_CALL_NOT_IMPLEMENTED)
    			return result;
    	}
    	/* It's either not present, or not implemented.
    	   Emulate using FindFirstFile. */
    	if (level != GetFileExInfoStandard) {
    		SetLastError(ERROR_INVALID_PARAMETER);
    		return FALSE;
    ...

    I'm neither a C nor a win32api programmer - can anyone explain the
    purpose of this code?

    @loewis
    Copy link
    Mannequin

    loewis mannequin commented Oct 25, 2007

    The purpose of Py_GetFileAttributesEx* is to wrap GetFileAttributesEx,
    on systems where it doesn't exist (Windows 95 in particular). If it
    doesn't exist, it is emulated; if it exists, it is directly called.

    @loewis
    Copy link
    Mannequin

    loewis mannequin commented Oct 25, 2007

    As Facundo found out, the behavior of os.path.exists is fairly
    irrelevant here, as that functions is trivial. What really matters is
    whether os.stat succeeds for NUL. Can those users for whom it succeeds
    please report what Windows versions they are using, and what precisely
    the stat result for NUL is (and perhaps also for CON, PRN, etc)?

    @loewis
    Copy link
    Mannequin

    loewis mannequin commented Oct 25, 2007

    Please disregard Cygwin Python for this discussion. It (probably) uses
    the stat implementation from cygwin1.dll, which may work differently
    from Cygwin release to Cygwin release.

    @facundobatista
    Copy link
    Member

    >>> import os.path
    >>> os.path.exists("con")
    False
    >>> os.path.exists("nul")
    False
    >>> os.path.exists("prn")
    False

    This is in Windows 2000 (5.00.2195) sp4, using *both* Python 2.3.5 and
    2.5.1, no cygwin.

    Personally, I'm +1 with Mark Hammond about this:

    I agree it is unfortunate that the behaviour has changed, 
    but these special names are broken enough on Windows that 
    rely on the kernel32 function and behaves like it says 
    seems the sanest thing to do.
    

    Taking in consideration that *now* it behaves equally in different
    windows systems, but not before, I think this issue should be closed as
    "invalid" (it's not a bug).

    I'll wait some days to get more behaviour responses, though.

    Regards,

    @ggenellina
    Copy link
    Mannequin

    ggenellina mannequin commented Oct 28, 2007

    All these tests on Windows XP SP4, executing os.stat("nul")

    Python 2.1 thru 2.4 raises:
    OSError: [Errno 22] Invalid argument: 'nul'

    Python 2.5 gives a different error:
    WindowsError: [Error 87] El parámetro no es correcto: 'nul'

    @zooba
    Copy link
    Member

    zooba commented Aug 21, 2019

    New changeset df2d4a6 by Steve Dower in branch 'master':
    bpo-37834: Normalise handling of reparse points on Windows (GH-15231)
    df2d4a6

    @zooba
    Copy link
    Member

    zooba commented Aug 21, 2019

    Well, I was trying not to resurrect this old issue, but looks like I did it accidentally.

    Hi there, old hands! We miss you :)

    We also figured out a new [l]stat() implementation on Windows that handles symlinks and junctions better, and also non-file types such as NUL. So I guess I'll mark this as resolved in Python 3.8

    @zooba zooba added 3.8 only security fixes 3.9 only security fixes and removed invalid labels Aug 21, 2019
    @zooba
    Copy link
    Member

    zooba commented Aug 21, 2019

    New changeset 9eb3d54 by Steve Dower in branch '3.8':
    bpo-37834: Normalise handling of reparse points on Windows (GH-15370)
    9eb3d54

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 only security fixes 3.9 only security fixes stdlib Python modules in the Lib dir
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants