Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

os.fspath is certain to crash when exception raised in __fspath__ #71699

Closed
zhangyangyu opened this issue Jul 14, 2016 · 22 comments
Closed

os.fspath is certain to crash when exception raised in __fspath__ #71699

zhangyangyu opened this issue Jul 14, 2016 · 22 comments
Assignees
Labels
extension-modules C modules in the Modules dir type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@zhangyangyu
Copy link
Member

BPO 27512
Nosy @brettcannon, @bitdancer, @ethanfurman, @serhiy-storchaka, @ztane, @zhangyangyu, @AraHaan
Files
  • fix_fspath_crash.patch
  • fix_fspath_crash_v2.patch
  • fix_fspath_crash_v3.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/brettcannon'
    closed_at = <Date 2016-07-15.18:27:45.271>
    created_at = <Date 2016-07-14.09:28:13.506>
    labels = ['extension-modules', 'type-crash']
    title = 'os.fspath is certain to crash when exception raised in __fspath__'
    updated_at = <Date 2016-08-16.19:53:11.705>
    user = 'https://github.com/zhangyangyu'

    bugs.python.org fields:

    activity = <Date 2016-08-16.19:53:11.705>
    actor = 'brett.cannon'
    assignee = 'brett.cannon'
    closed = True
    closed_date = <Date 2016-07-15.18:27:45.271>
    closer = 'brett.cannon'
    components = ['Extension Modules']
    creation = <Date 2016-07-14.09:28:13.506>
    creator = 'xiang.zhang'
    dependencies = []
    files = ['43713', '43717', '43723']
    hgrepos = []
    issue_num = 27512
    keywords = ['patch']
    message_count = 22.0
    messages = ['270386', '270387', '270390', '270392', '270393', '270396', '270397', '270398', '270402', '270408', '270416', '270417', '270420', '270428', '270430', '270491', '270492', '270495', '270497', '270498', '272877', '272878']
    nosy_count = 9.0
    nosy_names = ['brett.cannon', 'r.david.murray', 'SilentGhost', 'ethan.furman', 'python-dev', 'serhiy.storchaka', 'ztane', 'xiang.zhang', 'Decorater']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'crash'
    url = 'https://bugs.python.org/issue27512'
    versions = ['Python 3.6']

    @zhangyangyu
    Copy link
    Member Author

    When any exception is raised inside __fspath__, operations involving it like os.fspath, open are certain to crash.

    >>> import os
    >>> class Test:
    ...     def __fspath__(self):
    ...             raise RuntimeError
    ... 
    >>> os.fspath(Test())
    Segmentation fault (core dumped)

    Attach a patch to fix this.

    @zhangyangyu zhangyangyu added extension-modules C modules in the Modules dir type-crash A hard crash of the interpreter, possibly with a core dump labels Jul 14, 2016
    @zhangyangyu
    Copy link
    Member Author

    BTW, the code is also listed in PEP-519. So maybe that also needs to be fixed.

    @serhiy-storchaka
    Copy link
    Member

    Could you add tests?

    @zhangyangyu
    Copy link
    Member Author

    Ahh, adding tests is easy. But this seems to be just a careless omission, so maybe tests is not a must.

    @AraHaan
    Copy link
    Mannequin

    AraHaan mannequin commented Jul 14, 2016

    This does also happen in make_zip too as it does not known about the current changes. https://bpaste.net/show/ff826604a5f0

    @AraHaan
    Copy link
    Mannequin

    AraHaan mannequin commented Jul 14, 2016

    nevermind the above traceback from it is a issue with the code that makes it try to copy the redist file from the v14 C++ runtime.

    @ztane
    Copy link
    Mannequin

    ztane mannequin commented Jul 14, 2016

    I believe tests is that they should *especially* be in place for any previously found "careless omissions". If it has been done before, who is to say that it wouldn't happen again?

    @zhangyangyu
    Copy link
    Member Author

    I think such omissions are hard to predict and you won't know where is the omission until you encounter it. So if you don't know, how do you know what to put in the tests? And if it is encountered and fixed, such errors should not happen again.

    @serhiy-storchaka
    Copy link
    Member

    I think should be tests for following cases:

    1. __fspath__ doesn't exist;
    2. __fspath__ is not callable or calling __fspath__() raises an exception;
    3. the result of __fspath__() is not valid (wrong type).

    @zhangyangyu
    Copy link
    Member Author

    The test requests for the whole fspath are reasonable. I just don't want to add tests for these two lines.

    1. and 3) have already been tested. I add 2) and reorganize the existing tests a little.

    @ethanfurman
    Copy link
    Member

    I think such omissions are hard to predict and you won't know where is
    the omission until you encounter it. So if you don't know, how do you
    know what to put in the tests?

    True.

    And if it is encountered and fixed, such errors should not happen again.

    False.

    Such errors *will* happen again -- they are called regressions. In order to avoid those we add tests, and those tests serve several purposes:

    • confirm the problem has been fixed (not every bug is a core-dump)
    • confirm that changes in the future don't reintroduce the bug
    • communicate past difficulties and possible gotchas to new developers

    In short: add tests; patches are not complete without them.

    @ethanfurman
    Copy link
    Member

    I ... reorganize the existing tests a little.

    Please don't.

    Moving tests around makes it harder for reviewers to see what
    actually changed.

    Rewriting tests to do the same thing as before also takes more
    work from reviewers as we have to verify the new code does the
    same as the old code, and then wonder why the change was made.

    @zhangyangyu
    Copy link
    Member Author

    I have to explain for myself.

    First, I know tests *are* important. This is not the first time
    uploading patches here. I don't want to add tests for this case
    is because such codes: call something and test for failure appears
    all the places in the code base. This case is nothing special. For
    this reason, I think simply add the missing code back is enough.
    But then Serhiy presents his opinion that some test cases should be
    added, I think it's quite reasonable because it's testing the whole
    fspath behaviour, not just this case.

    Second, maybe I cannot agree with your opinions in your last message.
    I think changes are welcome if they *are* reasonable. The unreasonable
    aspects including what you have stated. For this patch, I don't know
    which part is unreasonable. Moving test_fsencode_fsdecode after
    test_pathlike makes the first 3 tests test for valid arguments.
    Extracting bad pathlike tests into a single test does not sounds like
    a bad idea. And it's small, right? So it's not hard to figure out
    what the change is made and what is actually changed.

    @brettcannon
    Copy link
    Member

    The reordering of the tests is unnecessary. Because you both changed the code *and* moved a test you have to be very careful to notice the change, otherwise it just looks like you cut-and-pasted the code to another location. Had you left the test method where it was and just simplified the code then it would be very obvious what you changed and reviewing it would be much easier.

    @brettcannon brettcannon self-assigned this Jul 14, 2016
    @zhangyangyu
    Copy link
    Member Author

    Thanks Brett, also others. Restore the moving around.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Jul 15, 2016

    New changeset 35bf83ff0828 by Brett Cannon in branch 'default':
    Issue bpo-27512: Don't segfault when os.fspath() calls an object whose
    https://hg.python.org/cpython/rev/35bf83ff0828

    @brettcannon
    Copy link
    Member

    Thanks for the patch, Xiang! Only thing I changed were braces around the if body in the C code and made the mock fspath() class raise an exception itself.

    @bitdancer bitdancer reopened this Jul 15, 2016
    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Jul 15, 2016

    New changeset bb7686a555cc by Brett Cannon in branch 'default':
    Fix a failing test introduced as part of issue bpo-27512
    https://hg.python.org/cpython/rev/bb7686a555cc

    @brettcannon
    Copy link
    Member

    Thanks for catching that, David.

    @SilentGhost
    Copy link
    Mannequin

    SilentGhost mannequin commented Aug 16, 2016

    Brett, Misc/NEWS entry needs a # before issue number.

    @brettcannon
    Copy link
    Member

    Thanks, I'll fix it at some point.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    extension-modules C modules in the Modules dir type-crash A hard crash of the interpreter, possibly with a core dump
    Projects
    None yet
    Development

    No branches or pull requests

    5 participants