Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add resource.prlimit #60799

Closed
tiran opened this issue Dec 2, 2012 · 20 comments
Closed

Add resource.prlimit #60799

tiran opened this issue Dec 2, 2012 · 20 comments
Assignees
Labels
extension-modules C modules in the Modules dir type-feature A feature request or enhancement

Comments

@tiran
Copy link
Member

tiran commented Dec 2, 2012

BPO 16595
Nosy @pitrou, @vstinner, @giampaolo, @tiran, @vajrasky
Files
  • prlimit2.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/tiran'
    closed_at = <Date 2013-12-10.18:40:35.123>
    created_at = <Date 2012-12-02.11:14:41.848>
    labels = ['extension-modules', 'type-feature']
    title = 'Add resource.prlimit'
    updated_at = <Date 2013-12-10.18:40:35.121>
    user = 'https://github.com/tiran'

    bugs.python.org fields:

    activity = <Date 2013-12-10.18:40:35.121>
    actor = 'pitrou'
    assignee = 'christian.heimes'
    closed = True
    closed_date = <Date 2013-12-10.18:40:35.123>
    closer = 'pitrou'
    components = ['Extension Modules']
    creation = <Date 2012-12-02.11:14:41.848>
    creator = 'christian.heimes'
    dependencies = []
    files = ['30685']
    hgrepos = []
    issue_num = 16595
    keywords = ['patch']
    message_count = 20.0
    messages = ['176783', '176785', '191759', '192578', '193361', '193367', '200737', '200901', '200902', '200906', '200910', '201203', '201206', '201207', '201208', '201236', '203505', '205841', '205842', '205843']
    nosy_count = 8.0
    nosy_names = ['pitrou', 'vstinner', 'giampaolo.rodola', 'christian.heimes', 'Arfrever', 'neologix', 'python-dev', 'vajrasky']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue16595'
    versions = ['Python 3.4']

    @tiran
    Copy link
    Member Author

    tiran commented Dec 2, 2012

    Linux has an additional function to setrlimit and getrlimit which supports to query and/or modify resource limits of another process.

    http://www.kernel.org/doc/man-pages/online/pages/man2/getrlimit.2.html

    The patch implements resource.prlimit(pid, resource[, limits]) on all platforms that have a prlimit() function. I haven't included the regenerated configure script. You have to run autoreconf in order to test the new function.

    @tiran tiran self-assigned this Dec 2, 2012
    @tiran tiran added extension-modules C modules in the Modules dir type-feature A feature request or enhancement labels Dec 2, 2012
    @tiran
    Copy link
    Member Author

    tiran commented Dec 2, 2012

    prlimit() needs glibc 2.13+ (thanks Arfrever).

    @tiran
    Copy link
    Member Author

    tiran commented Jun 24, 2013

    Updated patch, now raises PermissionError on EPERM.

    @tiran
    Copy link
    Member Author

    tiran commented Jul 7, 2013

    Does anybody want to review the code before I commit it?

    @giampaolo
    Copy link
    Contributor

    From my original post which unfortunately was ignored :( :
    http://mail.python.org/pipermail/python-ideas/2012-June/015323.html
    ...for completeness perhaps it also makes sense to expose Linux-specific RLIMIT constants:

    RLIMIT_MSGQUEUE
    RLIMIT_NICE
    RLIMIT_RTPRIO
    RLIMIT_RTTIME
    RLIMIT_SIGPENDING

    @vajrasky
    Copy link
    Mannequin

    vajrasky mannequin commented Jul 19, 2013

    Fedora 18, this is the result after applying your patch and execute your unit test:

    [sky@localhost cpython]$ ./python Lib/test/test_resource.py
    test_args (main.ResourceTest) ... ok
    test_fsize_enforced (main.ResourceTest) ... ok
    test_fsize_ismax (main.ResourceTest) ... ok
    test_fsize_toobig (main.ResourceTest) ... ok
    test_getrusage (main.ResourceTest) ... ok
    test_prlimit (main.ResourceTest) ... ERROR
    test_setrusage_refcount (main.ResourceTest) ... ok

    ======================================================================
    ERROR: test_prlimit (main.ResourceTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "Lib/test/test_resource.py", line 131, in test_prlimit
        1, resource.RLIMIT_AS)
      File "/home/sky/Code/python/programming_language/cpython/Lib/unittest/case.py", line 614, in assertRaises
        return context.handle('assertRaises', callableObj, args, kwargs)
      File "/home/sky/Code/python/programming_language/cpython/Lib/unittest/case.py", line 150, in handle
        callable_obj(*args, **kwargs)
    PermissionError: [Errno 1] Operation not permitted

    Ran 7 tests in 0.990s

    FAILED (errors=1)
    Traceback (most recent call last):
      File "Lib/test/test_resource.py", line 143, in <module>
        test_main()
      File "Lib/test/test_resource.py", line 140, in test_main
        support.run_unittest(ResourceTest)
      File "/home/sky/Code/python/programming_language/cpython/Lib/test/support.py", line 1615, in run_unittest
        _run_suite(suite)
      File "/home/sky/Code/python/programming_language/cpython/Lib/test/support.py", line 1590, in _run_suite
        raise TestFailed(err)
    test.support.TestFailed: Traceback (most recent call last):
      File "Lib/test/test_resource.py", line 131, in test_prlimit
        1, resource.RLIMIT_AS)
      File "/home/sky/Code/python/programming_language/cpython/Lib/unittest/case.py", line 614, in assertRaises
        return context.handle('assertRaises', callableObj, args, kwargs)
      File "/home/sky/Code/python/programming_language/cpython/Lib/unittest/case.py", line 150, in handle
        callable_obj(*args, **kwargs)
    PermissionError: [Errno 1] Operation not permitted

    With sudo, I got this:

    [sky@localhost cpython]$ sudo ./python Lib/test/test_resource.py
    [sudo] password for sky:
    test_args (main.ResourceTest) ... ok
    test_fsize_enforced (main.ResourceTest) ... ok
    test_fsize_ismax (main.ResourceTest) ... ok
    test_fsize_toobig (main.ResourceTest) ... ok
    test_getrusage (main.ResourceTest) ... ok
    test_prlimit (main.ResourceTest) ... FAIL
    test_setrusage_refcount (main.ResourceTest) ... ok

    ======================================================================
    FAIL: test_prlimit (main.ResourceTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "Lib/test/test_resource.py", line 131, in test_prlimit
        1, resource.RLIMIT_AS)
    AssertionError: ValueError not raised by prlimit

    Ran 7 tests in 1.002s

    FAILED (failures=1)
    Traceback (most recent call last):
      File "Lib/test/test_resource.py", line 143, in <module>
        test_main()
      File "Lib/test/test_resource.py", line 140, in test_main
        support.run_unittest(ResourceTest)
      File "/home/sky/Code/python/programming_language/cpython/Lib/test/support.py", line 1615, in run_unittest
        _run_suite(suite)
      File "/home/sky/Code/python/programming_language/cpython/Lib/test/support.py", line 1590, in _run_suite
        raise TestFailed(err)
    test.support.TestFailed: Traceback (most recent call last):
      File "Lib/test/test_resource.py", line 131, in test_prlimit
        1, resource.RLIMIT_AS)
    AssertionError: ValueError not raised by prlimit

    @tiran
    Copy link
    Member Author

    tiran commented Oct 21, 2013

    Thanks for your tests. I may have to rework my test scenario a bit.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Oct 22, 2013

    New changeset 796c21e27a92 by Christian Heimes in branch 'default':
    Issue bpo-16595: Add prlimit() to resource module
    http://hg.python.org/cpython/rev/796c21e27a92

    @tiran
    Copy link
    Member Author

    tiran commented Oct 22, 2013

    Let's see how the buildbots respond. I may have to disable some tests for ancient Kernel versions.

    @pitrou
    Copy link
    Member

    pitrou commented Oct 22, 2013

    + self.assertRaises(PermissionError, resource.prlimit,
    + 1, resource.RLIMIT_AS)

    Please skip this when run as root.

    + self.assertEqual(resource.prlimit(0, resource.RLIMIT_AS), (-1, -1))

    What if it's not that value by default? Please fix the test to make it
    more robust.

    @tiran
    Copy link
    Member Author

    tiran commented Oct 22, 2013

    Roundup bot hangs. I have addressed your concerns in http://hg.python.org/cpython/rev/8c77117f41a9

    @neologix
    Copy link
    Mannequin

    neologix mannequin commented Oct 25, 2013

    The test is failing with ENOSYS on one of the buildbots:
    """
    ======================================================================
    ERROR: test_prlimit (test.test_resource.ResourceTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "/var/lib/buildslave/3.x.murray-gentoo/build/Lib/test/test_resource.py", line 148, in test_prlimit
        1, resource.RLIMIT_AS)
      File "/var/lib/buildslave/3.x.murray-gentoo/build/Lib/unittest/case.py", line 689, in assertRaises
        return context.handle('assertRaises', callableObj, args, kwargs)
      File "/var/lib/buildslave/3.x.murray-gentoo/build/Lib/unittest/case.py", line 158, in handle
        callable_obj(*args, **kwargs)
    OSError: [Errno 38] Function not implemented
    """

    I wonder if we could add a @support.ignore_enosys decorator...

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Oct 25, 2013

    New changeset 87d41a5a9077 by Christian Heimes in branch 'default':
    Issue bpo-16595: prlimit() needs Linux kernel 2.6.36+
    http://hg.python.org/cpython/rev/87d41a5a9077

    @tiran
    Copy link
    Member Author

    tiran commented Oct 25, 2013

    The buildbot is Linux-2.6.35-vs2.3.0.36.32-gentoo-i686-Intel-R-Core-TM-2_CPU_6600@_2.40GHz-with-gentoo-2.1 but prlimit() requires 2.6.36+. I didn't expect to see a combination of glibc with prlimit() and Kernel without prlimit(). According to man prlimit it's not suppose to fail with ENOSYS, too.

    @vstinner
    Copy link
    Member

    Or we should extend with supress(OSerror, errno=errno.ENOSYS): ... :-)

    (Just kidding, ignored tests must be marked as skipped.)

    @neologix
    Copy link
    Mannequin

    neologix mannequin commented Oct 25, 2013

    I didn't expect to see a combination of glibc with prlimit() and Kernel without prlimit(). According to man prlimit it's not suppose to fail with ENOSYS, too.

    Yeah, we've seen this several times on some buildbots.
    Note that actually, any syscall can fail with ENOSYS: the configure
    check just checks that the libc exposes the wrapper library, but when
    the libc wrapper makes the actual syscall, the kernel can return
    -ENOSYS if it doesn't implement the syscall.

    @tiran
    Copy link
    Member Author

    tiran commented Nov 20, 2013

    Tests are passing on all buildbots for quite some time now.

    @tiran tiran closed this as completed Nov 20, 2013
    @giampaolo
    Copy link
    Contributor

    Just received this report on psutil bug tracker:
    https://code.google.com/p/psutil/issues/detail?id=455
    It seems os.prlimit() is affected by the same problem:

    >>> import resource
    >>> resource.RLIM_INFINITY
    -1
    >>>

    @giampaolo giampaolo reopened this Dec 10, 2013
    @giampaolo
    Copy link
    Contributor

    s/os.prlimit/resource.prlimit

    @pitrou
    Copy link
    Member

    pitrou commented Dec 10, 2013

    How is that a problem? In any case, this shouldn't have anything to do with prlimit(), please open another issue.

    @pitrou pitrou closed this as completed Dec 10, 2013
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    extension-modules C modules in the Modules dir type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants