Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

webbrowser module import has heavy side effects #73831

Closed
serhiy-storchaka opened this issue Feb 25, 2017 · 10 comments
Closed

webbrowser module import has heavy side effects #73831

serhiy-storchaka opened this issue Feb 25, 2017 · 10 comments
Labels
3.7 (EOL) end of life performance Performance or resource usage stdlib Python modules in the Lib dir

Comments

@serhiy-storchaka
Copy link
Member

BPO 29645
Nosy @terryjreedy, @ncoghlan, @serhiy-storchaka, @mlouielu
PRs
  • bpo-29645: Speed up importing the webbrowser module. #484
  • [Do Not Merge] Sample of CPython life with blurb. #703
    		•
    Files
  • webbrowser-delayed-initialization.patch
  • webbrowser-delayed-initialization-2.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2017-03-08.15:20:37.722>
created_at = <Date 2017-02-25.09:26:58.434>
labels = ['3.7', 'library', 'performance']
title = 'webbrowser module import has heavy side effects'
updated_at = <Date 2017-03-24.22:40:47.558>
user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2017-03-24.22:40:47.558>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2017-03-08.15:20:37.722>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2017-02-25.09:26:58.434>
creator = 'serhiy.storchaka'
dependencies = []
files = ['46667', '46670']
hgrepos = []
issue_num = 29645
keywords = ['patch']
message_count = 10.0
messages = ['288551', '288561', '288562', '288563', '288566', '288585', '288587', '288589', '288930', '290260']
nosy_count = 4.0
nosy_names = ['terry.reedy', 'ncoghlan', 'serhiy.storchaka', 'louielu']
pr_nums = ['484', '703']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue29645'
versions = ['Python 3.7']
    @serhiy-storchaka
    Copy link
    Member Author

    import webbrowser has heavy side effects. It searches a number of executables. On X Window it also runs external program xdg-settings.

    Cold start:

    $ time ./python -c ''

    real 0m1.719s
    user 0m0.088s
    sys 0m0.036s

    $ time ./python -c 'import webbrowser'

    real 0m5.713s
    user 0m0.308s
    sys 0m0.196s

    Hot start:

    $ time ./python -c ''

    real 0m0.094s
    user 0m0.072s
    sys 0m0.020s

    $ time ./python -c 'import webbrowser'

    real 0m1.026s
    user 0m0.284s
    sys 0m0.100s

    @serhiy-storchaka serhiy-storchaka added 3.7 (EOL) end of life stdlib Python modules in the Lib dir performance Performance or resource usage labels Feb 25, 2017
    @mlouielu
    Copy link
    Mannequin

    mlouielu mannequin commented Feb 25, 2017

    What is the different of Cold start and Hot start? It that CPU speed or something else.

    In Linux 4.9.11 with i7-2k, I can't reproduce the significant real time you gave:

    # CPU gov powersave
    $ time ./python -c 'import webbrowser'
    0.16s user 0.02s system 93% cpu 0.200 total

    # CPU gov performance
    $ time ./python -c 'import webbrowser'
    0.08s user 0.00s system 82% cpu 0.093 total

    @serhiy-storchaka
    Copy link
    Member Author

    Following patch makes searching of all platform browsers delayed until it is needed. In addition it makes webbrowser.register() thread safe.

    Cold start:

    $ time ./python -c 'import webbrowser'

    real 0m2.851s
    user 0m0.224s
    sys 0m0.056s

    Hot start:

    $ time ./python -c 'import webbrowser'

    real 0m0.259s
    user 0m0.232s
    sys 0m0.024s

    @serhiy-storchaka
    Copy link
    Member Author

    What is the different of Cold start and Hot start?

    Disk caches are dropped before cold start.

    $ echo 3 | sudo tee /proc/sys/vm/drop_caches

    @ncoghlan
    Copy link
    Contributor

    Nice, this is much cleaner than the current approach!

    The one thing I would suggest is a new test case that:

    • asserts webbrowser._tryorder is None
    • asserts webbrowser._browsers is empty
    • calls webbrowser.get()
    • asserts webbrowser._tryorder is non-empty
    • asserts webbrowser._browsers is non-empty

    I wouldn't worry about explicitly testing the thread safety. That's just a normal double-checked locking pattern, so I think code review is sufficient to address that - the only way for it to break is for something to go horribly wrong in threading.RLock().

    @serhiy-storchaka
    Copy link
    Member Author

    Added tests.

    @ncoghlan
    Copy link
    Contributor

    Patch LGTM.

    Serhiy, did you want to take this as a chance to run through the new GitHub PR workflow?

    Current details are at https://docs.python.org/devguide/pullrequest.html

    @serhiy-storchaka
    Copy link
    Member Author

    The new GitHub PR workflow still looks cumbersome and unclear to me. It needs using a lot of git commands different for different branches (and some command are failed or don't work as I expected when I try to repeat sequences from the devguide). How to convert a patch to a pull request? How to get a list of added and modified files? What is the best way to revert all changes and changes in selected files? How to retrieve changes from pull request for local testing? Is there a way to edit otherpeople's pull request (add an entry in Misc/NEWS, etc) before merging it in the main repository? I wait until the devguide be more comprehensive.

    @terryjreedy
    Copy link
    Member

    This will help IDLE startup (though webbrowser should not be imported on windows, where os.startfile is used instead).

    This is a somewhat separate issue, but should the Windows code be modified for Win10 and Microsoft Edge?

    @serhiy-storchaka
    Copy link
    Member Author

    New changeset a7cba27 by Serhiy Storchaka in branch 'master':
    bpo-29645: Speed up importing the webbrowser module. (#484)
    a7cba27

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life performance Performance or resource usage stdlib Python modules in the Lib dir
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @ncoghlan @serhiy-storchaka @terryjreedy