classification
Title: Optimize and refactor readline().
Type: enhancement Stage:
Components: IO Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, serhiy.storchaka, shlomif
Priority: normal Keywords:

Created on 2019-06-22 14:11 by shlomif, last changed 2019-06-25 09:42 by shlomif.

Pull Requests
URL Status Linked Edit
PR 14306 open shlomif, 2019-06-22 14:10
Messages (5)
msg346280 - (view) Author: Shlomi Fish (shlomif) * Date: 2019-06-22 14:10
See:

https://github.com/python/cpython/pull/14306

===================

1. Merge two duplicate branches.

2. Extract some constants.

3. Convert to prefix-++ instead of suffix-++.

===========

Benchmarks:

before:

+ PATH=/home/shlomif/apps/python3/bin:/home/shlomif/apps/fcs/bin:/home/shlomif/progs/freecell/git/0fc-b/:/home/shlomif/progs/freecell/git/0fc-b//board_gen:/home/shlomif/bin:/home/shlomif/apps/perl/modules/bin:/home/shlomif/apps/perl/modules/local/bin:/home/shlomif/apps/neovim/bin:/home/shlomif/apps/fop/fop-20140425:/home/shlomif/apps/vim/bin:/home/shlomif/apps/golang/bin:/home/shlomif/.local/bin:/home/shlomif/.perl6/bin:/home/shlomif/.cargo/bin:/usr/local/bin:/usr/bin:/usr/lib64/qt5/bin:/usr/lib64/qt4/bin:/usr/games
+ export LD_LIBRARY_PATH=/home/shlomif/apps/python3/lib
+ LD_LIBRARY_PATH=/home/shlomif/apps/python3/lib
+ python3 collect-stats.py 0fc-log.txt
S	18474775
Int	617438

real	0m6.655s
user	0m6.599s
sys	0m0.054s
+ PATH=/home/shlomif/apps/python3/bin:/home/shlomif/apps/fcs/bin:/home/shlomif/progs/freecell/git/0fc-b/:/home/shlomif/progs/freecell/git/0fc-b//board_gen:/home/shlomif/bin:/home/shlomif/apps/perl/modules/bin:/home/shlomif/apps/perl/modules/local/bin:/home/shlomif/apps/neovim/bin:/home/shlomif/apps/fop/fop-20140425:/home/shlomif/apps/vim/bin:/home/shlomif/apps/golang/bin:/home/shlomif/.local/bin:/home/shlomif/.perl6/bin:/home/shlomif/.cargo/bin:/usr/local/bin:/usr/bin:/usr/lib64/qt5/bin:/usr/lib64/qt4/bin:/usr/games
+ export LD_LIBRARY_PATH=/home/shlomif/apps/python3/lib
+ LD_LIBRARY_PATH=/home/shlomif/apps/python3/lib
+ python3 collect-stats.py 0fc-log.txt
S	18474775
Int	617438

real	0m6.817s
user	0m6.772s
sys	0m0.045s
+ PATH=/home/shlomif/apps/python3/bin:/home/shlomif/apps/fcs/bin:/home/shlomif/progs/freecell/git/0fc-b/:/home/shlomif/progs/freecell/git/0fc-b//board_gen:/home/shlomif/bin:/home/shlomif/apps/perl/modules/bin:/home/shlomif/apps/perl/modules/local/bin:/home/shlomif/apps/neovim/bin:/home/shlomif/apps/fop/fop-20140425:/home/shlomif/apps/vim/bin:/home/shlomif/apps/golang/bin:/home/shlomif/.local/bin:/home/shlomif/.perl6/bin:/home/shlomif/.cargo/bin:/usr/local/bin:/usr/bin:/usr/lib64/qt5/bin:/usr/lib64/qt4/bin:/usr/games
+ export LD_LIBRARY_PATH=/home/shlomif/apps/python3/lib
+ LD_LIBRARY_PATH=/home/shlomif/apps/python3/lib
+ python3 collect-stats.py 0fc-log.txt
S	18474775
Int	617438

real	0m6.763s
user	0m6.706s
sys	0m0.057s

============

After:

+ PATH=/home/shlomif/apps/python3/bin:/home/shlomif/apps/fcs/bin:/home/shlomif/progs/freecell/git/0fc-b/:/home/shlomif/progs/freecell/git/0fc-b//board_gen:/home/shlomif/bin:/home/shlomif/apps/perl/modules/bin:/home/shlomif/apps/perl/modules/local/bin:/home/shlomif/apps/neovim/bin:/home/shlomif/apps/fop/fop-20140425:/home/shlomif/apps/vim/bin:/home/shlomif/apps/golang/bin:/home/shlomif/.local/bin:/home/shlomif/.perl6/bin:/home/shlomif/.cargo/bin:/usr/local/bin:/usr/bin:/usr/lib64/qt5/bin:/usr/lib64/qt4/bin:/usr/games
+ export LD_LIBRARY_PATH=/home/shlomif/apps/python3/lib
+ LD_LIBRARY_PATH=/home/shlomif/apps/python3/lib
+ python3 collect-stats.py 0fc-log.txt
S	18474775
Int	617438

real	0m6.651s
user	0m6.595s
sys	0m0.056s
+ PATH=/home/shlomif/apps/python3/bin:/home/shlomif/apps/fcs/bin:/home/shlomif/progs/freecell/git/0fc-b/:/home/shlomif/progs/freecell/git/0fc-b//board_gen:/home/shlomif/bin:/home/shlomif/apps/perl/modules/bin:/home/shlomif/apps/perl/modules/local/bin:/home/shlomif/apps/neovim/bin:/home/shlomif/apps/fop/fop-20140425:/home/shlomif/apps/vim/bin:/home/shlomif/apps/golang/bin:/home/shlomif/.local/bin:/home/shlomif/.perl6/bin:/home/shlomif/.cargo/bin:/usr/local/bin:/usr/bin:/usr/lib64/qt5/bin:/usr/lib64/qt4/bin:/usr/games
+ export LD_LIBRARY_PATH=/home/shlomif/apps/python3/lib
+ LD_LIBRARY_PATH=/home/shlomif/apps/python3/lib
+ python3 collect-stats.py 0fc-log.txt
S	18474775
Int	617438

real	0m6.625s
user	0m6.559s
sys	0m0.066s
+ PATH=/home/shlomif/apps/python3/bin:/home/shlomif/apps/fcs/bin:/home/shlomif/progs/freecell/git/0fc-b/:/home/shlomif/progs/freecell/git/0fc-b//board_gen:/home/shlomif/bin:/home/shlomif/apps/perl/modules/bin:/home/shlomif/apps/perl/modules/local/bin:/home/shlomif/apps/neovim/bin:/home/shlomif/apps/fop/fop-20140425:/home/shlomif/apps/vim/bin:/home/shlomif/apps/golang/bin:/home/shlomif/.local/bin:/home/shlomif/.perl6/bin:/home/shlomif/.cargo/bin:/usr/local/bin:/usr/bin:/usr/lib64/qt5/bin:/usr/lib64/qt4/bin:/usr/games
+ export LD_LIBRARY_PATH=/home/shlomif/apps/python3/lib
+ LD_LIBRARY_PATH=/home/shlomif/apps/python3/lib
+ python3 collect-stats.py 0fc-log.txt
S	18474775
Int	617438

real	0m6.601s
user	0m6.559s
sys	0m0.042s

=========

Based on https://github.com/shlomif/freecell-pro-0fc-deals .

<!--
Thanks for your contribution!
Please read this comment in its entirety. It's quite important.

# Pull Request title

It should be in the following format:

```
bpo-NNNN: Summary of the changes made
```

Where: bpo-NNNN refers to the issue number in the https://bugs.python.org.

Most PRs will require an issue number. Trivial changes, like fixing a typo, do not need an issue.

# Backport Pull Request title

If this is a backport PR (PR made against branches other than `master`),
please ensure that the PR title is in the following format:

```
[X.Y] <title from the original PR> (GH-NNNN)
```

Where: [X.Y] is the branch name, e.g. [3.6].

GH-NNNN refers to the PR number from `master`.

-->
msg346286 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-06-22 17:21
Could you please provide some mickrobenchmarks that shows the performance boost?
msg346385 - (view) Author: Shlomi Fish (shlomif) * Date: 2019-06-24 12:04
@Serhiy: I used https://github.com/shlomif/freecell-pro-0fc-deals/blob/master/collect-stats.py .
msg346487 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-06-25 07:05
What are results of benchmarking? Could you please provide a test data so I could reproduce benchmarking?
msg346499 - (view) Author: Shlomi Fish (shlomif) * Date: 2019-06-25 09:42
@Serhiy: here is the test data - https://www.shlomifish.org/Files/files/arcs/0fc-log--python-37371-issue.7z (shlomif[0fc]:$this$ sha256sum 0fc-log--python-37371-issue.7z 
aad93f103e255222ab8a06b9cc1c0930c6fc451ea148c46e098c74aa19aec021  0fc-log--python-37371-issue.7z
shlomif[0fc]:$this$ 
).

run it like "time python3 collect-stats.py 0fc-log.txt". I gave the results of a sample run in the main comment.
History
Date User Action Args
2019-06-25 09:42:46shlomifsetmessages: + msg346499
2019-06-25 07:05:16serhiy.storchakasetmessages: + msg346487
2019-06-24 12:04:55shlomifsetmessages: + msg346385
2019-06-22 17:21:57serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg346286
2019-06-22 15:49:15mangrisanosetnosy: + benjamin.peterson
2019-06-22 14:11:00shlomifcreate