classification
Title: telnetlib uses select instead of poll - limited to FD_SETSIZE fds
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: gregory.p.smith Nosy List: akintayo, eric.araujo, gregory.p.smith, jackdied, neologix, pitrou, python-dev, r.david.murray
Priority: normal Keywords:

Created on 2012-04-20 19:20 by gregory.p.smith, last changed 2013-12-11 02:26 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
telnet_expect_read_until_using_poll akintayo, 2012-05-31 21:06 Use select.poll() to implement telnet.read_until and telnet.expect review
Messages (14)
msg158874 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2012-04-20 19:20
telnetlib uses select.select.  This limits it to being able to work when file descriptors are still below FD_SETSIZE (often 1024) meaning it can't be used in some large programs today.

It should use poll.

(it is probably easy to fix this and the telnetlib EINTR issue1049450 at the same time)
msg158889 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2012-04-20 20:47
See also issue #10527, dealing with multiprocessing.
Note that this probably affects other modules besides telnetlib, so it might be interesting to find a way to factorize code (i.e. use poll() if available or fallback to select()).
msg158894 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-04-20 21:57
Out of curiosity, are you reporting this because you ran into it in your code?  I was not aware that telnet was still in real use.
msg158916 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-04-21 12:21
> Note that this probably affects other modules besides telnetlib, so it 
> might be interesting to find a way to factorize code (i.e. use poll()
> if available or fallback to select()).

asyncore might have been the answer.
msg159045 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-04-23 16:52
Éric: there are devices that still only allow telnet.  Older cisco routers (*many* of which are still in the field) are just one example I'm familiar with.  I don't currently have any tools that use telnetlib to talk to them, but I've got at least two I'd like to find time to write...and I can easily imagine this limitation coming up as a real issue at Google :)
msg162020 - (view) Author: Akintayo Holder (akintayo) Date: 2012-05-31 21:06
Hi,

telnet.read_until() and telnet.expect() will use select.poll() instead of select.select() on systems where poll() is available.

The patch also includes updates to test_telnetlib, the read_until() tests were changed to test the case where poll() is unavailable. We also added unit tests for expect(), these are a copy of the read_until() tests.

This patch is against 2.7.

Akintayo
msg162490 - (view) Author: Akintayo Holder (akintayo) Date: 2012-06-07 19:35
Is my approach ok or is the plan to follow  neologix's suggestion and make one fix that works for all the select.select issues.
msg162499 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2012-06-07 21:01
I think your approach is fine.

BTW for anyone who wants to chase the larger idea of dealing with all select.select use, take a look at the prototype for a select.select() implemented using poll.poll() that I just put in http://bugs.python.org/issue15032.
msg165568 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2012-07-16 05:17
looking at this now.  2.7 commit first; then i'll forward port to 3.2/3.3.
msg165569 - (view) Author: Roundup Robot (python-dev) Date: 2012-07-16 05:17
New changeset c53e3aacb816 by Gregory P. Smith in branch '2.7':
Fixes Issue #14635: telnetlib will use poll() rather than select() when possible
http://hg.python.org/cpython/rev/c53e3aacb816
msg165574 - (view) Author: Roundup Robot (python-dev) Date: 2012-07-16 06:45
New changeset de229dde486b by Gregory P. Smith in branch '3.2':
Fixes Issue #14635: telnetlib will use poll() rather than select() when possible
http://hg.python.org/cpython/rev/de229dde486b

New changeset 558e5ed678c3 by Gregory P. Smith in branch 'default':
Fixes Issue #14635: telnetlib will use poll() rather than select() when possible
http://hg.python.org/cpython/rev/558e5ed678c3
msg165650 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-07-16 19:27
This broke the buildbots without poll() (Windows).
msg165665 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2012-07-16 23:10
Those should all be fixed as of my most recent commit.
msg205877 - (view) Author: Roundup Robot (python-dev) Date: 2013-12-11 02:26
New changeset d61e8050b7d7 by Gregory P. Smith in branch '2.7':
Fixes Issue #17200: telnetlib's read_until and expect timeout was broken by the
http://hg.python.org/cpython/rev/d61e8050b7d7

New changeset 46186736e91c by Gregory P. Smith in branch '3.3':
Fixes Issue #17200: telnetlib's read_until and expect timeout was broken by the
http://hg.python.org/cpython/rev/46186736e91c
History
Date User Action Args
2013-12-11 02:26:26python-devsetmessages: + msg205877
2012-07-16 23:10:38gregory.p.smithsetmessages: + msg165665
2012-07-16 19:27:44pitrousetmessages: + msg165650
2012-07-16 06:58:05gregory.p.smithsetstatus: open -> closed
resolution: fixed
stage: resolved
2012-07-16 06:45:00python-devsetmessages: + msg165574
2012-07-16 05:17:54python-devsetnosy: + python-dev
messages: + msg165569
2012-07-16 05:17:16gregory.p.smithsetassignee: jackdied -> gregory.p.smith
messages: + msg165568
2012-06-07 21:01:26gregory.p.smithsetmessages: + msg162499
2012-06-07 19:35:15akintayosetmessages: + msg162490
2012-05-31 21:06:26akintayosetfiles: + telnet_expect_read_until_using_poll
nosy: + akintayo
messages: + msg162020

2012-04-23 16:52:48r.david.murraysetnosy: + r.david.murray
messages: + msg159045
2012-04-21 12:21:09pitrousetnosy: + pitrou
messages: + msg158916
2012-04-20 21:57:54eric.araujosetnosy: + eric.araujo
messages: + msg158894
2012-04-20 20:47:53neologixsetnosy: + neologix
messages: + msg158889
2012-04-20 19:20:23gregory.p.smithcreate