classification
Title: idle: Errno 10035 a non-blocking socket operation could not be completed immediately
Type: behavior Stage: resolved
Components: IDLE, Windows Versions: Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: MJ, kristjan.jonsson, paul.moore, steve.dower, terry.reedy, tim.golden, zach.ware
Priority: normal Keywords: 3.4regression

Created on 2016-04-11 21:24 by MJ, last changed 2016-06-01 09:19 by kristjan.jonsson. This issue is now closed.

Files
File name Uploaded Description Edit
Skier MJ, 2016-04-11 21:24 Game
Messages (15)
msg263208 - (view) Author: Michael J (MJ) Date: 2016-04-11 21:24
So far I've got past the "bug in program" stage of debugging, but this came up:
IDLE internal error in runcode()
Traceback (most recent call last):
  File "C:\Python27\lib\idlelib\rpc.py", line 235, in asyncqueue
    self.putmessage((seq, request))
  File "C:\Python27\lib\idlelib\rpc.py", line 332, in putmessage
    n = self.sock.send(s[:BUFSIZE])
error: [Errno 10035] A non-blocking socket operation could not be completed immediately
I have no idea what a "socket" is so if you know please tell me!
msg263521 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2016-04-15 19:35
Michael, when you open an issue, you need to stay nosy so we can ask questions.  This is especially true when the issue involved 3rd party modules, like pygame, and even more, when personal files, like the images, are required.  In this case:

0. Which patch release of 2.7 are you running?  If not 2.7.11, please upgrade and retry.  There was a patch for Error 10035 in March 19, 2013, which would be about the time of 2.7.6 or so.
https://hg.python.org/cpython/rev/8ec39bfd1f01 #9090

1.Does your program run correctly when you run it directly with python from the console, with "python pathto/Skier"?

2. Were you running it from the IDLE editor?  If so, does it fail immediately? or after the game runs a bit?  Does it seem to always fail in exactly the same way, at the same time?

[A 'socket' is a class abstraction used to communicate between different processes.  It is usually used for processes on different machines, as with the browser process on your machine and a server process on another machine, but can be used for two processes on the same machine.]

--
This appears to be a socket issue, not an IDLE issue as such, just as #25332 was not a urllib issue.  This is the 10th issue found searching the tracker for 'Error 10035', but the first, I think, involving IDLE.  I know essentially nothing about the subject.
msg263662 - (view) Author: Michael J (MJ) Date: 2016-04-18 12:25
0. I have Python 2.7.5
1. I don't know what you mean.
2. It fails right when I run it.
IDLE stands for Python's Integrated DeveLopment Environment
I am running IDLE / Python on a Windows Vista, Service pack 2
msg263676 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2016-04-18 18:23
0. OK, 2.7.5 came out in 2013 May, after the 2.7 backport.  Kristján, you did the backport.  Do you have any idea about this?

1. Open a Command Prompt window, the command line console.  For Win 7, it can be found find it on the Start menu under Admin or something or started on the run line with 'cmd.exe'.  You should learn how to get the console.

On the command line, after the ...> prompt, enter "python patth/to/Skier.py".

Or, maybe, find Skier.py in Windows Explorer, and either double click or right-click and select "Run".  (I am not sure if this works with 2.7.5 on Vista.)
msg263689 - (view) Author: Michael J (MJ) Date: 2016-04-18 21:10
Is the command prompt itself admin?
I don't have access to the admin command prompt.
msg263697 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2016-04-18 22:13
No, admin privileges should not be needed to run Command Prompt.  Ditto for python if you did not already need same to run IDLE.
msg264104 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2016-04-24 09:44
Hi there.
I don't think this is in relation to issue #9090.
That one had to do with the internal mechanisms of doing blocking IO with timeout.  this is done internally by using non-blocking sockets and select(), and the backport dealt with some edge cases on windows where select() could falsely indicate that data were ready.

From what I can see in this error description, we are dealing with real non-blocking IO, i.e. an application is using select and non-blocking sockets.

It is possible that this windows edge case is now being elevated into the application code and whatever select() logic being used in rpc.py needs to be aware of it, or that for some reason this socket is supposed to be blocking, but isn't.

I'll have a quick look at idlelib and see if I can see anything.
msg264105 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2016-04-24 10:30
Caveat emptor:  I know nothing of IDLE, and I even suspect it to be dead or dying code.  Non the less, it could be patched.

I found this in the code:
def putmessage(self, message):
        self.debug("putmessage:%d:" % message[0])
        try:
            s = pickle.dumps(message)
        except pickle.PicklingError:
            print >>sys.__stderr__, "Cannot pickle:", repr(message)
            raise
        s = struct.pack("<i", len(s)) + s
        while len(s) > 0:
            try:
                r, w, x = select.select([], [self.sock], [])
                n = self.sock.send(s[:BUFSIZE])
            except (AttributeError, TypeError):
                raise IOError, "socket no longer exists"
            except socket.error:
                raise
            else:
                s = s[n:]


If the socket were non-blocking, this would be the place to add a handler to catch socket.error with errno=errno.EWOULDBLOCK

However, I can't see that this socket is non-blocking.  Perhaps I have some blindness, but the select calls seem to be redundant to me, I can't see any sock.setblocking(False) or sock.settimeout(0.0) being done anywhere.


Having said that, the following change can be made (which is the prudent way to use select/send anyway)
while len(s) > 0:
            try:
                while True:
                r, w, x = select.select([], [self.sock], [])
                try:
                    n = self.sock.send(s[:BUFSIZE])
                    break
                except socket.error as e:
                    import errno # should be done at the top
                    if e.errno != errno.EWOULDBLOCK:
                        raise
            except (AttributeError, TypeError):
                raise IOError, "socket no longer exists"
            except socket.error:
                raise
            else:
                s = s[n:]
msg264117 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2016-04-24 18:22
Kristján, thank you for the response.  The socket communication part of IDLE is pretty much a black box to me.  Just to clarify, you are saying that select polling is only needed for non-blocking sockets; sockets are blocking by default; and you see no override of the default.  Me neither.  It would be nice to know.  Idlelib.PyShell has the following lines: 
<445>        self.rpcclt.listening_sock.settimeout(10)
<561>            response = clt.pollresponse(self.active_seq, wait=0.05)
The wait is ultimately passed to rpc.py, line 353:
            r, w, x = select.select([self.sock.fileno()], [], [], wait)

(IDLE is very much in use, and I hope to modernize the 3.x version, and perhaps replace the sockets with non-blocking use of pipes.)

Michael: I cannot change 2.7.5 and I am not inclined to change 2.7.12+ until I know there is a problem with the current 2.7 release.  Numerous people are using IDLE with 2.7.11 (and 3.x) on Windows without problems.
msg264173 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2016-04-25 12:08
I think that the select.select calls there are a red herring, since I see no evidence that the rpc socket is ever put in non-blocking mode.
But the line
self.rpcclt.listening_sock.settimeout(10)
indicates that the socket is in timeout mode, and so, the error could be expected if it weren't for the backported fix for issue #9090

I'll have another look at that code and see if thera are any loopholes.

Also, Micahel could try commenting out this line in 
C:\Python27\Lib\idlelib\PyShell.py:
 self.rpcclt.listening_sock.settimeout(10)

and see if the problem goes away.
msg266786 - (view) Author: Michael J (MJ) Date: 2016-05-31 21:24
Kristján, when you say 'dead or dying code', it worries me a lot.  Can you tell me what you mean because I do NOT want to reinstall IDLE.
msg266787 - (view) Author: Michael J (MJ) Date: 2016-05-31 21:35
I also do not want to change something in the /lib folder because that is like going into the Windows Registry and 'fixing' a bunch of stuff.

I've made other games with PyGame and they generate errno 10035, too.
I've also installed the same version of Python, on a newer computer.
It is a Windows 10, ex-8.1
I haven't finished the program on my newer computer.
msg266788 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2016-05-31 21:42
If you're still using 2.7.5, we can't help you.  If you can reproduce the issue with 2.7.11 or 3.5.1, please reopen.
msg266790 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2016-05-31 23:54
Michael, IDLE is not dead, and I am working steadily to modernize it inside and out for 3.6.  There have even been improvements for 2.7 since 2.7.5.  For his work, Kristján has had no reason to keep up to date on IDLE's progress.
msg266811 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2016-06-01 09:19
Hi there, everyone.
I'm sorry for my rash remarks about the state of IDLE, I'm sure it is alive and well and its good to see that fine people like Terry are working on keeping it up to date.

Michael, please understand that python developers are volunteers and sometimes need help to fix things.  In this case, we have not been able to reproduce the problem, and are not sure what can be causing it.  My suggestion for you to modify code would be a step in identifying and diagnosing the problem.  Without such feedback it is hard to accomplish anything.

Cheers!
History
Date User Action Args
2016-06-01 09:19:53kristjan.jonssonsetmessages: + msg266811
2016-05-31 23:54:54terry.reedysetmessages: + msg266790
2016-05-31 21:42:30zach.waresetstatus: open -> closed
type: resource usage -> behavior
messages: + msg266788

resolution: out of date
stage: resolved
2016-05-31 21:35:57MJsetmessages: + msg266787
2016-05-31 21:24:55MJsetmessages: + msg266786
2016-04-25 12:08:35kristjan.jonssonsetmessages: + msg264173
2016-04-24 18:22:01terry.reedysetmessages: + msg264117
2016-04-24 10:30:23kristjan.jonssonsetmessages: + msg264105
2016-04-24 09:44:45kristjan.jonssonsetmessages: + msg264104
2016-04-18 22:13:01terry.reedysetmessages: + msg263697
2016-04-18 21:10:02MJsetmessages: + msg263689
2016-04-18 18:23:10terry.reedysetkeywords: + 3.4regression
nosy: + kristjan.jonsson
messages: + msg263676

2016-04-18 12:25:18MJsetmessages: + msg263662
2016-04-15 19:35:12terry.reedysetnosy: + terry.reedy, MJ
messages: + msg263521
2016-04-11 21:26:27vstinnersetnosy: + paul.moore, tim.golden, zach.ware, steve.dower

components: + IDLE
title: Errno 10035 a non-blocking socket operation could not be completed immediately -> idle: Errno 10035 a non-blocking socket operation could not be completed immediately
2016-04-11 21:25:47MJsetnosy: - paul.moore, tim.golden, zach.ware, steve.dower, MJ
-> (no value)
2016-04-11 21:24:19MJcreate