Jitterbug-Id: 41
Submitted-By: brian@digicool.com
Date: Fri, 30 Jul 1999 10:10:49 -0400 (EDT)
Version: 1.5.2
OS: NT


It appears that win32 has a default limit of 64 descriptors that 
can be handed to the select() function. This is pretty low for
any serious async servers, and causes them to raise lots of errors
under very moderate loads.

We at DC ran into this using Medusa as a basis for ZServer, which
serves Zope sites. It turns out that you can actually add a define
when compiling the python15.dll for windows to bump the default
fd limit to a more reasonable level.

The approach _I_ took was to add the define: FD_SETSIZE=1024
to the preprocessor options in the MSVC project settings for
python15.dll, though I imagine you could also roll the define
into config.h or something (so long as it's defined before
windows.h or any of the select / socket include files are
referenced). It would make life much easier for win32 server
developers if this define could find its way into the next
official python release :^)

Thanks!


Brian Lloyd        brian@digicool.com
Software Engineer  540.371.6909              
Digital Creations  http://www.digicool.com 




====================================================================
Audit trail:
Fri Jul 30 10:43:41 1999	guido	moved from incoming to request

I am a little worried that adding it to config.h may have side-effects when Python is embedded in other projects with their own socket config (eg, Mozilla :-)

Now that socket and select are external .pyd modules, will it be sufficient to only add it to these extension modules?

Changed summary to say "Win32" instead of "NT", as this is a general Win32 issue.

Mark, did you email your question directly to Brian?  (This bug got moved over from Jitterbug, so he didn't see your note otherwise.)

I certainly agree Python can't go changing the MS default value in any way visible from Python.h (which #includes config.h).

Brian has agreed to help with a specific patch that will remain local to the Python build.  Dropping priority to reflect that it wont affect most users, and wont make 2.0.

Mark, Guido is agreeable to Python adding its own

#ifndef FD_SETSIZE
#define FD_SETSIZE ???
#endif

block.  If other people are doing the define-this-thing-on-the-cmdline trick, fine, such a block won't interfere with their beliefs.

So the primary remaining question is what "???" should be.  Is 1024 enough?  Someone else just bumped into the 64 limit (Python-Help), but didn't commit to a specific amount.

I recently raised this in the help desk for python. 
I am running into this in the WInsock arena and I really want to get over this hump. Can I get a Python dll with 1024 sockets in 1.5.2? I would be happy to test this in the Win2k arena for you (like that is a major gold star).

Really even 256 would be ok for me .. but 1024 is a spot more attractive (as I wouldn't have to keep watch on this all the time). Many thanks to Martin von Loewis and Tim Peters for thier help.

Tim -- it's time to commit on this. I recommend 512 as a compromise. <0.5 wink>

Reassigned from MarkH to me.  Unclear what the new value should be (nothing is free ...).

Boosted the Windows default to 512, in selectmodule.c rev 2.49.