classification
Title: urllib.urlopen crashes in a thread on Snow Leopard
Type: crash Stage: resolved
Components: macOS Versions: Python 2.6, Python 2.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: ronaldoussoren Nosy List: barry, emlyn, jweber, ronaldoussoren
Priority: release blocker Keywords: 26backport, patch

Created on 2009-09-07 00:00 by jweber, last changed 2009-09-20 10:58 by ronaldoussoren. This issue is now closed.

Files
File name Uploaded Description Edit
crash_log.txt jweber, 2009-09-07 00:00 Details from crash reporter dialog
urllib-crash.patch ronaldoussoren, 2009-09-16 21:42
Messages (6)
msg92340 - (view) Author: (jweber) Date: 2009-09-07 00:00
The following program works fine under Mac OS 10.5.x. But in 10.6, it 
crashes Python, and displays Apple's crash reporter dialog. I've tried 
it on Python 2.6 and 2.5, both the 64-bit and 32-bit versions.

The crash seems to happen any time urllib.urlopen is done in a thread. 
The same thing happens if I do it in a BaseHTTPServer.HTTPServer, using 
SocketServer.ThreadingMixIn.


#!/usr/bin/python
import threading, urllib

class MyThread (threading.Thread):
	def run(self):
		c = urllib.urlopen("http://www.google.com")

MyThread().start()
msg92651 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2009-09-15 19:27
It seems that CoreFoundation doesn't like being loaded on a secondairy 
thread:

#0  0x00007fff8301bb90 in __CFInitialize ()
#1  0x00007fff5fc0d5ce in 
__dyld__ZN16ImageLoaderMachO11doImageInitERKN11ImageLoader11LinkContextE 
()
#2  0x00007fff5fc0d607 in 
__dyld__ZN16ImageLoaderMachO16doInitializationERKN11ImageLoader11LinkCon
textE ()
#3  0x00007fff5fc0bcec in 
__dyld__ZN11ImageLoader23recursiveInitializationERKNS_11LinkContextEj ()
#4  0x00007fff5fc0bc9d in 
__dyld__ZN11ImageLoader23recursiveInitializationERKNS_11LinkContextEj ()
#5  0x00007fff5fc0bda6 in 
__dyld__ZN11ImageLoader15runInitializersERKNS_11LinkContextE ()
#6  0x00007fff5fc08fbb in __dyld_dlopen ()
#7  0x00007fff84902d40 in dlopen ()
#8  0x000000010070a0d2 in py_dl_open ()
#9  0x00000001000bb4ed in PyEval_EvalFrameEx ()
(More stack frames follow...)
msg92652 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2009-09-15 20:07
This probably means that the ctypes code in urllib.py needs to be ported 
to C, although we won't know if that helps until said C code is written 
:-(

Doing that would be a good idea anyway, while trying to create a 
workaround I noticed that the ctypes code is invalid anyway because 
there are not enough ctypes annotations when running in 64-bit mode 
(where sizeof(int) != sizeof(long))

In 32-bit mode you can avoid the crash by calling urllib.urlopen or 
urllib.proxy_bypass once on the main thread, but you have to use a 
qualified name ("localhost.localdomain", not "localhost").
msg92680 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2009-09-16 12:48
Agreed this should be a release blocker for 2.6.3
msg92721 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2009-09-16 21:42
The attached patch seems to fix the issue, but needs further testing. 

Warning: the patch is not entirely clean, the patch contains an 
unrelated change to setup.py.

The patch replaces some code that uses ctypes to read configuration data 
using the SystemConfiguration framework by a regular extension. 

With this patch the crasher goes away, I still have to do more testing 
and proper checking of the C code.

The patch is for the trunk, I expect it will cleanly port to 2.6. It 
will also need porting to 3.x later on.
msg92887 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2009-09-20 10:58
The attached patch was not 100% correct in the error handling in the 
set_proxies functions. That is fixed in the actually commit.

Committed as r74962 (trunk), r74963 (2.6).
History
Date User Action Args
2009-09-20 10:58:24ronaldoussorensetstatus: open -> closed
resolution: accepted -> fixed
messages: + msg92887

stage: needs patch -> resolved
2009-09-16 21:42:34ronaldoussorensetkeywords: + patch, 26backport
files: + urllib-crash.patch
messages: + msg92721
2009-09-16 12:48:14barrysetnosy: + barry
messages: + msg92680
2009-09-15 21:18:30ronaldoussorensetpriority: release blocker
2009-09-15 20:07:48ronaldoussorensetresolution: accepted
messages: + msg92652
stage: needs patch
2009-09-15 19:27:15ronaldoussorensetmessages: + msg92651
2009-09-09 11:50:42emlynsetnosy: + emlyn
2009-09-07 00:00:57jwebercreate