I'm not sure this is python's bug or cygwin's bug, thread enabled python
crashes thread related tests on cygwin. (ex: test_exit on test_sys.py,
test_threading.py etc)

After some investigation, I found following workaround solves this crash.

Index: Modules/_ssl.c
===================================================================
--- Modules/_ssl.c      (revision 66562)
+++ Modules/_ssl.c      (working copy)
@@ -1580,7 +1580,7 @@

        /* Init OpenSSL */
        SSL_load_error_strings();
-#ifdef WITH_THREAD
+#if defined(WITH_THREAD) && !defined(__CYGWIN__)
        /* note that this will start threading if not already started */
        if (!_setup_ssl_threads()) {
                return;

So I applied following patch. (after reverted above workaround)


Index: Modules/_ssl.c
===================================================================
--- Modules/_ssl.c      (revision 66562)
+++ Modules/_ssl.c      (working copy)
@@ -1517,6 +1517,8 @@
           lock. They can be useful for debugging.
        */

+        printf("-------> %d (%u) %s %d: %ul\n", n, mode & CRYPTO_LOCK,
file, line, PyThread_get_thread_ident());
+
        if ((_ssl_locks == NULL) ||
            (n < 0) || ((unsigned)n >= _ssl_locks_count))
                return;

And this is result.

-------> 20 (1) mem_dbg.c 161: 6684680l
-------> 20 (0) mem_dbg.c 221: 6684680l
-------> 20 (1) mem_dbg.c 161: 6684680l
-------> 20 (0) mem_dbg.c 221: 6684680l
-------> 16 (1) ssl_ciph.c 273: 6684680l
-------> 16 (0) ssl_ciph.c 276: 6684680l
-------> 16 (1) ssl_ciph.c 277: 6684680l
-------> 20 (1) mem_dbg.c 161: 6684680l
-------> 20 (0) mem_dbg.c 221: 6684680l
-------> 20 (1) mem_dbg.c 161: 6684680l
-------> 20 (0) mem_dbg.c 221: 6684680l
-------> 16 (0) ssl_ciph.c 308: 6684680l
    started worker thread
    trying nonsensical thread id
    waiting for worker thread to get started
    verifying worker hasn't exited
    attempting to raise asynch exception in worker
    waiting for worker to say it caught the exception
-------> 1 (1) err.c 418: 7282896l
    all OK -- joining worker
-------> 1 (1) err.c 418: 7282896l
   6020 [unknown (0x650)] python 1156 _cygtls::handle_exceptions: Error
while du
mping state (probably corrupted stack)
Illegal instruction (core dumped)


Thread 7282896l tries to lock same object twice. I'm not familiar with
OpenSSL nor Python Thread, so I cannot fix this.

# Can callback function for CRYPTO_set_locking_callback() be called like
this? How does PyThread_allocate_lock behave in this situation? I don't
know.

I used OpenSSL0.9.8h installed via cygwin setup.

releast25-maint is fine probably because CRYPTO_set_locking_callback()
is not used in Modules/_ssl.c. I don't try configure --with-threads on
py3k, but probably same on trunk.

>Thread 7282896l tries to lock same object twice.
This was not cause of problem. I saw crash after one lock on another thread.

I could create the C code to reproduce crash. (reproduce.zip)
But strangely, I couldn't crash main.exe if it was built with
http://www.openssl.org/source/openssl-0.9.8h.tar.gz (same version)

I compiled openssl with "config -dCygwin" and "make". (I needed to fix
one broken link in Include dir though)

So it is a "Vendor OS problem", python is not involved.

Sorry, I noticed another bit. If main.exe is linked to libssl.dll.a and
libcrypto.dll.a it will crash, but linked to libssl.a and libcrypto.a it
won't crash. (I renamed *.dll.a temporary)

I'll try to build http://www.openssl.org/source/openssl-0.9.8h.tar.gz
with shared mode if possible.

>So it is a "Vendor OS problem", python is not involved.

Yes, possibly. Because other platform is not crashing on buildbot.

I think I have a beginning of an explanation:
libssl.dll implements a DllMain function, whose DLL_THREAD_DETACH event 
calls ERR_remove_state. 
At this time, the (posix) thread function has already exited; 
pthread::exit() was already called the pthread object has been deleted.

And the same (win32) thread will call sem_wait()... and maybe access 
freed resources.

Linking against the static library does not have this problem.

Thank you for great explanation! Probably you are right... I'll look
into the code.

Maybe I can fix this openssl bug with pthread_cleanup_push, but this is
openssl bug, we cannot fix it directly.

I propose to commit workaround in msg73649 for 2.6 release.

And after openssl will be fixed, change it to

#if defined(WITH_THREAD) && !(defined(__CYGWIN__) &&
OPENSSL_VERSION_NUMBER < ???)

That workaround leaves unused function warning. This patch is revised patch.

I'm not sure that reported issue is openssl bug.
Just tested a GCC(mingw) build of test case reproduce.zip with
openssl(0.9.8i) and "pthreads-w32". The test run without problems on
nt5.1(xp).

With cygwin, calling sem_wait() in the DLL_THREAD_DETACH section of a DllMain function 
can crash the program. 
See attached zip file, it contains two C files which only include pthread.h and 
semaphore.h (no python, no openssl). The resulting program crashes ~30% of the time.

If this pattern (in dll.c) is not allowed, it's a problem in the openssl code.
If it is allowed, it's a bug in cygwin's threads implementation.

We should really move this discussion to cygwin. This is no more a python issue.

OK, I'll close this entry, and I'll post message to cygwin ml about this
issue.
# I already posted it to openssl-dev, but there was no response.
http://www.nabble.com/Bug%3A-crash-on-cygwin-if-uses-CRYPTO_set_locking_callback-and-shared-library-to19699111.html#a19712690