New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Python could crash while creating weakref for a given object #73533
Comments
We are using python 2.7.8 on Ubuntu 14.04 to host our services. In one of the crashes python interpreter got segmentation fault while initializing weakref for a given object. Please find snip of backtraces as given below. #0 0x00007f62aa86951a in clear_weakref (self=0x7f5a1ed17520) at Objects/weakrefobject.c:65 Have tried to root cause the issue and found that PyWeakref_NewProxy@Objects/weakrefobject.c creates new isntance of PyWeakReference struct and does not intialize wr_prev and wr_next of new isntance. These pointers can have garbage and point to random memory locations. As per comment in the code there could be a race while creating new instance and some other thread could have created weakref by the time current thread returns from new_weakref function. If it finds weakref created, current thread destroys instance created by itself and uses the one created by some other thread. Python should not crash while destroying the isntance created in the same interpreter function. As per my understanding, both wr_prev and wr_next of PyWeakReference instance should be initialized to NULL to avoid segfault. |
Can you reproduce the issue with a more recent version of Python 2.7? 2.7.8 is pretty old. |
Hi Christian Heimes, PFA. I have written a some code to simulate and test PyWeakReference struct instantion and then hit segfault based on flag passed-in to C code. Here I am trying to execute some of the operations from new_weakref and dealloc_weakref of Objects/weakrefobject.c to show that new isntance of PyWeakReference is not initialized properly. Have also checked latest 3.6 source and there is no difference in alloc and dealloc routines of 2.7.8 and 3.6.0. Have run test code on 2.7.8, 2.7.12+, 3.4m and 3.5m interpreters and got segfault in all runs. |
Hi Christian Heimes, Please ignore typos in the previous post. I have written some code to simulate and test PyWeakReference struct instantiation and then hit segfault based on the flag passed to C code. Here I am trying to execute some of the operations from new_weakref and dealloc_weakref of Objects/weakrefobject.c to show that new instance of PyWeakReference is not initialized properly and results in segfault. Have also checked python 3.6.0 source and I do not see any change in weakref alloc and dealloc routines of 2.7.8 and 3.6.0 versions. Have run test code on 2.7.8, 2.7.12+, 3.4m and 3.5m interpreters and got segfault in all runs. Please find the sample output as given below. ubuntu@ubuntu1610saida:~/weakref$ make build PYVERSION=2.7 ubuntu@ubuntu1610saida:~/weakref$ python test.py 0 0 ubuntu@ubuntu1610saida:~/weakref$ python test.py 0 1 ubuntu@ubuntu1610saida:~/weakref$ python test.py 1 0 ubuntu@ubuntu1610saida:~/weakref$ python test.py 1 1 ubuntu@ubuntu1610saida:~/weakref$ make clean ubuntu@ubuntu1610saida: |
After reading the code I could see the possibility. A weakref object gets two linkedlist pointers which are not initialized by new_weakref (actually they are initialized by insert_head or insert_after). But the weakref object is possible to be destroyed in [1] and [2]. So we are going to dereference two uninitialized pointers in clear_weakref and then crash. So simply initialize the two pointers to NULL in init_weakref could solve this problem? Are you willing to test Saida? [1] https://github.com/python/cpython/blob/master/Objects/weakrefobject.c#L770 |
Hi xiang, I have already patched our build environment with the fix and tested it locally. Although I cannot test it in production, have taken out the code of new_weakref, init_weakref and clear_weakref from weakrefobject.c. Changed init_weakref to initialize wr_prev and wr_next with NULL and verified that test program does crash after fix. |
Hmm, what's your test program? Would you mind show it? |
Please find the test program attached. Readme.txt has steps to comiple and run program. |
But your weakref_crash.c doesn't look correct to me. Your test.object type doesn't support weak references at all. How could you use GET_WEAKREFS_LISTPTR then? See https://docs.python.org/3/extending/newtypes.html#weakref-support. |
Saida, I changed your test program to use set instead of self created type (see attachment). I tested it under Py2.7 and it seems no crash happens. python test.py 1 1 python test.py 1 1 It's appreciated if you are willing to make more tests or even in your build environment. :-) |
Xiang, Sure, I will run it with other python versions and post the results. |
Ping. |
Although no feedback from Saida, but IMHO the problem is solved so I close it now. |
Hi Xiang, Sorry for the delay. I have not checked my inbox since last week. The proposed fix works for me. |
Thanks for your confirmation Saida! :-) |
Misc/NEWS
so that it is managed by towncrier #552Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: