classification
Title: xmlrpclib crash when PyXML installed - sgmlop is available
Type: crash Stage: test needed
Components: Extension Modules Versions: Python 3.0, Python 2.6, 3rd party
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: georg.brandl, mak
Priority: normal Keywords:

Created on 2007-08-13 08:15 by ldeller, last changed 2010-08-01 22:53 by georg.brandl. This issue is now closed.

Messages (9)
msg32637 - (view) Author: lplatypus (ldeller) * Date: 2007-08-13 08:15
The xmlrpclib module in the standard library will use a 3rd party C extension called "sgmlop" if it is present.

The last version of PyXML (0.8.4) includes this module, but it causes crashes with Python 2.5 due to the use of mismatched memory allocation/deallocation functions (PyObject_NEW and PyMem_DEL).

It is unlikely that sgmlop will be fixed, as PyXML is no longer maintained.  Therefore sgmlop support should be removed from xmlrpclib.

(In case you're wondering why anyone would install PyXML with Python 2.5 anyway:  there are still some 3rd party libraries which depend upon PyXML, such as ZSI and twisted).
msg32638 - (view) Author: Alan McIntyre (alanmcintyre) * (Python committer) Date: 2007-08-13 11:06
I'm assuming that stuff won't be removed from 2.5 because it's in maintenance, so should this be removed or changed to raise a deprecation warning in 2.6?

As an aside, how about removing references to _xmlrpclib (which appears to have been removed long ago) as well?  
msg32639 - (view) Author: lplatypus (ldeller) * Date: 2007-08-14 01:07
Choice of XML parser is an implementation detail of xmlrpclib not visible to users of the module.  This change would not affect the behaviour of xmlrpclib (other than to fix a crash introduced in Python 2.5).  Does this mean that a DeprecationWarning would not be necessary?  Does it also mean that the fix might qualify for the maintenance branch?

Adding a DeprecationWarning in 2.6 without removing use of sgmlop is pointless, because the DeprecationWarning would be followed by a process crash anyway.
msg32640 - (view) Author: Alan McIntyre (alanmcintyre) * (Python committer) Date: 2007-08-14 03:32
I tried out xmlrpclib on the trunk (2.6) with sgmlop-1.1.1 and it works just fine.  At the moment I don't have time to try out PyXML, but seeing that the most recent sgmlop works with xmlrpclib makes me lean towards not removing sgmlop support (not that I have a say about it, but still).

How did you install PyXML? If it wasn't from source or from an installer compiled for 2.5, that might be a problem. If PyXML installed from source or compiled for 2.5 still causes this problem, it could be that it needs to be updated to the current sgmlop.
msg32641 - (view) Author: lplatypus (ldeller) * Date: 2007-08-14 06:23
Yes the standalone sgmlop-1.1.1 looks fine: in its sgmlop.c I can see that matching allocator and deallocator functions are used.

I installed PyXML-0.8.4 from source ("python setup.py install" on Win32 which picked up the C compiler from MSVS7.1).  The cause of the problem is quite visible in the PyXML source code (see that PyObject_NEW and PyMem_DEL are used together):
    http://pyxml.cvs.sourceforge.net/pyxml/xml/extensions/sgmlop.c?view=markup

Interestingly PyXML-0.8.4 was released more recently than sgmlop-1.1.1.  I guess they weren't keeping in sync with each other.
msg32642 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2007-08-21 08:27
Fredrik, can you please comment? If not, unassign.
msg32643 - (view) Author: Fredrik Lundh (effbot) * (Python committer) Date: 2007-08-21 08:43
I don't really have an opinion here; the best solution would of course be to find someone that cares enough about PyXML to cut a bugfix release, it's probably easiest to just remove it (or disable, with a note that it can be re-enabled if you have a stable version of sgmlop).  I'm tempted to suggest removing SlowParser as well, but there might be some hackers on very small devices that rely on that one.

(Ideally, someone should sit down and rewrite the Unmarshaller to use xml.etree.cElementTree's iterparse function instead.  Contact me if you're interested).
msg57097 - (view) Author: Grzegorz Makarewicz (mak) Date: 2007-11-04 13:09
Minimalistic test crash (python 2.5 cvs, sgmlop cvs(pyxml)) - compiled
with msvc 2005, where after 10 loops ms-debugger is invoked:
data='''\
<?xml version="1.0"?>
<methodCall>
  <methodName>mws.ScannerLogout</methodName>
  <params>
    <param>
      <value>
        <i4>7</i4>
      </value>
    </param>
  </params>
</methodCall>
'''

import xmlrpclib

def main():
    i = 1
    while 1:
        print i
        params, method = xmlrpclib.loads(data)
        i+=1
main()
msg112394 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2010-08-01 22:53
Both PyXML and sgmlop are deprecated now, and support has been removed in xmlrpclib as of Python 2.7.  I think this can be closed.
History
Date User Action Args
2010-08-01 22:53:27georg.brandlsetstatus: open -> closed

nosy: + georg.brandl
messages: + msg112394

resolution: out of date
2009-04-07 04:05:05ajaksu2setstage: test needed
type: crash
versions: + Python 2.6, Python 3.0
2007-11-04 13:09:41maksetnosy: + mak, - loewis, effbot, alanmcintyre, ldeller
title: xmlrpclib crash when PyXML installed -> xmlrpclib crash when PyXML installed - sgmlop is available
messages: + msg57097
components: + Extension Modules, - Library (Lib)
versions: + 3rd party, - Python 2.5
2007-08-13 08:15:58ldellercreate