The xmlrpclib module in the standard library will use a 3rd party C extension called "sgmlop" if it is present.

The last version of PyXML (0.8.4) includes this module, but it causes crashes with Python 2.5 due to the use of mismatched memory allocation/deallocation functions (PyObject_NEW and PyMem_DEL).

It is unlikely that sgmlop will be fixed, as PyXML is no longer maintained. Therefore sgmlop support should be removed from xmlrpclib.

(In case you're wondering why anyone would install PyXML with Python 2.5 anyway: there are still some 3rd party libraries which depend upon PyXML, such as ZSI and twisted).

I'm assuming that stuff won't be removed from 2.5 because it's in maintenance, so should this be removed or changed to raise a deprecation warning in 2.6?

As an aside, how about removing references to _xmlrpclib (which appears to have been removed long ago) as well?

Choice of XML parser is an implementation detail of xmlrpclib not visible to users of the module. This change would not affect the behaviour of xmlrpclib (other than to fix a crash introduced in Python 2.5). Does this mean that a DeprecationWarning would not be necessary? Does it also mean that the fix might qualify for the maintenance branch?

Adding a DeprecationWarning in 2.6 without removing use of sgmlop is pointless, because the DeprecationWarning would be followed by a process crash anyway.

I tried out xmlrpclib on the trunk (2.6) with sgmlop-1.1.1 and it works just fine. At the moment I don't have time to try out PyXML, but seeing that the most recent sgmlop works with xmlrpclib makes me lean towards not removing sgmlop support (not that I have a say about it, but still).

How did you install PyXML? If it wasn't from source or from an installer compiled for 2.5, that might be a problem. If PyXML installed from source or compiled for 2.5 still causes this problem, it could be that it needs to be updated to the current sgmlop.

Yes the standalone sgmlop-1.1.1 looks fine: in its sgmlop.c I can see that matching allocator and deallocator functions are used.

I installed PyXML-0.8.4 from source ("python setup.py install" on Win32 which picked up the C compiler from MSVS7.1). The cause of the problem is quite visible in the PyXML source code (see that PyObject_NEW and PyMem_DEL are used together):
http://pyxml.cvs.sourceforge.net/pyxml/xml/extensions/sgmlop.c?view=markup

Interestingly PyXML-0.8.4 was released more recently than sgmlop-1.1.1. I guess they weren't keeping in sync with each other.

Fredrik, can you please comment? If not, unassign.

I don't really have an opinion here; the best solution would of course be to find someone that cares enough about PyXML to cut a bugfix release, it's probably easiest to just remove it (or disable, with a note that it can be re-enabled if you have a stable version of sgmlop). I'm tempted to suggest removing SlowParser as well, but there might be some hackers on very small devices that rely on that one.

(Ideally, someone should sit down and rewrite the Unmarshaller to use xml.etree.cElementTree's iterparse function instead. Contact me if you're interested).

Minimalistic test crash (python 2.5 cvs, sgmlop cvs(pyxml)) - compiled
with msvc 2005, where after 10 loops ms-debugger is invoked:
data='''\
<?xml version="1.0"?>
<methodCall>
<methodName>mws.ScannerLogout</methodName>
<params>
<param>
<value>
<i4>7</i4>
</value>
</param>
</params>
</methodCall>
'''

import xmlrpclib

def main():
    i = 1
    while 1:
        print i
        params, method = xmlrpclib.loads(data)
        i+=1
main()

Both PyXML and sgmlop are deprecated now, and support has been removed in xmlrpclib as of Python 2.7. I think this can be closed.