classification
Title: xml.dom.minidom cannot parse ISO-2022-JP
Type: Stage: resolved
Components: XML Versions: Python 2.7
process
Status: closed Resolution: works for me
Dependencies: Superseder:
Assigned To: Nosy List: amaury.forgeotdarc, dcallagh, iritkatriel
Priority: normal Keywords:

Created on 2012-09-07 06:38 by dcallagh, last changed 2020-10-19 19:27 by iritkatriel. This issue is now closed.

Messages (4)
msg169974 - (view) Author: Dan Callaghan (dcallagh) Date: 2012-09-07 06:38
Python 2.7.3 (default, Jul 24 2012, 10:05:38) 
[GCC 4.7.0 20120507 (Red Hat 4.7.0-5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> c = u'\u65e5\u672c\u8a9e'
>>> import xml.dom.minidom

Encoded as UTF-8, everything is fine:

>>> xml.dom.minidom.parseString('<?xml version="1.0" encoding="UTF-8" ?><x>%s</x>' % c.encode('UTF-8'))
<xml.dom.minidom.Document instance at 0x7f310d27dcf8>

but not ISO-2022-JP:

>>> xml.dom.minidom.parseString('<?xml version="1.0" encoding="ISO-2022-JP" ?><x>%s</x>' % c.encode('ISO-2022-JP'))
Traceback (most recent call last):
  File "<stdin>", line 3, in <module>
  File "/usr/lib64/python2.7/site-packages/_xmlplus/dom/minidom.py", line 1925, in parseString
    return expatbuilder.parseString(string)
  File "/usr/lib64/python2.7/site-packages/_xmlplus/dom/expatbuilder.py", line 942, in parseString
    return builder.parseString(string)
  File "/usr/lib64/python2.7/site-packages/_xmlplus/dom/expatbuilder.py", line 223, in parseString
    parser.Parse(string, True)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 1, column 48

lxml can handle it fine though:

>>> import lxml.etree
>>> lxml.etree.fromstring('<?xml version="1.0" encoding="ISO-2022-JP" ?><x>%s</x>' % c.encode('ISO-2022-JP'))
<Element x at 0x7f310d284960>
>>> _.text == c
True
msg169982 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2012-09-07 09:22
This is similar to issue13612: pyexpat does not support multibytes encodings.
msg377715 - (view) Author: Irit Katriel (iritkatriel) * (Python triager) Date: 2020-09-30 18:06
I don't see this problem on 3.10. Is this still an issue or can this issue be closed?


Running Release|Win32 interpreter...
Python 3.10.0a0 (heads/bpo17490-dirty:00eb063b66, Sep 27 2020, 13:20:24) [MSC v.1916 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> c = u'\u65e5\u672c\u8a9e'
>>> import xml.dom.minidom
>>> xml.dom.minidom.parseString('<?xml version="1.0" encoding="UTF-8" ?><x>%s</x>' % c.encode('UTF-8'))
<xml.dom.minidom.Document object at 0x015FC9E8>
>>> xml.dom.minidom.parseString('<?xml version="1.0" encoding="ISO-2022-JP" ?><x>%s</x>' % c.encode('ISO-2022-JP'))
<xml.dom.minidom.Document object at 0x01493208>
>>>
msg378996 - (view) Author: Irit Katriel (iritkatriel) * (Python triager) Date: 2020-10-19 19:27
Closing - this now works for me on Python 3.8 and 3.10. It was fixed sometime in the last 8 years.
History
Date User Action Args
2020-10-19 19:27:05iritkatrielsetstatus: open -> closed
resolution: works for me
messages: + msg378996

stage: resolved
2020-09-30 18:06:32iritkatrielsetnosy: + iritkatriel
messages: + msg377715
2012-09-07 09:22:32amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg169982
2012-09-07 06:38:03dcallaghcreate