This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: test_io.py: codecs.IncrementalDecoder is sometimes None
Type: behavior Stage: needs patch
Components: Tests Versions: Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: amaury.forgeotdarc, benjamin.peterson, loewis, pitrou, srid, vstinner
Priority: normal Keywords: patch

Created on 2009-10-01 01:01 by srid, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
codecs_none.txt vstinner, 2010-04-27 13:54
regrtest_preload_ascii.patch vstinner, 2010-04-27 17:42
Messages (8)
msg93387 - (view) Author: Sridhar Ratnakumar (srid) Date: 2009-10-01 01:01
This test failure occurs on Windows XP (x86) with 2.6.3.rc1

======================================================================
ERROR: Test seek/tell using the StatefulIncrementalDecoder.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Python26\lib\test\test_io.py", line 1063, in testSeekAndTell
    testSeekAndTellWithData(input)
  File "C:\Python26\lib\test\test_io.py", line 1043, in
testSeekAndTellWithData
    decoded = f.read()
  File "C:\Python26\lib\io.py", line 1725, in read
    decoder = self._decoder or self._get_decoder()
  File "C:\Python26\lib\io.py", line 1514, in _get_decoder
    decoder = make_decoder(self._errors)
  File "C:\Python26\lib\test\test_io.py", line 636, in __init__
    codecs.IncrementalDecoder.__init__(self, errors)
AttributeError: 'NoneType' object has no attribute 'IncrementalDecoder'
msg93397 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-10-01 10:34
This is something that sometimes happens when running the test suite due
to module initialization or finalization oddities (I don't understand
the precise reasons myself). It isn't specific to test_io, I think.
msg104321 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-04-27 13:54
Same error here:
http://www.python.org/dev/buildbot/builders/AMD64 Ubuntu 2.6/builds/555/steps/test/logs/stdio

(codecs_none.txt is a copy of stdio)
msg104337 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-04-27 17:19
This issue is specific to Python 2.6. It can be reproduced with:

./python -E -tt ./Lib/test/regrtest.py -l -w test_pep263 test_operator test_asynchat test_zipimport_support test_pydoc test_code test_dis test_quopri test_doctest test_class test_sax test_fileio test_asyncore test_mailbox test_abstract_numbers test_uuid test_io test_enumerate test_property test_codecs

Or with this little test case:
----
import codecs, _codecs, sys, encodings

encoding = "cp1252"

codec = _codecs.lookup(encoding)

modname = 'encodings.%s' % encoding
__import__(modname)
del sys.modules[modname]
delattr(encodings, encoding)

encoder = codecs.getincrementalencoder(encoding)()
encoder.encode("", True)
----

module_dealloc() was modified in python trunk: the dictionary is only cleared if the reference counter is equal to 1.

/* If we are the only ones holding a reference, we can clear the dictionary. */
if (Py_REFCNT(m->md_dict) == 1) _PyModule_Clear((PyObject *)m);
msg104338 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-04-27 17:24
module-dealloc() was changed by #7140 by r75437: "imp.new_module does not function correctly if the module is returned from a function and used directly".

Can we backport the fix to 2.6? It would be complex to write a workaround in the tests.
msg104339 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-04-27 17:42
Here is a workaround. I didn't expected a short patch, but it's enough.
msg104346 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-04-27 18:28
> Can we backport the fix to 2.6?

It's too late to fix such subtle bug in module machinery, so I commited my workaround in regrtest.py: r80538. Wait for the buildbots before closing the issue.

test_io and/or test_codecs failed on buildbots:
 - AMD64 Ubuntu 2.6 (r80535)
 - x86 Tiger 2.6 (r80535)
 - x86 Windows7 2.6 (r80531)
msg104358 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-04-27 20:54
test_io and test_codecs didn't fail in the last build of x86 Tiger 2.6 and x86 Windows7 2.6. I suppose that the issue is now closed. Repopen the issue if it's not the case.
History
Date User Action Args
2022-04-11 14:56:53adminsetgithub: 51276
2010-04-27 20:54:20vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg104358
2010-04-27 18:28:02vstinnersetmessages: + msg104346
2010-04-27 17:42:42vstinnersetfiles: + regrtest_preload_ascii.patch
keywords: + patch
messages: + msg104339
2010-04-27 17:24:20vstinnersetmessages: + msg104338
2010-04-27 17:21:19pitrousetnosy: + loewis, amaury.forgeotdarc
2010-04-27 17:19:29vstinnersetmessages: + msg104337
versions: - Python 3.1, Python 2.7, Python 3.2
2010-04-27 13:54:07vstinnersetfiles: + codecs_none.txt
nosy: + vstinner
messages: + msg104321

2009-10-01 10:34:57pitrousetmessages: + msg93397
versions: + Python 3.1, Python 2.7, Python 3.2
2009-10-01 01:20:52r.david.murraysetpriority: normal
nosy: + pitrou, benjamin.peterson

type: behavior
stage: needs patch
2009-10-01 01:01:38sridcreate