classification
Title: marshal.load() reads entire remaining file instead of just next value
Type: behavior Stage: resolved
Components: IO Versions: Python 3.1
process
Status: closed Resolution: duplicate
Dependencies: Superseder: file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3
View: 12291
Assigned To: Nosy List: mattchaput, r.david.murray, weirdink13
Priority: normal Keywords:

Created on 2012-03-29 20:10 by mattchaput, last changed 2012-03-30 01:25 by r.david.murray. This issue is now closed.

Messages (4)
msg157093 - (view) Author: Matt Chaput (mattchaput) Date: 2012-03-29 20:10
In Python 3.2, if you write several values to a file with multiple calls to marshal.dump(), and then try to read them back, the first marshal.load() returns the first value, but reads to the end of the file, so subsequent calls to marshal.load() raise an EOFError.

E.g.:

  import marshal
  f = open("test", "wb")
  marshal.dump(("hello", 1), f)
  marshal.dump(("there", 2), f)
  marshal.dump(("friend", 3), f)
  f.close()
  f = open("test", "rb")
  print(marshal.load(f))  # ('hello', 1)
  print(marshal.load(f))  # ERROR

This page seems to indicate this was also a bug in Python 3.1: http://www.velocityreviews.com/forums/t728526-python-3-1-2-and-marshal.html
msg157096 - (view) Author: Daniel Swanson (weirdink13) Date: 2012-03-29 20:48
You are correct.
I got:
Python 3.1.2 (release31-maint, Dec  9 2011, 20:50:50) 
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import marshall
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named marshall
>>> import marshal
>>> f = open("test", "wb")
>>> marshal.dump(("hello", 1), f)
20
>>> marshal.dump(("there", 2), f)
20
>>> marshal.dump(("friend", 3), f)
21
>>> f.close()
>>> f = open("test", "rb")
>>> print(marshal.load(f))
('hello', 1)
>>> print(marshal.load(f))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
EOFError: EOF read where object expected
>>>
msg157099 - (view) Author: Daniel Swanson (weirdink13) Date: 2012-03-29 21:04
The previous test was on linux mint 10 (Julia) with python 3.1.2
here is the same test on windows XP with python 3.2.2

Python 3.2.2 (default, Sep  4 2011, 09:51:08) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import marshal
>>> f = open('t', 'wb')
>>> marshal.dump(('skd', 1), f)
18
>>> marshal.dump(('slkd', 2), f)
19
>>> marshal.dump('lkdss', 3), f)
SyntaxError: invalid syntax
>>> marshal.dump(('lskda', 3), f)
20
>>> f.close()
>>> f = open('t', 'rb')
>>> print(marshal.load(f))
('skd', 1)
>>> print(marshal.load(f))
('slkd', 2)
>>> print(marshal.load(f))
('lskda', 3)
>>> print(marshal.load(f))
Traceback (most recent call last):
  File "<pyshell#11>", line 1, in <module>
    print(marshal.load(f))
EOFError: EOF read where object expected
>>> 

As you can see, this problem appearently does not apply to 3.2.2
msg157117 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-03-30 01:25
This is a duplicate of issue 12291.  3.1 is in security-fix only mode.
History
Date User Action Args
2012-03-30 01:25:24r.david.murraysetstatus: open -> closed

superseder: file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

nosy: + r.david.murray
messages: + msg157117
resolution: duplicate
stage: resolved
2012-03-29 21:04:37weirdink13setmessages: + msg157099
versions: - Python 3.2
2012-03-29 20:48:12weirdink13setnosy: + weirdink13
messages: + msg157096
2012-03-29 20:10:45mattchaputcreate