classification
Title: test_wave: failures on PPC64 buildbot
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: David.Edelsohn, haypo, python-dev, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2013-11-17 22:26 by haypo, last changed 2013-11-21 15:21 by haypo. This issue is now closed.

Files
File name Uploaded Description Edit
wave_byteswap.patch serhiy.storchaka, 2013-11-17 23:12 review
wave_byteswap_2.patch serhiy.storchaka, 2013-11-18 08:15 review
Messages (11)
msg203217 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2013-11-17 22:26
Some test_wave are failing, but only on PPC64 (big endian, right?).

It may be related to issue #1575020.

http://buildbot.python.org/all/builders/PPC64%20PowerLinux%203.x/builds/1099/steps/test/logs/stdio


======================================================================
ERROR: test_unseekable_incompleted_write (test.test_wave.WavePCM16Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/test/audiotests.py", line 243, in test_unseekable_incompleted_write
    self.check_file(testfile, self.nframes + 1, self.frames)
  File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/test/audiotests.py", line 84, in check_file
    self.assertEqual(f.readframes(nframes), frames)
  File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/wave.py", line 257, in readframes
    data.fromfile(chunk.file.file, nitems)
EOFError: read() didn't return enough bytes

======================================================================
ERROR: test_unseekable_incompleted_write (test.test_wave.WavePCM32Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/test/audiotests.py", line 243, in test_unseekable_incompleted_write
    self.check_file(testfile, self.nframes + 1, self.frames)
  File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/test/audiotests.py", line 84, in check_file
    self.assertEqual(f.readframes(nframes), frames)
  File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/wave.py", line 257, in readframes
    data.fromfile(chunk.file.file, nitems)
EOFError: read() didn't return enough bytes

======================================================================
FAIL: test_write_array (test.test_wave.WavePCM16Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/test/audiotests.py", line 163, in test_write_array
    self.check_file(TESTFN, self.nframes, self.frames)
  File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/test/audiotests.py", line 83, in check_file
    self.assertEqual(f.getnframes(), nframes)
AssertionError: 96 != 48

======================================================================
FAIL: test_write_memoryview (test.test_wave.WavePCM16Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/test/audiotests.py", line 171, in test_write_memoryview
    self.check_file(TESTFN, self.nframes, self.frames)
  File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/test/audiotests.py", line 83, in check_file
    self.assertEqual(f.getnframes(), nframes)
AssertionError: 96 != 48

======================================================================
FAIL: test_write_array (test.test_wave.WavePCM32Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/test/audiotests.py", line 163, in test_write_array
    self.check_file(TESTFN, self.nframes, self.frames)
  File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/test/audiotests.py", line 83, in check_file
    self.assertEqual(f.getnframes(), nframes)
AssertionError: 192 != 48

======================================================================
FAIL: test_write_memoryview (test.test_wave.WavePCM32Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/test/audiotests.py", line 171, in test_write_memoryview
    self.check_file(TESTFN, self.nframes, self.frames)
  File "/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/test/audiotests.py", line 83, in check_file
    self.assertEqual(f.getnframes(), nframes)
AssertionError: 192 != 48
msg203227 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-11-17 23:12
Thank you for your report.

This patch should fix the issue.
msg203229 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2013-11-17 23:21
-    a = array.array('h', data)
+    a = array.array('h')
+    a.frombytes(data)

I don't understand why it would change anything. According to the doc, passing data to the construction is like calling array.frombytes():
http://docs.python.org/dev/library/array.html#array.array

If it behaves differently, it looks like a bug a in the array module. Am I wrong?
msg203251 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-11-18 08:15
array's constructor interprets its second memoryview argument as an iterable of integers.

>>> import array
>>> array.array('h', b'abcd')
array('h', [25185, 25699])
>>> array.array('h', memoryview(b'abcd'))
array('h', [97, 98, 99, 100])

array.frombytes() always interpret its argument as bytes-like object.

>>> a = array.array('h')
>>> a.frombytes(memoryview(b'abcd'))
>>> a
array('h', [25185, 25699])

First patch fixes only a half of the issue. test_unseekable_incompleted_write() still failed because array.fromfile() fails read incomplete data. Second patch also adds unittest.expectedFailure decorators for these tests.
msg203252 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2013-11-18 08:25
> array's constructor interprets its second memoryview argument as an iterable of integers.

Ok so, so your fix is correct.

> First patch fixes only a half of the issue. test_unseekable_incompleted_write() still failed because array.fromfile() fails read incomplete data.

Why the test succeeded on little endian?
msg203254 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-11-18 08:30
> Why the test succeeded on little endian?

Because array.fromfile() is used only to swap 16- and 32-bit samples on bigendian platform.

Perhaps we need the byteswap() function in the audioop module.
msg203257 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2013-11-18 08:38
> Because array.fromfile() is used only to swap 16- and 32-bit samples on bigendian platform.

If the file is truncated, why would the test suceed on little endian? The file doesn't have the same size in bytes? The test doesn't check the number of frames?
msg203272 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-11-18 10:05
> If the file is truncated, why would the test suceed on little endian? The
> file doesn't have the same size in bytes? The test doesn't check the number
> of frames?

Because this code is not used on little endian. On little endian a data is 
read by file's read() method.
msg203444 - (view) Author: David Edelsohn (David.Edelsohn) * Date: 2013-11-19 23:44
By the way, test_wave also fails on zLinux, which also is Big Endian.
msg203590 - (view) Author: Roundup Robot (python-dev) Date: 2013-11-21 09:05
New changeset 7b040bc289e8 by Serhiy Storchaka in branch '3.3':
Issue #19633: Fixed writing not compressed 16- and 32-bit wave files on
http://hg.python.org/cpython/rev/7b040bc289e8

New changeset 7cf7f19445ba by Serhiy Storchaka in branch 'default':
Issue #19633: Fixed writing not compressed 16- and 32-bit wave files on
http://hg.python.org/cpython/rev/7cf7f19445ba

New changeset 03a32ead9c7d by Serhiy Storchaka in branch '2.7':
Issue #19633: Fixed writing not compressed 16- and 32-bit wave files on
http://hg.python.org/cpython/rev/03a32ead9c7d
msg203638 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2013-11-21 15:21
"PPC64 PowerLinux 3.x" buildbot is green again! I'm closing the issue. I didn't check 2.7 and 3.3 buildbots.
History
Date User Action Args
2013-11-21 15:21:08hayposetstatus: open -> closed
resolution: fixed
messages: + msg203638
2013-11-21 09:05:19python-devsetnosy: + python-dev
messages: + msg203590
2013-11-19 23:44:51David.Edelsohnsetmessages: + msg203444
2013-11-18 10:05:23serhiy.storchakasetmessages: + msg203272
2013-11-18 08:38:15hayposetmessages: + msg203257
2013-11-18 08:30:08serhiy.storchakasetmessages: + msg203254
2013-11-18 08:25:00hayposetmessages: + msg203252
2013-11-18 08:15:57serhiy.storchakasetfiles: + wave_byteswap_2.patch

messages: + msg203251
2013-11-18 04:29:29David.Edelsohnsetnosy: + David.Edelsohn
2013-11-17 23:21:01hayposetmessages: + msg203229
2013-11-17 23:12:55serhiy.storchakasetfiles: + wave_byteswap.patch
messages: + msg203227

components: + Library (Lib), - Tests
keywords: + patch
type: behavior
stage: patch review
2013-11-17 22:26:57haypocreate