classification
Title: AssertionError in Doc/includes/mp_benchmarks.py
Type: behavior Stage: patch review
Components: Extension Modules Versions: Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: jnoller Nosy List: amaury.forgeotdarc, barry, christian.heimes, jnoller, loewis, rhettinger
Priority: release blocker Keywords: needs review, patch

Created on 2008-11-28 11:17 by christian.heimes, last changed 2009-01-18 19:46 by jnoller. This issue is now closed.

Files
File name Uploaded Description Edit
mp_array.patch amaury.forgeotdarc, 2008-12-02 21:10
mp_array_2.patch amaury.forgeotdarc, 2008-12-02 22:20
Messages (24)
msg76525 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-11-28 11:17
./python Doc/includes/mp_benchmarks.py

       ######## testing Array("i", ..., lock=False)

Traceback (most recent call last):
  File "Doc/includes/mp_benchmarks.py", line 235, in <module>
    test()
  File "Doc/includes/mp_benchmarks.py", line 203, in test
    test_seqspeed(multiprocessing.Array('i', range(10), lock=False))
  File
"/home/heimes/dev/python/release26-maint/Lib/multiprocessing/__init__.py",
line 254, in Array
    return Array(typecode_or_type, size_or_initializer, **kwds)
  File
"/home/heimes/dev/python/release26-maint/Lib/multiprocessing/sharedctypes.py",
line 87, in Array
    assert hasattr(lock, 'acquire')
AssertionError

The assertion error occurs when using Python 2.6 and our backports to
2.4 and 2.5.
msg76526 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-11-28 11:25
The examples in 3.0 didn't work at at all because nobody did a 2to3 run
on them. See r67417: mp_benchmarks, mp_newtypes and mp_distribution are
still broken but the others are working properly. We should include the
examples in our unit test suite ...
msg76530 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2008-11-28 14:35
The 3.0 doc/example issue is in issue 3256

I plan on fixing all the doc/example issue this/next week.
msg76531 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-11-28 14:36
Are you able to fix the examples before 3.0.0 and 2.6.1 are released?
They are scheduled for Wednesday 3rd of December.
msg76532 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2008-11-28 14:38
Yes, I have a pending patch. I'll see if I can steal some time today
to check it in.

On Fri, Nov 28, 2008 at 9:36 AM, Christian Heimes
<report@bugs.python.org> wrote:
>
> Christian Heimes <lists@cheimes.de> added the comment:
>
> Are you able to fix the examples before 3.0.0 and 2.6.1 are released?
> They are scheduled for Wednesday 3rd of December.
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue4449>
> _______________________________________
>
msg76544 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2008-11-28 18:18
I guess you just 2to3'ed the examples
msg76656 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2008-11-30 19:42
Jesse, please apply so we can close this issue.
msg76725 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-12-02 06:06
This is a documentation bug, not a library bug, right? (unless someone
claims that the documentation is right and the library is wrong)

So: a) this seems to be a duplicate of issue 3256, and b) it's "just" a
documention bug. Not sure why this is marked as release blocker.
msg76742 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2008-12-02 14:12
This is not a doc bug - in actuality, the mp_benchmarks.py example is 
exposing an assertion error in sharedctypes.py.

The doc-related bugs Christian and I spoke about have been fixed, however 
the main issue for this (the assertion error) is still in the code, and I 
haven't had a chance to trace it down and fix it.
msg76773 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2008-12-02 20:16
Jesse, does this affect Python 3.0?  If not, then I will lower the
priority and release 2.6.1 without it (there's always 2.6.2 :).

If so, then we need to know if 1) it's really a release blocker; 2) what
the ETA on a fix is.  I would like to tag the 3.0 branch today.
msg76774 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2008-12-02 20:18
2.6.0 shipped with the assertion error. Unfortunately, I'm tapped out at 
the day job right now, so I won't have a fix prepped and tested in time.
msg76784 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-12-02 21:10
Here is a patch nonetheless. It makes the code match the the documentation:

http://docs.python.org/library/multiprocessing.html#multiprocessing.sharedctypes.Arra
y
"""
- If lock is True (the default) then a new lock object is created to synchronize 
access to the value.
- If lock is a Lock or RLock object then that will be used to synchronize access to 
the value.
- If lock is False then access to the returned object will not be automatically 
protected by a lock, so it will not necessarily be process-safe.
"""

I changed multiprocessing.sharedctypes.Array and multiprocessing. sharedctypes.Array
I had to change some tests: now "lock=None" is an error.
+ some markup fixes in the documentation.
msg76785 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-12-02 21:15
The patch looks fine. However I propose to replace the assert statement
with a proper check that raises a meaningful exception.

if not hasattr(lock, 'acquire'):
    raise AttributeError("'%r' has no method 'acquire'" % lock)
msg76788 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2008-12-02 21:36
+1 on Amaury's patch, however I wouldn't change the assert right now - Christian's suggestion does make sense to change globally post 3.0

Amaury, do you want to submit it?
msg76790 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2008-12-02 21:42
I don't much like the

lock is True
lock is False

idioms much, but I don' thave a better suggestion, and it would be nice
to fix this for the releases.
msg76792 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-12-02 21:48
Indeed, it seems that multiprocessing uses assertions in many places to 
handle errors in user code. This could be changed, but it is another 
task.

I can apply the patch, but what about 2.6? it's an incompatible API 
change, unless we choose to process "lock=None" the same way as 
"lock=False".
msg76793 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2008-12-02 21:53
Does the language guarantee that True and False are singletons (to
support the is-test for identity)?  Does this API port to Jython and
IronPython?  Is it a problem that 1 cannot be substituted for True?
msg76794 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-12-02 22:08
The one I know is pypy: bool(x) always return one of the two prebuilt singletons 
(doubletons?).

The docs explicitly mention "If lock is True...", "If lock is False...":
http://docs.python.org/library/multiprocessing.html#multiprocessing.sharedctypes.Value

I fear that testing the boolean value of the lock variable may have undesired effect; if 
even the main multiprocessing.RLock object has a _is_zero() method (that seems to return 
whether the lock is held or not), it is very possible that other implementations choose 
__nonzero__ for this.
msg76795 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2008-12-02 22:11
I've changed my mind based on the API change.  This should be discussed
on the mailing list and deferred until 2.6.2 at least.
msg76796 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-12-02 22:20
What about this other patch:
- lock=None has the same semantics as before
- lock=True is the same as lock=None (it failed before)
- lock=False return the object without the "synchronized" wrap (it 
failed before)
- no need to change previous tests
msg79623 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-01-11 20:17
I think Amaury's patch (mp_array_2.patch) is correct. If there is no
objection to its acceptance within the next two days, I think it should
be applied.
msg79759 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2009-01-13 18:30
I agree with Martin - if no one else gets to this before me, I should be 
able to submit it within the next day.
msg80072 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2009-01-18 02:35
FYI, there was a small issue in the patch - namely the only way to get 
the 
AttributeError to raise in the Value/Array creation is to not pass in a 
lock, True/None - the latter two get to a lock/RLock while the 
former will have the acquire method. If you pass in False, you 
immediately get the RawValue() object back.

I changed the test to reflect this:

    self.assertRaises(AttributeError, self.Value, 'i', 5, 
lock='navalue')
    self.assertRaises(AttributeError,
                      self.Array, 'i', range(10), lock='notalock')
msg80073 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2009-01-18 02:47
Checked into trunk in r68708 - I pinged benjamin to see how we're handling 
merges right now as this needs to go into py3k
History
Date User Action Args
2009-01-18 19:46:38jnollersetresolution: accepted -> fixed
2009-01-18 02:47:18jnollersetstatus: open -> closed
messages: + msg80073
2009-01-18 02:35:09jnollersetmessages: + msg80072
2009-01-18 02:33:43jnollersetmessages: - msg80071
2009-01-18 02:32:22jnollersetmessages: + msg80071
2009-01-13 18:30:26jnollersetmessages: + msg79759
2009-01-11 20:17:28loewissetresolution: accepted
messages: + msg79623
2008-12-28 22:44:45amaury.forgeotdarcsetkeywords: + needs review
stage: needs patch -> patch review
2008-12-20 02:42:25loewissetpriority: deferred blocker -> release blocker
2008-12-10 08:23:48loewissetpriority: release blocker -> deferred blocker
2008-12-08 17:03:05amaury.forgeotdarclinkissue3206 superseder
2008-12-06 23:29:36barrysetpriority: deferred blocker -> release blocker
2008-12-02 22:20:25amaury.forgeotdarcsetfiles: + mp_array_2.patch
messages: + msg76796
2008-12-02 22:11:19barrysetresolution: accepted -> (no value)
messages: + msg76795
2008-12-02 22:08:58amaury.forgeotdarcsetmessages: + msg76794
2008-12-02 21:53:33rhettingersetnosy: + rhettinger
messages: + msg76793
2008-12-02 21:48:51amaury.forgeotdarcsetmessages: + msg76792
2008-12-02 21:42:10barrysetresolution: accepted
messages: + msg76790
2008-12-02 21:36:43jnollersetmessages: + msg76788
2008-12-02 21:15:11christian.heimessetmessages: + msg76785
2008-12-02 21:10:33amaury.forgeotdarcsetfiles: + mp_array.patch
nosy: + amaury.forgeotdarc
messages: + msg76784
keywords: + patch
2008-12-02 20:18:27jnollersetpriority: release blocker -> deferred blocker
messages: + msg76774
2008-12-02 20:16:37barrysetmessages: + msg76773
2008-12-02 14:12:10jnollersetmessages: + msg76742
components: + Extension Modules, - Documentation
2008-12-02 06:06:31loewissetnosy: + loewis
messages: + msg76725
components: + Documentation, - Library (Lib)
2008-11-30 19:42:15barrysetnosy: + barry
messages: + msg76656
2008-11-28 18:18:31jnollersetmessages: + msg76544
2008-11-28 14:38:06jnollersetmessages: + msg76532
2008-11-28 14:36:31christian.heimessetmessages: + msg76531
2008-11-28 14:35:01jnollersetmessages: + msg76530
2008-11-28 11:25:30christian.heimessetpriority: high -> release blocker
messages: + msg76526
2008-11-28 11:17:04christian.heimescreate