classification
Title: Parallel build race condition on AIX since python-2.7
Type: Stage:
Components: Build Versions: Python 3.7, Python 3.6, Python 3.5, Python 3.4, Python 3.2, Python 3.3, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: David.Edelsohn, ericvw, haubi, loewis
Priority: normal Keywords: patch

Created on 2013-11-07 16:04 by haubi, last changed 2017-03-16 13:09 by haubi.

Files
File name Uploaded Description Edit
python-tip-aix-parallel.patch haubi, 2013-11-07 16:04 Fix parallel build race condition on AIX review
issue19521-parallel-build-race-on-aix.patch haubi, 2014-06-04 06:35 patch for 3.4
aix-parallel-build-race-refresh.patch ericvw, 2016-11-27 14:26 Refreshed 2014 patch for tip of CPython review
Pull Requests
URL Status Linked Edit
PR 682 open haubi, 2017-03-16 13:09
Repositories containing patches
http://hg.code.sf.net/p/prefix-launcher/cpython#issue19521
Messages (7)
msg202357 - (view) Author: Michael Haubenwallner (haubi) * Date: 2013-11-07 16:04
Since python-3.2, there is a race condition building in parallel on AIX:

Consider these Makefile(.pre.in) rules:

$(BUILDPYTHON): ...
  $(LINKCC) ... $(LINKFORSHARED) ...

Modules/_testembed: ...
  $(LINKCC) ... $(LINKFORSHARED) ...

Modules/_freeze_importlib: ...
  $(LINKCC) ...

On AIX, the variables get these values:

LINKCC = $(srcdir)/Modules/makexp_aix Modules/python.exp ...
LINKFORSHARED = -Wl,-bE:Modules/python.exp ...

Now $(BUILDPYTHON) and Modules/_testembed may run in parallel, causing Modules/python.exp to be created by two instances of makexp_aix eventually running at the same time.

Attached patch fixes this problem for cpython tip (doubt supporting AIX 4.1 and earlier still is necessary).

Thank you!
msg202379 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2013-11-07 18:01
Wouldn't it be better if linking _testembed generated _testembed.exp instead of generating python.exp? I hope using $@.exp somehow could help. Hard-coding the name of the export file sounds like a flaw in the first place.
msg202407 - (view) Author: Michael Haubenwallner (haubi) * Date: 2013-11-08 07:14
I'm unsure about the real purpose of _testembed, but given the name it does make sense to me to export the same symbols as $(BUILDPYTHON), thus reusing python.exp.
msg202435 - (view) Author: David Edelsohn (David.Edelsohn) * Date: 2013-11-08 16:45
+1
msg219742 - (view) Author: Michael Haubenwallner (haubi) * Date: 2014-06-04 06:35
Patch including configure update now.
msg281827 - (view) Author: Eric N. Vander Weele (ericvw) * Date: 2016-11-27 14:26
I also have found this goes back since Python 2.7.

I have refreshed the patched for the tip of CPython.  What can I do to help push this forward?
msg281833 - (view) Author: Eric N. Vander Weele (ericvw) * Date: 2016-11-27 18:53
I may be able to simplify the build on AIX by removing ld_so_aix and python.exp entirely.  Would this be a preferred solution if I am able to get something working?  If so, should I create a separate issue to track the change?
History
Date User Action Args
2017-03-16 13:09:47haubisetpull_requests: + pull_request560
2016-11-27 18:53:54ericvwsetmessages: + msg281833
2016-11-27 14:26:10ericvwsetfiles: + aix-parallel-build-race-refresh.patch

messages: + msg281827
title: parallel build race condition on AIX since python-3.2 -> Parallel build race condition on AIX since python-2.7
2016-11-27 13:56:32ericvwsetnosy: + ericvw

versions: + Python 2.7, Python 3.6, Python 3.7
2014-06-06 11:08:26haubisethgrepos: + hgrepo251
2014-06-04 06:35:59haubisetfiles: + issue19521-parallel-build-race-on-aix.patch

messages: + msg219742
2014-06-04 06:33:35haubisethgrepos: - hgrepo248
2014-06-04 06:03:14haubisethgrepos: + hgrepo248
2013-11-08 16:45:49David.Edelsohnsetnosy: + David.Edelsohn
messages: + msg202435
2013-11-08 07:14:47haubisetmessages: + msg202407
2013-11-07 18:01:12loewissetnosy: + loewis
messages: + msg202379
2013-11-07 16:04:54haubicreate