classification
Title: identify cross builds by a more generic environment setting.
Type: behavior Stage:
Components: Cross-Build Versions: Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: doko Nosy List: Alex.Willmer, doko, vstinner, xdegaye, yan12125, zach.ware
Priority: normal Keywords:

Created on 2016-09-13 10:53 by doko, last changed 2016-09-14 13:27 by vstinner.

Messages (9)
msg276244 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2016-09-13 10:53
I don't like the _PYTHON_SYSCONFIGDATA_NAME hack introduced in #28046.  This should be based on the target, not on a name for just a particular file.  Plus the abi flags should not be set by hand, you'll get in trouble at least for naming extensions, or building extensions for the wrong abi flags. This was an explicit decision when I first started adding the cross-build changes.

So proposing instead to base a cross build on a _PYTHON_MULTIARCH environment variable and deriving the sysconfigdata name on that.

Working on a patch for that.
msg276279 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-09-13 14:40
> Plus the abi flags should not be set by hand, you'll get in trouble at least for naming extensions, or building extensions for the wrong abi flags. This was an explicit decision when I first started adding the cross-build changes.

What do you mean ? The abi flags are set by configure and not by hand.
In which issue is this explicit decision taken, and where is it motivated ?

> So proposing instead to base a cross build on a _PYTHON_MULTIARCH environment variable and deriving the sysconfigdata name on that.

The _PYTHON_SYSCONFIGDATA_NAME environment variable as the merit to document its purpose with its name. The CPython cross-compilation system is difficult enough to understand that this is a significant advantage.  What are the advantages of introducing this new _PYTHON_MULTIARCH environment variable instead, where else would it be used ?
msg276342 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2016-09-13 20:55
On 13.09.2016 16:40, Xavier de Gaye wrote:
> 
> Xavier de Gaye added the comment:
> 
>> Plus the abi flags should not be set by hand, you'll get in trouble at least for naming extensions, or building extensions for the wrong abi flags. This was an explicit decision when I first started adding the cross-build changes.
> 
> What do you mean ? The abi flags are set by configure and not by hand.
> In which issue is this explicit decision taken, and where is it motivated ?

please try to build extension modules with mismatching abi flags (pydebug is the
relevant one). Post your results for both mismatch cases.

>> So proposing instead to base a cross build on a _PYTHON_MULTIARCH environment variable and deriving the sysconfigdata name on that.
> 
> The _PYTHON_SYSCONFIGDATA_NAME environment variable as the merit to document its purpose with its name. The CPython cross-compilation system is difficult enough to understand that this is a significant advantage.  What are the advantages of introducing this new _PYTHON_MULTIARCH environment variable instead, where else would it be used ?

No, at least it was not difficult when it was created. How did you make it more
difficult?

If we can derive the name, we should do it, not having more special environment
variables when we find the need for it.
msg276349 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-09-13 21:43
You do not answer my questions and resort to sarcasm instead: "How did you make it more difficult?". I understand that you may be upset because issue 23968 has been dismantled by the recent decision of removing the platdir files, but please keep exchanges on the bug tracker strictly on a technical level.

I am strongly opposed to this change for the reasons given in my first post.  Unless you can explain why this change is needed, please close this issue as rejected.
msg276351 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2016-09-13 21:50
On 13.09.2016 23:43, Xavier de Gaye wrote:
> 
> Xavier de Gaye added the comment:
> 
> You do not answer my questions and resort to sarcasm instead: "How did you make it more difficult?". I understand that you may be upset because issue 23968 has been dismantled by the recent decision of removing the platdir files, but please keep exchanges on the bug tracker strictly on a technical level.
> 
> I am strongly opposed to this change for the reasons given in my first post.  Unless you can explain why this change is needed, please close this issue as rejected.

you do not listen. _PYTHON_HOST_PLATFORM is already defined, and released in
3.5. Why can't you have a look at the code?  Please calm down.  Don't invent new
solutions but instead use the existing ones. You can reach me on irc too, if you
think I offend you (I don't intend to), I'd like to carry on with the cross
builds including android, but not having the existing cross build infrastructure
being replaced by some new android concepts.

Matthias
msg276357 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-09-13 22:21
What do you mean ? When cross-compiling, _PYTHON_HOST_PLATFORM is defined as $MACHDEP-$_host_cpu, but _PYTHON_SYSCONFIGDATA_NAME is suffixed with $(MACHDEP)_$(MULTIARCH) instead.
msg276361 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-09-13 22:31
> but not having the existing cross build infrastructure being replaced by some new android concepts.

Running 'hg log' on configure.ac and Makefile.pre.in shows that there is not a single change in the cross build infrastructure done for some new android concept, whatever that means.
msg276438 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-09-14 13:20
> please try to build extension modules with mismatching abi flags (pydebug is the
relevant one). Post your results for both mismatch cases.

Hum, you are claiming that there is a problem with mismatching abi flags but don't care to explain why or to demonstrate the problem, and asking instead that it should be proven by someone else running a test that you are right...

Anyway, you are mistaken. The cross-compilation of Android with 'pydebug' set, using a native interpreter built without 'pydebug' gives the following results:

The native interpreter:
[xavier@bilboquet python-native]$ ./python
Python 3.7.0a0 (default:879bde95a456+, Sep 13 2016, 18:59:47) 
[GCC 6.1.1 20160707] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.abiflags
'm'

The Android interpreter built with this native interpreter:
root@generic_x86:/data/data/org.bitbucket.pyona # python
Python 3.7.0a0 (default:879bde95a456+, Sep 14 2016, 14:53:50) 
[GCC 4.2.1 Compatible Clang 3.8.243773 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.abiflags
'dm'
>>> import _socket
>>> _socket.__file__
'/data/data/org.bitbucket.pyona/python/lib/python3.7/lib-dynload/_socket.cpython-37dm-i686-linux-android.so'
msg276440 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-09-14 13:27
@Matthias Klose: I concur with Xavier, can you please behave as a professional and "keep exchanges on the bug tracker strictly on a technical level"? Personal attacks doesn't help us to contribute with you.

I didn't follow the discussion, but I'm sure that we can find a solution with work for all use cases: multiarch, cross compilation, etc.
History
Date User Action Args
2016-09-14 13:27:07vstinnersetmessages: + msg276440
2016-09-14 13:20:05xdegayesetmessages: + msg276438
2016-09-13 22:31:47xdegayesetmessages: + msg276361
2016-09-13 22:21:35xdegayesetmessages: + msg276357
2016-09-13 21:50:00dokosetmessages: + msg276351
2016-09-13 21:43:49xdegayesetmessages: + msg276349
2016-09-13 20:55:59dokosetmessages: + msg276342
2016-09-13 14:40:12xdegayesetmessages: + msg276279
2016-09-13 13:47:41yan12125setnosy: + yan12125
2016-09-13 13:12:57xdegayesetnosy: + xdegaye
2016-09-13 10:59:14vstinnersetnosy: + vstinner, zach.ware
2016-09-13 10:53:53dokocreate