classification
Title: Steps for Android Native Build of Python 3.4.2
Type: enhancement Stage:
Components: Build Versions: Python 3.4
process
Status: open Resolution:
Dependencies: 16353 20305 21668 Superseder:
Assigned To: Nosy List: Chi Hsuan Yen, Roman.Evstifeev, Ryan.Gonzalez, chaselton, cvrebert, eric.snow, freakboy3742, mpaolini, r.david.murray, terry.reedy, tritium
Priority: normal Keywords: patch

Created on 2015-02-21 20:46 by chaselton, last changed 2016-06-07 17:47 by cvrebert.

Files
File name Uploaded Description Edit
patch.diff chaselton, 2015-02-21 20:46 patch for pythonrun.c
android_segfault_fix.patch Ryan.Gonzalez, 2015-03-20 00:43
kbox_fix.patch Ryan.Gonzalez, 2015-03-20 00:43
issue_16353.patch Ryan.Gonzalez, 2015-03-20 00:43 review
issue_20305_tweaked.patch Ryan.Gonzalez, 2015-03-20 00:43
issue_21668.patch Ryan.Gonzalez, 2015-03-20 00:44
unused_var.patch Ryan.Gonzalez, 2015-03-20 00:44
rjmatthews62_fixes_tweaked.patch Ryan.Gonzalez, 2015-03-20 00:44
androidfn.patch Ryan.Gonzalez, 2015-04-06 16:19 review
issue_20306.patch Ryan.Gonzalez, 2015-04-06 20:21
rjmatthews64_fixes2.patch Ryan.Gonzalez, 2015-04-09 15:24
rjmatthews64_fixes2.patch Ryan.Gonzalez, 2015-04-09 15:38
rjmatthews64_fixes2.patch Ryan.Gonzalez, 2015-04-09 17:07
kbox_fix.patch Ryan.Gonzalez, 2015-04-20 00:26
python.patch chaselton, 2015-04-20 00:30 patch for python.c. that prevents android segfault
kbox_fix.patch Ryan.Gonzalez, 2015-06-04 21:52
lib_fixes.patch Ryan.Gonzalez, 2015-06-04 21:56
py_test_results.log chaselton, 2015-07-23 22:14 ./python -m test log
test_gdb.log chaselton, 2015-09-26 14:10
Messages (176)
msg236389 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-02-21 20:46
This is a (hopefully) complete list of patches and modifications required to natively build Python 3.4.2 on an Android device where
patches == listed issues in this bug tracker
modifications (mods) == unsubmitted patches and/or edits made to source

CAVEATS
Build was completed on a tablet running Android 4.4.2 and within the KBOX environment; an app that simulates a Linux filesystem by using a ported version of fakechroot.  Details of the environment can be found here: http://kevinboone.net/kbox2_how_it_works.html. NOTE: During the Python build it was necessary to patch this environment with a modified version of fakechroot; the KBOX version currently available as of 2.21.2015 may not contain the modified fakechroot.

CONFIGURE && POST-CONFIGURE && MAKE
The configure command was run with the following parameters
--enable-shared
--without-ensurepip
--prefix=/usr/python

After running configure. changes were made to pyconfig.h to account for various platform limitations and to Setup to ensure that a) all modules specified were built as shared and b) modules that used symbols from libpython were explicitly linked to said library. See attached pyconfig.h and Setup.

Make fails when the newly built python binary tries to execute and run setup.py. This is resolved by copying the shared library from the build directory to /lib in the KBOX environment.  Otherwise make && make install runs without errors provided patches/mods are applied. I did not run make test or the equivalent for this build.

PATCHES/MODS
The attached patches from the following issues must be applied:
http://bugs.python.org/issue20305
http://bugs.python.org/issue16353 (NOTE: use patch with 'defpath' not 'confstr', which is not supported in Android)
http://bugs.python.org/issue21668

The following modifications to source must be made:
The instructions under 'Code Changes' at this link: https://code.google.com/p/python-for-android/wiki/CrossCompilingPython
The attached patch, contributed by Ryan Gonzales (patch.diff)
The changes listed below by file (apologies for formatting)
        
/* patch for strdup segfault issue in readline.c */
 #define RESTORE_LOCALE(sl) { if (sl) { setlocale(LC_CTYPE, sl); free(sl); } }
 
 char *_locale = setlocale(LC_CTYPE, NULL); 
 char *saved_locale; 
 if(!_locale) 
 _locale = ""; 
 saved_locale = strdup(_locale);
 
 
 char *saved_locale = setlocale(LC_CTYPE, NULL);
    if (saved_locale != NULL) {
        saved_locale = strdup(saved_locale);
        if (!saved_locale)
            Py_FatalError("not enough memory to save locale");
    }
    

/* fix for Lib/ctypes/__init.py__ */
 30 DEFAULT_MODE ==  RTLD_GLOBAL
 
 /* patch for python.c */
/* here line for oldloc is commented out */
     24     int i, res;
     25     /* char *oldloc; */
     26 #ifdef __FreeBSD__

/* here oldloc lines commented out */
     46
     47   /*  oldloc = _PyMem_RawStrdup(setlocale(LC_ALL, NULL));
     48    * if (!oldloc) {
     49    *     fprintf(stderr, "out of memory\n");
     50    *     return 1;
     51    *
     52    * }
     53    */

 /* patch for pythonrun.c 307, --309-310, ++311-313 --1009-1012 ++1013-1014 */
 
    305     return get_codec_name(codeset);
    306 #else
    307    char* m;
    308    PyErr_SetNone(PyExc_NotImplementedError);
    309    /*  return NULL; */
    310    /* char* m; */
    311    m = malloc(6);
    312    strcpy(m, "ascii");
    313    return m;
    314 #endif
    315 }
    
   1002 static int
   1003 initfsencoding(PyInterpreterState *interp)
   1004 {
   1005     PyObject *codec;
   1006
   1007     if (Py_FileSystemDefaultEncoding == NULL)
   1008     {
   1009         /* Py_FileSystemDefaultEncoding = get_locale_encoding();
   1010          * if (Py_FileSystemDefaultEncoding == NULL)
   1011          *   Py_FatalError("Py_Initialize: Unable to get the locale encoding
   1011 ");
   1012          */
   1013            Py_FileSystemDefaultEncoding = malloc(6);
   1014            strcpy(Py_FileSystemDefaultEncoding, "ascii");
   1015         Py_HasFileSystemDefaultEncoding = 0;
   1016         interp->fscodec_initialized = 1;
   1017         return 0;
   1018     }
   1019
   
/* changes to Lib/plat-linux/DLFCN.py */
     76 # Included from bits/dlfcn.h
     77 # Changed for Android
     78 RTLD_LAZY = 1
     79 RTLD_NOW = 0
     80 #RTLD_BINDING_MASK = 0x3
     81 #RTLD_NOLOAD = 0x00004
     82 RTLD_GLOBAL = 2
     83 RTLD_LOCAL = 0
     84 #RTLD_NODELETE = 0x01000
     85
     86 # Also from Android's dlfcn.h
     87 RTLD_DEFAULT ((void*) 0xffffffff)
     88 RTLD_NEXT     ((void*) 0xfffffffe)
     89
msg236444 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-02-23 14:13
The attached g-zipped file contains the first set of patches required to build Python 3.4.2 from source in the environment specified in the origonal post.

Will post the second/final set ASAP
msg236445 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2015-02-23 14:38
It's not convinient to have patches in a tarball. Please at least open patches one by one, or better: open one issue per patch. You should group similar changes in a single patch: for example all changes related to wcstombs().

You should explain each change.


For example, the following change (os.patch) looks like a bug. You cannot simply rename a function. It completly breaks backward compatibility.

-.. function :: get_shell()
+.. function :: get_shell_executable()


selectmodule.patch:

-        timeout = (int)ceill(dtimeout * 1000.0);
+        timeout = (int)ceil(dtimeout * 1000.0);

I don't understand where ceill() does come from. I cannot find it in the current source code. Maybe the patch was produced in the wrong direction?


patch.diff: please generate unified patches. Your format is ugly, I cannot read it. And the bug tracker failed to create a "review" button.


It's hard to follow instructions to apply patches. Maybe you should use a Mercurial repository, which include all changes?
msg236447 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-02-23 15:26
Apologies for the tarball, but all patches within are related to this "issue"

Removing tarball and will re-post individual, cleaned-up patches, grouped by issue.  

Ryan, can you re-do patch for pythonrun.c? If not. I'l. work it in.
msg236462 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-02-23 19:47
Cyd,

What revision of the Python 3.4 source is this against? The tarball?

I'm not quite sure what to redo that patch against.

Also, shouldn't the code be selectively enabled with macros/configure.ac edits? Or did I miss where someone said this should be in its own branch (which, in my opinion, isn't very clean)?
msg236469 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-02-23 23:07
Ryan,
I manually applied the patch to the Python 3.4.2 source tarball, dowmloaded from the main site to my Android tablet. I'm guessing (re)generating the patch from that source should work.

I've no knowledge of configure syntax, but most of the patches apply only if __ANDROID__ is defined.  This can be enabled with the -mandroid/-mbionic CFLAGS that work with GCC 4.8.x and later.
msg236488 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-02-24 12:32
Additionally, for the Lib/plat-<os> specific changes, I can create Lib/plat-android and move the modified DLFCN.py and others there, provided someone could help with the appropriate configure.ac syntax.

I can/could do it myself, but it would take a *while* as I'd need to learn a fair amount of configure scripting first.
msg236530 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-02-24 19:08
So...

I know a decent amount of configure scripts, although Python's is easily going to be the absolute longest I've ever touched.

Maybe these should be made against the 3.6 Mercurial source? I'm not sure if the Python devs would prefer that.

Also, should the configure script be based on the Android target triple?
msg236566 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-02-25 10:04
Pulling from Mercurial will require taking a couple of huge steps back, as I've ported git to the environment I am using but not Mercurial.  It would also mean getting up to speed *fast* as I know nothing about VCS...including Mercurial.

I'm willing to do this (what's another port after all) but it may be a while before I can get everything set up correctly.
msg236567 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2015-02-25 10:54
Take a look at https://docs.python.org/devguide/gitdevs.html
"Mercurial for git developers"
msg236571 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-02-25 11:38
Thanks for the suggestion Victor, however I've only ported git to this environment; the most I;ve used it is to run 'git clone' to pull source from a repo. The recommendation from the mobile-sig mailing list is...given my lack of familiarity with VCS..is to attempt to port and work with Mercurial.  If doing so starts to take too much bandwidth I'll fall back to git and the article you linked.

Side Note: I cannot emphasize enough how little I know about how to use git and most of the other tools I've ported to this environment. My goal was to get as many of the 'standard' Linux utilities running in this environment, then start learning them.
msg236578 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-02-25 14:35
There is a github mirror of the hg repository (https://github.com/python/cpython).  Patches produced from it don't automatically get review links in the tracker, but at least they would then be against the most current source (and someone else could regenerate them).  That would at least allow you to skip the "port mercurial" step.

On the other hand, it sounds like Ryan may be volunteering to update the patches to apply to the head of default for you.  (And yes, Ryan, the patches should be against the head of the default branch.)

Now, whether or not we will accept the patches is a different discussion, but I'd certainly like to see more Android support in our tree myself.
msg236584 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-02-25 15:31
Thanks for that suggestion R. David...that would be ideal given the 2.x requirements of Mercurial would mean i'd need to re-port Python before tackling a port of Mercurial.

I'll wait until Ryan chimes in before I start serious work with git, just in case.
msg236611 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-02-25 18:06
That's what I was saying; I'll update the patches myself. I highly doubt Cyd wants to re-port Python again. :0

I'll ask on python-dev on which would be preferred: configure.ac edits or macro tests.
msg236667 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-02-26 12:45
Thanks Ryan.
Should I create a branch off the master at github and patch there or just clone the master and patch?
msg236683 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-02-26 16:29
...or, should I fork the 3.4 branch?  If so, which one...3.4 or origin/3.4?
msg236685 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2015-02-26 16:38
I don't think that Android support should be added to a stable branche
(2.7 or 3.4), only in the development branch.

You may maintain your own 2.7 or 3.4 fork if you want.
msg236722 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2015-02-27 01:30
Once this android Python is built, can it:

  - be copied from/to other android devices?

  - run without KBOX?
msg236746 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-02-27 13:35
Ethan,
The binary produced runs in the KBOX environment. It can be copied between devices provided the target device has the KBOX environment installed. 

Victor et al,
I read https://docs.python.org/devguide/devcycle.html#indevbranch a couple of times and, unless I;m missing something the dev branch only contains the 3.5 release.  This may be an obvious question but if I fork the 3.4 branch would Android-related patches be eligible for that branch?

I'm hesitant to clone the dev branch as I;ve worked hard to get the latest stable version (3.4.2?) working on my device; I'm more interested in a) making my efforts available to others and b) learning the Python language than c) constantly working at making the latest and greatest Android-friendly
msg236756 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2015-02-27 14:55
> c) constantly working at making the latest and greatest Android-friendly

But that is precisely what "Android support", should it be added, means:
It does take constant work (and build slaves) to support a platform.


Unrelatedly, regarding the localeconv changes: 

I cannot find any bugreport for localeconv at

  https://code.google.com/p/android/issues/list .


Before we add workarounds to Python, it would be nice to have
at least a record that the Android developers actually *did*
refuse to add a simple struct lconv (trivial for en_US!).

How about opening an Android issue?
msg236761 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-02-27 15:11
Given Stephan's comments regarding Android support I'm beginning to think that I may have bitten off more than I can chew. Is there an option between 'constant work' and 'zero contribution'?

I am not by any means an Android developer so the following is possibly riddled with miconceptions and errata, but, regarding locale support, I think there's a difference between 'regular' and 'native' development.  Native development involves using the NDK to port various bits of C-written utilities and libraries to Android...like python... and it is where the locale is broken, because of Android's limited libc.
msg236763 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-02-27 15:23
Most of us fall toward the lower end of "constant work" and "zero contribution", honestly, since the majority of us are doing it in spare time and not getting paid for it.  What is needed is a long-term commitment to fix bugs if they show up when new changes are made to CPython.  Which is where a buildbot is fairly critical...and I have no idea how complicated it will be to set up a buildbot for your environment.  I think it should be possible, though, since buildbot and twisted are written in python.
msg236766 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2015-02-27 15:31
But NDK bugs are reported to the same bug tracker, aren't they?
msg236773 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2015-02-27 16:05
BTW, PEP 11 now demands a stable buildbot for official platform support
(IMO a very sane policy).
msg236779 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2015-02-27 16:25
Cyd, if you want to be a CPython/Android resource that's great.

If you don't have time for it, I completely understand.  What I'm hoping for is to take your initial efforts and build from there, as there are others who can take what you've started and run with it.

Make no mistake, your taking the time to share what you've done is greatly appreciated.
msg236797 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-02-27 17:17
I'll try to do as much as I can to get this through. Once I reinstall the NDK (I accidently wiped my hard drive with a bad dd command recently), I'll test this on my old Android phone that still runs 2.3 and would be very happy if someone else can test it something newer (my laptop doesn't seem to like the Android emulator). I'll also try to do this against the development branch or tip or head or whatever it's called.

I'd be very happy if someone could contribute a build slave, because I can't.
msg236800 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2015-02-27 17:27
I will work on the build slave (note: it will definitely be /work/ so if anyone has the resource and know-how to just do it, I will not be offended ;) .
msg236801 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-02-27 17:28
Stefan,
I wouldn't know if they're reported to the same bug tracker...it's possible they aren't.  Additionally it's possible that the lack of locale support in libc isn't considered a bug.

Ethan,
Given Victor's recommendation of using the dev branch and the fact that my mods were made to a stable download (3.4.2), what would be the recommended way to go about getting my efforts into github so that they can be built upon?  Should I start working with the fork I've already made of cpython (with 3.4.2 as the default branch) or is there a different method I should use?
msg236810 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2015-02-27 18:25
I'm afraid I know next to nothing about git, so cannot help there.

I would think that it wouldn't be too hard for someone (such as Ryan or myself) to forward port a set of 3.4.2 patches to 3.5 -- so whatever is easiest for you.
msg236819 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-02-27 19:10
How does this sound?
* I'll clone the fork of the 3.4 branch I made in github, build and patch.
* Ryan will (as best as he can) grab said patches, regenerate them for the bug tracker or forward port them to 3.5 (in github? to Mercurial)
* I'll figure out the upgrade/maintain/patch workflow as best I can so i can set it up on my build device and help out with porting later versions of Python.
msg236820 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2015-02-27 19:13
Sounds great!
msg236857 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-02-27 23:26
Ok, if I don't finish it today, I *will* finish it tomorrow. Or Sunday. Sunday is the latest, though I'm pretty sure I'll be done by Saturday.

Hardest part is likely going to be re-configuring the Android emulator to test it.

Cyd (and anyone else who can), do you think you could test the binaries, too? I'll get them to you somehow. You have a real device; I have an old real device and an emulator.

Side note for Cyd: I'd advise you just use the fork of CPython you have and just push them to GitHub:

$ git remote add origin <url of your GitHub repo with .git at the end>
$ git push --all origin -u

That will push your changes (and every branch of CPython, I have no clue what your branch name is, so this just does everything) to the GitHub mirror.

So...I'm understanding that the game plan will go like this:

- Cyd pushes the changes he made to a GitHub fork of Python (and hopefully posts the URL here ;)
- I clone the repo and try to get the changes working with the dev version
- Somewhere in between now and the end of the world, Ethan will set up a build slave

I'll also help all I can with keeping these changes working with future versions of Python.
msg236858 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2015-02-27 23:36
Yup, that sounds right!  ;)
msg236883 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2015-02-28 14:00
> I wouldn't know if they're reported to the same bug tracker...it's possible they aren't.

Well, why don't you try? :)


> Additionally it's possible that the lack of locale support in libc isn't considered a bug.

Their "struct lconv" violates both the C standard and POSIX. It is
100% a bug.  Lack of locale support would be defensible, shipping
a broken localeconv() isn't.

I'm just furious that 20+ OSS projects go out of their way to work
around this trivial bug.  It's a waste of resources.
msg236884 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-02-28 14:04
>Well, why don't you try? :)

Resource optimization.
msg236886 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2015-02-28 14:10
Fine, I'm also optimizing and the "fix" isn't going into libmpdec.
Android can use the Python version of decimal.
msg236890 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-02-28 15:07
>Fine, I'm also optimizing and the "fix" isn't going into libmpdec.
>Android can use the Python version of decimal.

Er, ok. Not entirely sure what this means, mostly because the handful of spare cycles not allocated to a) getting a fork up, running and patched b) learning git on the fly for a) c) porting spidermonkey (why I ported Python) and d) learning Python, are already dedicated to other projects.  And work.
msg236911 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-02-28 21:26
Ok...so the joys of autoconf configuring bite again. ALL Android devices have /dev/ptmx (adb even assumes it)...should the configure script be modified to skip that check if cross-compiling for Android?
msg236926 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-03-01 00:17
*ALL Android devices have /dev/ptmx (adb even assumes it)...should the *configure script be modified to skip that check if cross-compiling for *Android?

Yes, definitely. See the mods to pyconfig.h here"
https://code.google.com/p/python-for-android/wiki/CrossCompilingPython

Ignore the #define ANDROID 1.  GCC 4.8.x and later add -mandroid and/or -mbionic which defines __ANDROID__; I've been putting android-related changes under that macro.

*Cyd (and anyone else who can), do you think you could test the binaries, *too? I'll get them to you somehow. You have a real device; I have an old *real device and an emulator.

Sure...especially since the KBOX env is needed.

*Side note for Cyd: I'd advise you just use the fork of CPython you have *and just push them to GitHub:

$ git remote add origin <url of your GitHub repo with .git at the end>
$ git push --all origin -u

*That will push your changes (and every branch of CPython, I have no clue *what your branch name is, so this just does everything) to the GitHub *mirror.

I forked the Cpython repo on Github, set 3.4 as the default branch and

$git clone https://github.com/cydhaselton/cpython.git

After modifying files and/or adding dirs, copying files

$git add <modified file>

Hopefully the above was ok.

Off to read up on commits
msg236999 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-03-02 00:47
Cyd:

Just run:

$ git commit -m "Put some message here" # commit the changes
$ git push # push the changes

I'm working on it, but I haven't even gotten it to build (CURSE YOU, AUTOCONF!)
msg237009 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-03-02 03:23
Ryan,
Thanks. First commit in; all fixes for Android's broken mbstowcs, new plat-android and plat-android/DLFCN.py.

What are you trying to build?
msg237073 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-03-02 18:44
Cyd:

I was trying to see if I could build the latest Python for Android (not to get it to run; just to get it to compile) so that your changes would (theoretically) easier to update to the latest Python. That failed. Miserably. Joys of cross-compiling Python.

So...I'm just going to wait for you to finish uploaded the changes and work from there. :)
msg237118 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-03-03 10:45
Ryan,
Sounds good. I think I've got all of the bug tracker patches committed/pushed...now I need to do all of the other edits.  Aiming to be finished by Friday/Saturday.
msg237215 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-03-04 22:37
Ryan,
I believe I've gotten all non-configure dependent commits in. 
How do you want me to handle post-configure edits I;ve made to get Python working (i.e. changes to pyconfig.h, Modules/Setup...)
msg237216 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-03-04 22:42
Cyd:

What exactly do you mean by post-configure edits?
msg237219 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-03-04 23:23
Ryan,
Edits made to files generated or modified by running ./configure.  Example: pyconfig.h.
msg237222 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-03-05 00:05
Cyd:

Don't upload them. Git won't even let you add pyconfig.h; running `git add PC/pyconfig.h` won't say anything, but running `git status` will show that it did not do anything.
msg237260 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-03-05 11:11
I understand; I haven't made the edits to these files but the edits are needed in order for the Android build to work properly.

How do you want me to handle them? I can post the expected values here as a diff, or just as plain text...?
msg237280 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-03-05 17:57
So...you didn't make the changes by hand? Could you just post whatever you passed to configure?
msg237284 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-03-05 20:09
I can...

./configure --prefix=/usr/python --enable-shared --without-ensurepip

...but configure didn't set up pyconfig.h correctly with those options. There are some changes I had to make post-configure to pyconfig.h and others when I initially built python...see this link: https://code.google.com/p/python-for-android/wiki/CrossCompilingPython
specifically the part about manually editing pyconfig.h
msg237285 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-03-05 20:33
Ahh...ok, then please post the changes. I want to burn autotools enough right now; I doubt that redoing those changes would ease my anger. :)
msg237374 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-03-06 19:52
Ryan,
Here are the changes to pyconfig.h.  I left out this
#define ANDROID 1
because GCC 4.8.x and up have -mandroid -mbionic that define __ANDROID__ so I've been using that.

#define HAVE_BROKEN_MBSTOWCS 1
#undef HAVE_DEV_PTMX
#undef HAVE_GETHOSTBYNAME_R
#undef HAVE_MBRTOWC
#define HAVE_NCURSES_H 1  /* This only if you've cross compiled curses */
#undef HAVE_SETLOCALE
#undef HAVE_WCSCOLL
#undef HAVE_WCSFTIME
#undef HAVE_WCSXFRM

The following changes were made to Modules/Setup

*shared*

<snip>

array arraymodule.c     -L. -lpython3.4m # array objects
cmath cmathmodule.c _math.c -L. -lpython3.4m -lm # complex math library functions
math mathmodule.c _math.c -L. -lpython3.4m -lm # math library functions, e.g. sin()
_struct _struct.c       -L. -lpython3.4m # binary structure packing/unpacking
time timemodule.c  -L. -lpython3.4m -lm # time operations and variables
_testcapi _testcapimodule.c    -L. -lpython3.4m # Python C API test module
_random _randommodule.c -L. -lpython3.4m # Random number generator
_elementtree -I$(srcdir)/Modules/expat -DHAVE_EXPAT_CONFIG_H -DUSE_PYEXPAT_CAPI _ele
menttree.c      -L -lpython3.4m # elementtree accelerator
_pickle _pickle.c       -L. -lpython3.4m # pickle accelerator
_datetime _datetimemodule.c     -L. -lpython3.4m # datetime accelerator
_bisect _bisectmodule.c -L. -lpython3.4m # Bisection algorithms


unicodedata unicodedata.c    -L. -lpython3.4m # static Unicode character database

<snip>

#grp grpmodule.c                # grp(3)                                             select selectmodule.c   -lm -L. -lpython3.4m # select(2); not on ancient System      V                                                                                    

# CSV file helper
_csv _csv.c -L. -lpython3.4m

# Socket module helper for socket(2)
_socket socketmodule.c -L. -lpython3.4m

# Socket module helper for SSL support; you must comment out the other               # socket line above, and possibly edit the SSL variable:                             #SSL=/usr/local/ssl                                                                  _ssl _ssl.c \                                                                                -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \                                -L. -lpython3.4m -L$(SSL)/lib -lssl -lcrypto
msg237574 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-03-08 22:03
I'm getting a weird error message building python:

ryan@DevPC-LX:~/stuff/cpydroid$ make
_PYTHON_PROJECT_BASE=/home/ryan/stuff/cpython _PYTHON_HOST_PLATFORM=linux-arm PYTHONPATH=./Lib:./Lib/plat-linux python3 -S -m sysconfig --generate-posix-vars ;\
	if test $? -ne 0 ; then \
		echo "generate-posix-vars failed" ; \
		rm -f ./pybuilddir.txt ; \
		exit 1 ; \
	fi
Could not import runpy module
Traceback (most recent call last):
  File "./Lib/runpy.py", line 15, in <module>
    import importlib.util
  File "./Lib/importlib/util.py", line 6, in <module>
    from ._bootstrap import module_from_spec
ImportError: cannot import name 'module_from_spec'
generate-posix-vars failed
make: *** [pybuilddir.txt] Error 1
ryan@DevPC-LX:~/stuff/cpydroid$ 


I'm trying to build for generic ARM Linux before I attempt updating the changes. Makes sense theoretically.

Then again, when using autotools, NOTHING ever makes sense. I'm probably looking in the wrong direction...
msg237639 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-03-09 12:05
Ryan,
What version of Android and KBOX are you using? Are you cross-compiling?
If not could you post the output of printenv from within KBOX here?
msg237654 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2015-03-09 14:45
> ImportError: cannot import name 'module_from_spec'

The command line uses the system python3, which is "too old"
and does not have 'module_from_spec' yet.

Try running ...

   python3 -S -m sysconfig --generate-posix-vars

... and then continue with `make`.


We should open an issue for that, it's annoying.
msg237680 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-03-09 17:28
Python really needs some way of separating the host C compiler and the target C compiler.

I'm giving up cross-compiling from my computer and am going to install KBOX on an Android virtual device. It worked for Cyd, it'll work for me. Beats compiling two separate versions of Python (one just to get host version of pgen and _freeze_importlib), passing 20+ arguments to configure segregated in flags and environment variables, etc.

Please bear with me.
msg237681 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2015-03-09 17:38
Cross compiling worked for a while in 3.4. It broke again
because we don't have a buildbot for that.

You should not need 20 args for ./configure. At least on
Ubuntu the script from #5404 generally works, but indeed
pgen, importlib and the above issue are currently broken
(again).
msg237682 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-03-09 17:42
This is what I'm using:

PYTHON_FOR_BUILD=`realpath ../cpython/python` CC=arm-linux-androideabi-clang CXX=arm-linux-androideabi-clang++ ./configure --host=arm-linux-androideabi --build=x86_64 --disable-ipv6 ac_cv_file__dev_ptmx=no ac_cv_file__dev_ptc=no ac_cv_have_long_long_format=yes

I know I could use config.site, but I just popped all this in a shell script.
msg237683 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2015-03-09 17:52
On Ubuntu, when I uploaded the script to #5404, I only needed:

  ./configure --prefix=/tmp/arm-install --without-ensurepip --host=arm-linux-gnueabi --build=x86_64 --disable-ipv6


(And config.site of course.)


We just have to fix the pgen etc. issues and someone has to provide
a buildbot, otherwise it's guaranteed to break again.
msg237684 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2015-03-09 17:56
This would be a build-bot for cross-compiling?  As opposed to an android build-bot (which we'll also need) ?
msg237693 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2015-03-09 19:01
Technically we could use the Android buildbot to test both
(cross-compiling and Android breakage), but it would be nice
to have a separate buildbot that just tests cross-compiling
with an easier target (e.g. arm-linux-gnueabi).
msg237724 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-03-09 23:50
Ryan,
There are utilities you'll need in the KBOX environment, some of which may not be available at the KBOX download site.  

You'll need to download the make, gcc 4.8.0 and gawk packages from the download site, and vim and/or nano depending on your preference. 

If you get an 'undefined reference to dlopen' error during the build, you'll need the updated libfakechroot; update this ticket and I'll figure out a way to get it to you (I'm not sure if the currently available KBOX includes it or not).
msg237725 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-03-10 00:02
Ugh...

I'll just cross my fingers and hope that the libfakechroot KBOX2 comes with is new enough.

As for a text editor, I'll just probably use something external.

*crosses fingers*
msg237774 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-03-10 15:29
Ok, please bear with me. I'm going to be REALLY busy for a bit; I have a BIG test/exam coming up next week, a music recital shortly afterwards, and lots of stuff/studying in between.

Sorry for any delays!
msg237785 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-03-10 17:45
Dog just died unexpectedly, behind on photograph project, need to start on Python tutorial, have a chat in a different language on Sunday I need to prep for and its patch day tomorrow.

So yeah. Completely understand.
msg238592 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-03-20 00:43
I am slowly attaching the patches I created.

Titles:

- issue_16353.patch - Issue 16353 for tip
- issue_20305_tweaked.patch - Issue 2305 for tip
- rjmatthews62_fixes_tweaked.patch - rjmatthews62@gmail.com's Android fixes (a bit more modularized)
- unused_var.patch - An extension of 2305. Can't figure out how to merge them.
- issue_21668.patch - Issue 21668 for tip
- issue_20306.patch - Guess...
- android_segfault_fix.patch - The actual segfault fix
- kbox_fix.patch - Cyd's KBox fixes

Someone, please test them!
msg238744 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-03-21 00:44
...except for the pyconfig.h changes. Need to do something with autoconf for that.
msg238932 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-03-22 20:04
Ryan,
Has anyone taken you up on the non-KBOX testing?
Do you need me to test the patches within the KBOX environment?
msg238934 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-03-22 20:10
Cyd: if you could do that, it would be great!
msg239256 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-03-25 13:53
Ryan,
From where do you want me to download Python for testing?
I'm assuming I need to test by a) downloading an unpatched 3.4.2, b) applying patches, c) running ./configure && make && make install.. If this is not correct let me know
msg239264 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-03-25 15:41
No; you need the CPython tip. You can use https://github.com/python/cpython. Other than that, it's the configure+make thing.

As I said before, the pyconfig.h changes aren't there yet. Trying to figure out how I want to approach those.
msg239466 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-03-28 20:42
Ryan,
For now I'll make the edits to pyconfig.h manually after downloading the zip from the tip and running configure with necessary params.

I'll do my best to get this done this weekend, but it may have to wait until next.
msg240117 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-05 13:08
Hey Ryan,
Just now patching downloaded/unzipped tip and was wondering if there was an order in which patches should be applied.

I ask because i'm getting the following when applying the android_segfault_fix.patch

/bld/python/cpython-3.4/cpython-3.4/Python $ patch < android_segfault_fix.patch
patching file frozenmain.c
Hunk 3 FAILED 53/56.
     }

     setlocale(LC_ALL, "");
+#endif
     for (i = 0; i < argc; i++) {
         argv_copy[i] = Py_DecodeLocale(argv[i], NULL);
         argv_copy2[i] = argv_copy[i];
patching file pylifecycle.c
patch: can't open 'pylifecycle.c': No such file or directory

Thoughts?
msg240118 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-05 13:14
UPDATE: I found the file in github, under master, in Python/. It's not in the 3.4 or origin/3.4 branches...aren't we working on those? Or does it not matter?
msg240119 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-04-05 13:25
I thought this was for the tip, i.e. the 3.5 dev...?

But I created the patches in the order that I wrote the descriptions in the comment. So you might want to use that order.

If that fails, I can figure out the revision I was at when I created the patches.
msg240127 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-05 17:42
From previous post:
**********************
How does this sound?
* I'll clone the fork of the 3.4 branch I made in github, build and patch.
* Ryan will (as best as he can) grab said patches, regenerate them for the bug tracker or forward port them to 3.5 (in github? to Mercurial)
**********************

For some reason I was thinking that the 3.4 branch I cloned was a dev branch.  I'll start again with a download of 3.5 dev.
msg240143 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-06 01:30
issue_20306.patch won't install; attempting to do so yields the following:

patching file configure.ac                                                           Hunk 56 FAILED 4944/4944.
 AC_MSG_RESULT($ENSUREPIP)
 AC_SUBST(ENSUREPIP)                                                                 
+AC_CHECK_MEMBER([struct passwd.pw_gecos],                                           +          [AC_DEFINE(HAVE_PASSWD_GECOS_FIELD, 1, [Define if <pwd.h> defines field passwd.pw_gecos])],                                                                    +          [],                                                                       +          [[#include <pwd.h>]])
+                                                                                     # generate output files
 AC_CONFIG_FILES(Makefile.pre Modules/Setup.config Misc/python.pc Misc/python-config.sh)
 AC_CONFIG_FILES([Modules/ld_so_aix], [chmod +x Modules/ld_so_aix])

For the time being I'll just use --without-ensurepip when configuring.
msg240144 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-06 02:10
Latest error:

gcc --sysroot=/usr/gcc-4.9.2/sysroot -c -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g  -O3 -Wall -Wstrict-prototypes    -Werror=declaration-after-statement   -I. -IInclude -I./Include    -DPy_BUILD_CORE -o Objects/unicodeobject.o Objects/unicodeobject.c
Objects/unicodeobject.c:45:23: fatal error: androidfn.h: No such file or directory
 #include "androidfn.h"
                       ^
compilation terminated.
make: *** [Objects/unicodeobject.o] Error 1

No idea what androidfn.h is, but will troubleshoot in the AM
msg240157 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-04-06 16:19
Ah, crud. androidfn.h was a header I added but forgot to put the patch for. I just attached the patch for that now.

The commit I based the patches on was by Alexander Belopolsky and was named "merge". Committed on February 28.
msg240176 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-06 20:07
Thanks Ryan, the patch worked.

Latest error (when building Modules/pwdmodule.c:)

^
./Modules/pwdmodule.c:86:2: error: stray '#' in program
 +#endif
  ^
./Modules/pwdmodule.c:86:3: error: 'endif' undeclared (first use in this function)
 +#endif
   ^
./Modules/pwdmodule.c:86:3: note: each undeclared identifier is reported only once for each function it appears in
./Modules/pwdmodule.c:75:21: error: expected ';' before 'sets'
 #define SETS(i,val) sets(v, i, val)
                     ^
./Modules/pwdmodule.c:87:5: note: in expansion of macro 'SETS'
     SETS(setIndex++, p->pw_dir);
     ^
./Modules/pwdmodule.c: At top level:
./Modules/pwdmodule.c:81:0: error: unterminated #else
 #ifdef HAVE_PASSWD_GECOS_FIELD
 ^
make: *** [Modules/pwdmodule.o] Error 1
/bld/python/cpython-master $
msg240178 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-04-06 20:21
Remove the + at the beginning of line 87.

I re-uploaded issue_20306.patch to fix that issue.
msg240208 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-07 13:20
Will do, re: removing the '+'.

Also, attempted to apply the updated patch and got the following:

/bld/python/cpython-master $ patch -p1 < issue_20306\ \(1\).patch
patching file Modules/pwdmodule.c
Possibly reversed hunk 1 at 244
Hunk 1 FAILED 79/79.
     SETS(setIndex++, p->pw_passwd);
     PyStructSequence_SET_ITEM(v, setIndex++, _PyLong_FromUid(p->pw_uid));
     PyStructSequence_SET_ITEM(v, setIndex++, _PyLong_FromGid(p->pw_gid));
+#ifdef HAVE_PASSWD_GECOS_FIELD
     SETS(setIndex++, p->pw_gecos);
+#else
+    SETS(setIndex++, Py_None);
+    Py_INCREF(Py_None);
+#endif
     SETS(setIndex++, p->pw_dir);
     SETS(setIndex++, p->pw_shell);

patching file configure.ac
Hunk 56 FAILED 4944/4944.
 AC_MSG_RESULT($ENSUREPIP)
 AC_SUBST(ENSUREPIP)

+AC_CHECK_MEMBER([struct passwd.pw_gecos],
+          [AC_DEFINE(HAVE_PASSWD_GECOS_FIELD, 1, [Define if <pwd.h> defines field passwd.pw_gecos])],
+          [],
+          [[#include <pwd.h>]])
+
 # generate output files
 AC_CONFIG_FILES(Makefile.pre Modules/Setup.config Misc/python.pc Misc/python-config.sh)
 AC_CONFIG_FILES([Modules/ld_so_aix], [chmod +x Modules/ld_so_aix])
/bld/python/cpython-master $
msg240218 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-07 16:42
Update: Removing the random '+' on line 87 of pwdmodule.c allowed the build to continue but it failed again with the following during or soon after linking:

Objects/unicodeobject.o: In function `PyUnicode_EncodeLocale':
/bld/python/cpython-master/Objects/unicodeobject.c:3236: undefined reference to `android_wcstombs'
/bld/python/cpython-master/Objects/unicodeobject.c:3248: undefined reference to `android_wcstombs'
Objects/unicodeobject.o: In function `wcstombs_errorpos':
/bld/python/cpython-master/Objects/unicodeobject.c:3154: undefined reference to `android_wcstombs'
Objects/unicodeobject.o: In function `PyUnicode_DecodeLocaleAndSize':
/bld/python/cpython-master/Objects/unicodeobject.c:3518: undefined reference to `android_mbstowcs'
/bld/python/cpython-master/Objects/unicodeobject.c:3518: undefined reference to `android_mbstowcs'
Objects/complexobject.o: In function `_Py_c_pow':
/bld/python/cpython-master/Objects/complexobject.c:129: undefined reference to `sincos'
Python/fileutils.o: In function `Py_DecodeLocale':
/bld/python/cpython-master/Python/fileutils.c:322: undefined reference to `android_mbstowcs'
Python/fileutils.o: In function `Py_EncodeLocale':
/bld/python/cpython-master/Python/fileutils.c:487: undefined reference to `android_wcstombs'
/bld/python/cpython-master/Python/fileutils.c:489: undefined reference to `android_wcstombs'
Python/fileutils.o: In function `_Py_wfopen':
/bld/python/cpython-master/Python/fileutils.c:1008: undefined reference to `android_wcstombs'
collect2: error: ld returned 1 exit status
make: *** [Programs/_freeze_importlib] Error 1

Maybe the androidfn.h wasn't added to the files above?
msg240329 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-09 14:23
FYI, running 'make clean' && make does not resolve the last reported issue.
Trying 'make distclean' && ./configure && make.
msg240338 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-04-09 15:24
Ack...I feel smart. Attached is kind of a "part 2" to the rjmatthews patch. Apply and the errors shall be solved. :)
msg240339 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-09 15:33
Thanks Ryan.
(Probably should remove original androidfn.h patch; patch complains with 'file already exists' if I don't delete Include/androidfn.h before applying the latest patch)
msg240342 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-04-09 15:38
Whoops. Updated the patch.
msg240350 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-09 16:56
Hello Ryan,
Error from latest patch:

Python/pythonrun.c:44:8: error: conflicting types for 'android_mbstowcs'
 size_t android_mbstowcs(wchar_t *dest, char * in, int maxlen) {
        ^
In file included from Python/pythonrun.c:18:0:
Include/androidfn.h:10:8: note: previous declaration of 'android_mbstowcs' was here
 size_t android_mbstowcs(wchar_t *dest, const char * source, int maxlen);
        ^
Python/pythonrun.c:61:8: error: conflicting types for 'android_wcstombs'
 size_t android_wcstombs(char * dest, wchar_t *source, int maxlen)
        ^
In file included from Python/pythonrun.c:18:0:
Include/androidfn.h:9:8: note: previous declaration of 'android_wcstombs' was here
 size_t android_wcstombs(char * dest, const wchar_t *source, int maxlen);
        ^
Python/pythonrun.c: In function 'android_wcstombs':
Python/pythonrun.c:68:5: error: expected ';' before 'if'
     if (c >= 0xdc800 && c <= 0xdcff)
     ^
Python/pythonrun.c:63:11: warning: variable 'c' set but not used [-Wunused-but-set-variable]
   wchar_t c;
           ^
make: *** [Python/pythonrun.o] Error 1
msg240351 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-09 17:02
Perhaps the 'include androidfn.h' should be removed from pythonrun.c? Or the function definition added to the androidfn.h?

(IANACC)
msg240352 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-04-09 17:07
ARGH!

Fixed. Re-apply rjmatthews64_fixes2.patch.
msg240356 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-09 17:28
Hmmmm. That patch failed to apply:

Possibly reversed hunk 1 at 1582
Hunk 1 failed 35/35
msg240357 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-04-09 17:34
Maybe it's conflicted with the last one...

Try:

git checkout Python/pythonrun.c
git apply rjmatthews...

On Thu, Apr 9, 2015 at 12:28 PM, Cyd Haselton <report@bugs.python.org>
wrote:

>
> Cyd Haselton added the comment:
>
> Hmmmm. That patch failed to apply:
>
> Possibly reversed hunk 1 at 1582
> Hunk 1 failed 35/35
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue23496>
> _______________________________________
>
msg240363 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-09 18:51
Done, but got this error:

Python/pythonrun.c: In function 'android_wcstombs':
Python/pythonrun.c:67:5: error: expected ';' before 'if'
     if (c >= 0xdc800 && c <= 0xdcff)
     ^
Python/pythonrun.c:62:11: warning: variable 'c' set but not used [-Wunused-but-set-variable]
   wchar_t c;
           ^
make: *** [Python/pythonrun.o] Error 1
msg240366 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-04-09 19:13
I put a fix in the patch; I don't know why the heck it didn't apply. Just
go to Python/pythonrun.c, line 66 and put a semicolon (;) at the end of the
line.

On Thu, Apr 9, 2015 at 1:51 PM, Cyd Haselton <report@bugs.python.org> wrote:

>
> Cyd Haselton added the comment:
>
> Done, but got this error:
>
> Python/pythonrun.c: In function 'android_wcstombs':
> Python/pythonrun.c:67:5: error: expected ';' before 'if'
>      if (c >= 0xdc800 && c <= 0xdcff)
>      ^
> Python/pythonrun.c:62:11: warning: variable 'c' set but not used
> [-Wunused-but-set-variable]
>    wchar_t c;
>            ^
> make: *** [Python/pythonrun.o] Error 1
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue23496>
> _______________________________________
>
msg240401 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-09 23:12
Done.
Unfortunately:

./python -E -S -m sysconfig --generate-posix-vars ;\
if test $? -ne 0 ; then \
        echo "generate-posix-vars failed" ; \
        rm -f ./pybuilddir.txt ; \
        exit 1 ; \
fi
Segmentation fault
generate-posix-vars failed
make: *** [pybuilddir.txt] Error 1
/bld/python/cpython-master $

I'll break out logcat and addr2line...hopefully it's something that was just overlooked
msg240403 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-10 00:37
FYI, Here's the addr2line output

LD_LIBRARY_PATH=/bld/python/cpython-master:/data/data/jackpal.androidterm/kbox2/lib ./python -E -S -m sysconfig --generate-posix-vars ;\
if test $? -ne 0 ; then \
        echo "generate-posix-vars failed" ; \
        rm -f ./pybuilddir.txt ; \
        exit 1 ; \
fi
Segmentation fault
generate-posix-vars failed
make: *** [pybuilddir.txt] Error 1
/bld/python/cpython-master $ addr2line -C -f -e /lib/libpython3.5m.so.1.0 0008f1a4   _PyMem_RawStrdup
/bld/python/cpython-master/Objects/obmalloc.c:358
msg240404 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-10 00:53
FYI, I think this is the culprit.
In Python/pylifecycle.c:

static char*
get_codec_name(const char *encoding)
{
    char *name_utf8, *name_str;
    PyObject *codec, *name = NULL;

    codec = _PyCodec_Lookup(encoding);
    if (!codec)
        goto error;

    name = _PyObject_GetAttrId(codec, &PyId_name);
    Py_CLEAR(codec);
    if (!name)
        goto error;

    name_utf8 = _PyUnicode_AsString(name);
    if (name_utf8 == NULL)
        goto error;
    name_str = _PyMem_RawStrdup(name_utf8);
    Py_DECREF(name);
    if (name_str == NULL) {
        PyErr_NoMemory();
        return NULL;
    }
    return name_str;

error:
    Py_XDECREF(codec);
    Py_XDECREF(name);
    return NULL;
}

If I figure out a working patch I'll post it here
msg240406 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-04-10 01:10
I think it was the same issue as before, but, for some reason, it didn't applu or something...? Will look into it tomorrow.

Cyd Haselton <report@bugs.python.org> wrote:
>
>Cyd Haselton added the comment:
>
>FYI, I think this is the culprit.
>In Python/pylifecycle.c:
>
>static char*
>get_codec_name(const char *encoding)
>{
>    char *name_utf8, *name_str;
>    PyObject *codec, *name = NULL;
>
>    codec = _PyCodec_Lookup(encoding);
>    if (!codec)
>        goto error;
>
>    name = _PyObject_GetAttrId(codec, &PyId_name);
>    Py_CLEAR(codec);
>    if (!name)
>        goto error;
>
>    name_utf8 = _PyUnicode_AsString(name);
>    if (name_utf8 == NULL)
>        goto error;
>    name_str = _PyMem_RawStrdup(name_utf8);
>    Py_DECREF(name);
>    if (name_str == NULL) {
>        PyErr_NoMemory();
>        return NULL;
>    }
>    return name_str;
>
>error:
>    Py_XDECREF(codec);
>    Py_XDECREF(name);
>    return NULL;
>}
>
>If I figure out a working patch I'll post it here
>
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue23496>
>_______________________________________
msg240423 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-10 14:52
I think the android_segfault_fix patch doesn't include main.c and python.c...which use _PyMem_RawStrdup and also need patching...
msg240489 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-11 18:01
FYI, even with patches applied to main.c and python.c, the newly-built python binary segfaults in the same location.

I'll tear down and re-do everything (git clone master, patch, configure and make) but I may not get to it until next weekend.
msg240822 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-14 01:16
Unfortunately starting from scratch did not solve the issue.  Still getting a segfault when the newly built binary tries to run sysconfig
msg241154 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-04-15 20:23
Can you try to see exactly *which* line of the function is segfaulting?
msg241385 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-18 00:10
Ryan,
Here's the output from addr2line

bld/python/cpython-master/cpython $ addr2line -C -f -e /lib/libpython3.5m.so.1.0 0008f42c
_PyMem_RawStrdup
/bld/python/cpython-master/cpython/Objects/obmalloc.c:358
msg241427 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-04-18 16:59
Ok...try going to Python/pylifecycle.c and changing lines 220-230 from:

#elif defined(HAVE_LANGINFO_H) && defined(CODESET)
    char* codeset = nl_langinfo(CODESET);
    if (!codeset || codeset[0] == '\0') {
        PyErr_SetString(PyExc_ValueError, "CODESET is not set or empty");
        return NULL;
    }
    return get_codec_name(codeset);
#elif defined(__ANDROID__)
    char* m = malloc(6);
    strcpy(m, "ascii");
    return m;

to:

#elif defined(__ANDROID__)
    char* m = malloc(6);
    strcpy(m, "ascii");
    return m;
#elif defined(HAVE_LANGINFO_H) && defined(CODESET)
    char* codeset = nl_langinfo(CODESET);
    if (!codeset || codeset[0] == '\0') {
        PyErr_SetString(PyExc_ValueError, "CODESET is not set or empty");
        return NULL;
    }
    return get_codec_name(codeset);

I just swapped the `elif`'s around.
msg241432 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-18 17:39
Do you have the time/means to create a quick patch for that?

I ask because even a simple flip like that becomes a major pain when working with nano on a tablet.

If not, I'll start on it. Just thought I"d ask
msg241437 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-04-18 18:08
Here:

diff -r 38f5b3beeb2a Python/pylifecycle.c
--- a/Python/pylifecycle.c Thu Mar 19 15:16:03 2015 -0500
+++ b/Python/pylifecycle.c Sat Apr 18 13:07:36 2015 -0500
@@ -217,6 +217,10 @@
     char codepage[100];
     PyOS_snprintf(codepage, sizeof(codepage), "cp%d", GetACP());
     return get_codec_name(codepage);
+#elif defined(__ANDROID__)
+    char* m = malloc(6);
+    strcpy(m, "ascii");
+    return m;
 #elif defined(HAVE_LANGINFO_H) && defined(CODESET)
     char* codeset = nl_langinfo(CODESET);
     if (!codeset || codeset[0] == '\0') {
@@ -224,10 +228,6 @@
         return NULL;
     }
     return get_codec_name(codeset);
-#elif defined(__ANDROID__)
-    char* m = malloc(6);
-    strcpy(m, "ascii");
-    return m;
 #else
     PyErr_SetNone(PyExc_NotImplementedError);
     return NULL;

On Sat, Apr 18, 2015 at 12:39 PM, Cyd Haselton <report@bugs.python.org>
wrote:

>
> Cyd Haselton added the comment:
>
> Do you have the time/means to create a quick patch for that?
>
> I ask because even a simple flip like that becomes a major pain when
> working with nano on a tablet.
>
> If not, I'll start on it. Just thought I"d ask
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue23496>
> _______________________________________
>
msg241457 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-18 20:38
Ryan,
Sorry...same problem.

Segmentation fault
generate-posix-vars failed
make: *** [pybuilddir.txt] Error 1
/bld/python/cpython-master/cpython $ addr2line -C -f -e /lib/libpython3.5m.so.1.0 0008f42c
_PyMem_RawStrdup
/bld/python/cpython-master/cpython/Objects/obmalloc.c:358
msg241574 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-20 00:16
Ryan, 
Found the missing fix.

In ./Programs/python.c

#ifndef __ANDROID__
    oldloc = _PyMem_RawStrdup(setlocale(LC_ALL, NULL));
    if (!oldloc) {
        fprintf(stderr, "out of memory\n");                                                  return 1;
    }                                                                                #endif

Added the #ifndef __ANDROID__ #endif around lines 46-51

I'm now looking at the following syntax error:

File "./setup.py", line 1950
    elif host_platform.startswith('arm-linux')
                                             ^
SyntaxError: invalid syntax
make: *** [sharedmods] Error 1
msg241576 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-04-20 00:25
On Sun, Apr 19, 2015 at 7:16 PM, Cyd Haselton <report@bugs.python.org>
wrote:

>
> Cyd Haselton added the comment:
>
> Ryan,
> Found the missing fix.
>
> In ./Programs/python.c
>
> #ifndef __ANDROID__
>     oldloc = _PyMem_RawStrdup(setlocale(LC_ALL, NULL));
>     if (!oldloc) {
>         fprintf(stderr, "out of memory\n");
>                   return 1;
>     }
>           #endif
>
> Added the #ifndef __ANDROID__ #endif around lines 46-51
>
>
...that was in android_segfault_fix.patch...

are you sure you're on the same commit as I am?

> I'm now looking at the following syntax error:
>
> File "./setup.py", line 1950
>     elif host_platform.startswith('arm-linux')
>                                              ^
> SyntaxError: invalid syntax
> make: *** [sharedmods] Error 1
>
>
I forgot to put a colon and the end of the line. I'll upload a fixed
"kbox_fixes.patch".

> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue23496>
> _______________________________________
>
msg241577 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-20 00:30
Patch for python.c that prevents segfault on Android
msg241578 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-20 00:39
Ryan,
That fix is in the android_segfault patch, but it's for frozenmain.c not python.c

I cloned from master on Fri/Sat. Will double-check commit tomorrow but I think the problem is with the unpatched python.c
msg241579 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-04-20 00:40
That's the thing; my repo has no python.c!

On April 19, 2015 7:39:19 PM CDT, Cyd Haselton <report@bugs.python.org> wrote:
>
>Cyd Haselton added the comment:
>
>Ryan,
>That fix is in the android_segfault patch, but it's for frozenmain.c
>not python.c
>
>I cloned from master on Fri/Sat. Will double-check commit tomorrow but
>I think the problem is with the unpatched python.c
>
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue23496>
>_______________________________________
msg241634 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-20 11:33
Ryan,
There's not a python.c in the ./Programs file?
msg241642 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-20 13:27
FYI, I'm on commit c917493dc4ea2c32371da861aca2235f0a08e68e
msg241657 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-04-20 14:54
Nevermind. I was just being stupid. I kept searching the "Python" directory
for references to get_codec_name and _PyMem_RawStrdup and completely missed
the Programs directory. Sorry.

Did the updated kbox_fix.patch work?
msg241663 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-20 15:38
Ryan,
Completely forgot to download and apply it, but based on what I see there are a few things that need to be corrected.

1) -python3.4m needs to be changed to -python3.5m...or the appropriate versioning variable.
2) colons at the end of the elif lines in setup.py (1950- )
3) The _crypt and readline module builds in setup.py need additional libraries to work...python3.5m for both and possibly ncurses for readline, I'll test and get back to you.

With #1 and #2 added, the build and install completes successfully, although you obviously can't import readline and _crypt.
msg241805 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-22 14:02
Still working at this; I can't get the readline module to work.  It is built successfully but importing it produces the following error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: dlopen failed: could not load library "libreadline.so.6" needed by "readline.cpython-35m-arm-linux-gnueabi.so"; caused by cannot locate symbol "tgetnum" referenced by "libreadline.so.6"...

tgetnum is undefined in libreadline but defined in -libncurses and libtinfo. I've built and re-built readline (--with-curses and -without that option...linking -lncurses, then -lncurses and -ltinfo) in the KBOX environment. I've edited Modules/Setup and setup.py so as to specify the readline library plus a combination of others (-ncurses, -tinfo) to no avail.
msg241862 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-23 10:57
Finally got readline to work with this build, but since the fixes are to the readline source tree I'm not sure if they should go here.

With that plus whatever fixes to patches I suggested previously, I think this build works. What would be the next steps for this...running tests?
msg242062 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-04-26 17:12
On a related note, I managed to get pip working with this build...minus some errors with verbose mode.  I first had to make some edits to setup.py and Modules/Setup so that the build would find and make the _ssl and lzip modules. After running make install, I ran 'python -m ensurepip'. I tested it by searching for and installing the BeautifulSoup module.

Ryan,
Let me know what are the next steps for this issue/build...and I'll do my best to get them done.
msg244366 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-05-29 10:54
All,
Just checking in as it has been a while...is there anything I need to do on my end for this port?
msg244387 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-05-29 15:52
I thought you were running the tests...

Or was there something I had to do that I missed?

On May 29, 2015 5:54:18 AM CDT, Cyd Haselton <report@bugs.python.org> wrote:
>
>Cyd Haselton added the comment:
>
>All,
>Just checking in as it has been a while...is there anything I need to
>do on my end for this port?
>
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue23496>
>_______________________________________
msg244389 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-05-29 16:38
On May 29, 2015 10:52:23 AM CDT, Ryan Gonzalez <report@bugs.python.org> wrote:
>
>Ryan Gonzalez added the comment:
>
>I thought you were running the tests...
>

Which tests?

>Or was there something I had to do that I missed?

From one of the earlier messages I posted:

1) -python3.4m needs to be changed to -python3.5m...or the appropriate versioning variable. 
2) colons at the end of the elif lines in setup.py (1950- ) 
3) The _crypt and readline module builds in setup.py need additional libraries to work...python3.5m for both and possibly ncurses for readline, I'll test and get back to you. With #1 and #2 added, the build and install completes successfully, although you obviously can't import readline and _crypt.

*snip*

The messages after that one document how I got readline to work
>
>On May 29, 2015 5:54:18 AM CDT, Cyd Haselton <report@bugs.python.org>
>wrote:
>>
>>Cyd Haselton added the comment:
>>
>>All,
>>Just checking in as it has been a while...is there anything I need to
>>do on my end for this port?
>>
>>----------
>>
>>_______________________________________
>>Python tracker <report@bugs.python.org>
>><http://bugs.python.org/issue23496>
>>_______________________________________
>
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue23496>
>_______________________________________
msg244845 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-06-04 21:52
#1 and #2 are fixed. I hand-edited the patch file (!!), so here's to hoping it'll work...

Working on #3. You probably should try the tests now (which is obviously the scary part ;).
msg244846 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-06-04 21:56
Fixes for readline and _crypt done.
msg244859 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-06-05 11:22
Will do...if by "try the tests now" you mean "run 'make tests'"...but it may be about a week or two as a) I'm currently in the last stages of a mono port and b) will be working on a PIE gcc for android 5.0 after, so that I have a KBOX environment to work with after verizon updates my tablet.

If I can keep putting off the verizon 5.0 update

On June 4, 2015 4:52:59 PM CDT, Ryan Gonzalez <report@bugs.python.org> wrote:
>
>Ryan Gonzalez added the comment:
>
>#1 and #2 are fixed. I hand-edited the patch file (!!), so here's to
>hoping it'll work...
>
>Working on #3. You probably should try the tests now (which is
>obviously the scary part ;).
>
>----------
>Added file: http://bugs.python.org/file39628/kbox_fix.patch
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue23496>
>_______________________________________
msg244868 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-06-05 14:57
On June 5, 2015 6:22:07 AM CDT, Cyd Haselton <report@bugs.python.org> wrote:
>
>Cyd Haselton added the comment:
>
>Will do...if by "try the tests now" you mean "run 'make tests'"

Pretty much.

>...but it may be about a week or two as a) I'm currently in the last stages of
>a mono port and b) will be working on a PIE gcc for android 5.0 after,
>so that I have a KBOX environment to work with after verizon updates my
>tablet.

Try CCTools. Works on Android 5.0. Clang can be a little broken, but with a bit of flag twiddling it works.

>
>If I can keep putting off the verizon 5.0 update
>
>On June 4, 2015 4:52:59 PM CDT, Ryan Gonzalez <report@bugs.python.org>
>wrote:
>>
>>Ryan Gonzalez added the comment:
>>
>>#1 and #2 are fixed. I hand-edited the patch file (!!), so here's to
>>hoping it'll work...
>>
>>Working on #3. You probably should try the tests now (which is
>>obviously the scary part ;).
>>
>>----------
>>Added file: http://bugs.python.org/file39628/kbox_fix.patch
>>
>>_______________________________________
>>Python tracker <report@bugs.python.org>
>><http://bugs.python.org/issue23496>
>>_______________________________________
>
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue23496>
>_______________________________________
msg245445 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-06-17 20:15
Update:
Now that I;ve finished porting a much-needed gdb to my device, I should have time to tackle patch and testing this weekend.  Will post results when I have them.
msg245674 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-06-23 09:50
UPDATE:
Spent this past weekend fixing the broken on-device compiler.  Will get to tests this weekend
msg245894 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-06-27 22:51
I've run into an error when building python in preparation for running tests (i. e. with --with-pydebug in the ./configure params).

./libpython3.4dm.so: undefined reference to `_PyUnicode_CheckConsistency'

It doesn't seem to be defined anywhere in source that I can see and i'm not sure why.

Google turns up a few hits from the bug-tracker but none have resolutions
http://bugs.python.org/issue13869
http://bugs.python.org/msg151986

Any thoughts?
msg245932 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-06-29 12:09
FYI, Figured out that running ./configure with --with-pydebug does NOT define Py_DEBUG in pyconfig.h.

Defining it in pyconfig.h got me past the "undefined reference to `_PyUnicode_CheckConsistency'" error, but i'm now getting this:

if test $? -ne 0 ; then \
        echo "generate-posix-vars failed" ; \
        rm -f ./pybuilddir.txt ; \
        exit 1 ; \
fi
Debug memory block at address p=0x90b7b0: API ''
    2416312320 bytes originally requested
    The 3 pad bytes at p-3 are not all FORBIDDENBYTE (0xfb):
        at p-3: 0x00 *** OUCH
        at p-2: 0x00 *** OUCH
        at p-1: 0x00 *** OUCH
    Because memory is corrupted at the start, the count of bytes requested
       may be bogus, and checking the trailing pad bytes may segfault.
    The 4 pad bytes at tail=0x9096b7b0 are Segmentation fault
generate-posix-vars failed
make: *** [pybuilddir.txt] Error 1

Will re-check patch application and try again
msg245937 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-06-29 14:00
You compiled with -fPIE and GCC, right? I know the Android Clang seems broken.

On June 29, 2015 7:09:25 AM CDT, Cyd Haselton <report@bugs.python.org> wrote:
>
>Cyd Haselton added the comment:
>
>FYI, Figured out that running ./configure with --with-pydebug does NOT
>define Py_DEBUG in pyconfig.h.
>
>Defining it in pyconfig.h got me past the "undefined reference to
>`_PyUnicode_CheckConsistency'" error, but i'm now getting this:
>
>if test $? -ne 0 ; then \
>        echo "generate-posix-vars failed" ; \
>        rm -f ./pybuilddir.txt ; \
>        exit 1 ; \
>fi
>Debug memory block at address p=0x90b7b0: API ''
>    2416312320 bytes originally requested
>    The 3 pad bytes at p-3 are not all FORBIDDENBYTE (0xfb):
>        at p-3: 0x00 *** OUCH
>        at p-2: 0x00 *** OUCH
>        at p-1: 0x00 *** OUCH
> Because memory is corrupted at the start, the count of bytes requested
>       may be bogus, and checking the trailing pad bytes may segfault.
>    The 4 pad bytes at tail=0x9096b7b0 are Segmentation fault
>generate-posix-vars failed
>make: *** [pybuilddir.txt] Error 1
>
>Will re-check patch application and try again
>
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue23496>
>_______________________________________
msg245938 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-06-29 14:29
No...haven't upgraded to Android 5.0 yet.  

On June 29, 2015 9:00:01 AM CDT, Ryan Gonzalez <report@bugs.python.org> wrote:
>
>Ryan Gonzalez added the comment:
>
>You compiled with -fPIE and GCC, right? I know the Android Clang seems
>broken.
>
>On June 29, 2015 7:09:25 AM CDT, Cyd Haselton <report@bugs.python.org>
>wrote:
>>
>>Cyd Haselton added the comment:
>>
>>FYI, Figured out that running ./configure with --with-pydebug does NOT
>>define Py_DEBUG in pyconfig.h.
>>
>>Defining it in pyconfig.h got me past the "undefined reference to
>>`_PyUnicode_CheckConsistency'" error, but i'm now getting this:
>>
>>if test $? -ne 0 ; then \
>>        echo "generate-posix-vars failed" ; \
>>        rm -f ./pybuilddir.txt ; \
>>        exit 1 ; \
>>fi
>>Debug memory block at address p=0x90b7b0: API ''
>>    2416312320 bytes originally requested
>>    The 3 pad bytes at p-3 are not all FORBIDDENBYTE (0xfb):
>>        at p-3: 0x00 *** OUCH
>>        at p-2: 0x00 *** OUCH
>>        at p-1: 0x00 *** OUCH
>> Because memory is corrupted at the start, the count of bytes
>requested
>>       may be bogus, and checking the trailing pad bytes may segfault.
>>    The 4 pad bytes at tail=0x9096b7b0 are Segmentation fault
>>generate-posix-vars failed
>>make: *** [pybuilddir.txt] Error 1
>>
>>Will re-check patch application and try again
>>
>>----------
>>
>>_______________________________________
>>Python tracker <report@bugs.python.org>
>><http://bugs.python.org/issue23496>
>>_______________________________________
>
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue23496>
>_______________________________________
msg245988 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-06-30 13:14
Your question about -fPIE brings up a question: How should the differences between Android 5 and previous versions be handled in regards to this issue?

Other than mandatory -fPIE, there are changes to Android[s libc that may make  patches for python on Android 4.x incompatible with Android 5.

Thoughts?

On June 29, 2015 9:29:35 AM CDT, Cyd Haselton <report@bugs.python.org> wrote:
>
>Cyd Haselton added the comment:
>
>No...haven't upgraded to Android 5.0 yet.  
>
>On June 29, 2015 9:00:01 AM CDT, Ryan Gonzalez <report@bugs.python.org>
>wrote:
>>
>>Ryan Gonzalez added the comment:
>>
>>You compiled with -fPIE and GCC, right? I know the Android Clang seems
>>broken.
>>
>>On June 29, 2015 7:09:25 AM CDT, Cyd Haselton <report@bugs.python.org>
>>wrote:
>>>
>>>Cyd Haselton added the comment:
>>>
>>>FYI, Figured out that running ./configure with --with-pydebug does
>NOT
>>>define Py_DEBUG in pyconfig.h.
>>>
>>>Defining it in pyconfig.h got me past the "undefined reference to
>>>`_PyUnicode_CheckConsistency'" error, but i'm now getting this:
>>>
>>>if test $? -ne 0 ; then \
>>>        echo "generate-posix-vars failed" ; \
>>>        rm -f ./pybuilddir.txt ; \
>>>        exit 1 ; \
>>>fi
>>>Debug memory block at address p=0x90b7b0: API ''
>>>    2416312320 bytes originally requested
>>>    The 3 pad bytes at p-3 are not all FORBIDDENBYTE (0xfb):
>>>        at p-3: 0x00 *** OUCH
>>>        at p-2: 0x00 *** OUCH
>>>        at p-1: 0x00 *** OUCH
>>> Because memory is corrupted at the start, the count of bytes
>>requested
>>>       may be bogus, and checking the trailing pad bytes may
>segfault.
>>>    The 4 pad bytes at tail=0x9096b7b0 are Segmentation fault
>>>generate-posix-vars failed
>>>make: *** [pybuilddir.txt] Error 1
>>>
>>>Will re-check patch application and try again
>>>
>>>----------
>>>
>>>_______________________________________
>>>Python tracker <report@bugs.python.org>
>>><http://bugs.python.org/issue23496>
>>>_______________________________________
>>
>>----------
>>
>>_______________________________________
>>Python tracker <report@bugs.python.org>
>><http://bugs.python.org/issue23496>
>>_______________________________________
>
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue23496>
>_______________________________________
msg245990 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-06-30 14:20
On June 30, 2015 8:14:34 AM CDT, Cyd Haselton <report@bugs.python.org> wrote:
>
>Cyd Haselton added the comment:
>
>Your question about -fPIE brings up a question: How should the
>differences between Android 5 and previous versions be handled in
>regards to this issue?
>
>Other than mandatory -fPIE, there are changes to Android[s libc that
>may make  patches for python on Android 4.x incompatible with Android
>5.
>

WHAT??? :( I can't find a list anywhere; how do you know?

>Thoughts?

Not really...

>
>On June 29, 2015 9:29:35 AM CDT, Cyd Haselton <report@bugs.python.org>
>wrote:
>>
>>Cyd Haselton added the comment:
>>
>>No...haven't upgraded to Android 5.0 yet.  
>>
>>On June 29, 2015 9:00:01 AM CDT, Ryan Gonzalez
><report@bugs.python.org>
>>wrote:
>>>
>>>Ryan Gonzalez added the comment:
>>>
>>>You compiled with -fPIE and GCC, right? I know the Android Clang
>seems
>>>broken.
>>>
>>>On June 29, 2015 7:09:25 AM CDT, Cyd Haselton
><report@bugs.python.org>
>>>wrote:
>>>>
>>>>Cyd Haselton added the comment:
>>>>
>>>>FYI, Figured out that running ./configure with --with-pydebug does
>>NOT
>>>>define Py_DEBUG in pyconfig.h.
>>>>
>>>>Defining it in pyconfig.h got me past the "undefined reference to
>>>>`_PyUnicode_CheckConsistency'" error, but i'm now getting this:
>>>>
>>>>if test $? -ne 0 ; then \
>>>>        echo "generate-posix-vars failed" ; \
>>>>        rm -f ./pybuilddir.txt ; \
>>>>        exit 1 ; \
>>>>fi
>>>>Debug memory block at address p=0x90b7b0: API ''
>>>>    2416312320 bytes originally requested
>>>>    The 3 pad bytes at p-3 are not all FORBIDDENBYTE (0xfb):
>>>>        at p-3: 0x00 *** OUCH
>>>>        at p-2: 0x00 *** OUCH
>>>>        at p-1: 0x00 *** OUCH
>>>> Because memory is corrupted at the start, the count of bytes
>>>requested
>>>>       may be bogus, and checking the trailing pad bytes may
>>segfault.
>>>>    The 4 pad bytes at tail=0x9096b7b0 are Segmentation fault
>>>>generate-posix-vars failed
>>>>make: *** [pybuilddir.txt] Error 1
>>>>
>>>>Will re-check patch application and try again
>>>>
>>>>----------
>>>>
>>>>_______________________________________
>>>>Python tracker <report@bugs.python.org>
>>>><http://bugs.python.org/issue23496>
>>>>_______________________________________
>>>
>>>----------
>>>
>>>_______________________________________
>>>Python tracker <report@bugs.python.org>
>>><http://bugs.python.org/issue23496>
>>>_______________________________________
>>
>>----------
>>
>>_______________________________________
>>Python tracker <report@bugs.python.org>
>><http://bugs.python.org/issue23496>
>>_______________________________________
>
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue23496>
>_______________________________________
msg245993 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-06-30 15:09
On June 30, 2015 9:20:45 AM CDT, Ryan Gonzalez <report@bugs.python.org> wrote:
>
>Ryan Gonzalez added the comment:
>
>On June 30, 2015 8:14:34 AM CDT, Cyd Haselton <report@bugs.python.org>
>wrote:
>>
>>Cyd Haselton added the comment:
>>
>>Your question about -fPIE brings up a question: How should the
>>differences between Android 5 and previous versions be handled in
>>regards to this issue?
>>
>>Other than mandatory -fPIE, there are changes to Android[s libc that
>>may make  patches for python on Android 4.x incompatible with Android
>>5.
>>
>
>WHAT??? :( I can't find a list anywhere; how do you know?
>

I tried building GCC for Android 5 on device and ran into a bunch of errors like 'undefined symbol __swbuf.'  Googling the errors led to this page: https://developer.android.com/ndk/downloads/revision_history.html.

Relevant Part here:

" Removed the following symbols from all architectures: get_malloc_leak_info,free_malloc_leak_info, __srget, __swbuf, __srefill,__swsetup, __sdidinit, __sflags, __sfp, __sinit, __smakebuf,__sflush, __sread, __swrite, __sseek, __sclose, _fwalk,__sglue, __get_thread, __wait4, __futex_wake, __open,__get_tls, __getdents64, and dlmalloc."

>>Thoughts?
>
>Not really...

Would setting up two repos be advisable...one for 5 and one for 4.x and earlier?
>
>>
>>On June 29, 2015 9:29:35 AM CDT, Cyd Haselton <report@bugs.python.org>
>>wrote:
>>>
>>>Cyd Haselton added the comment:
>>>
>>>No...haven't upgraded to Android 5.0 yet.  
>>>
>>>On June 29, 2015 9:00:01 AM CDT, Ryan Gonzalez
>><report@bugs.python.org>
>>>wrote:
>>>>
>>>>Ryan Gonzalez added the comment:
>>>>
>>>>You compiled with -fPIE and GCC, right? I know the Android Clang
>>seems
>>>>broken.
>>>>
>>>>On June 29, 2015 7:09:25 AM CDT, Cyd Haselton
>><report@bugs.python.org>
>>>>wrote:
>>>>>
>>>>>Cyd Haselton added the comment:
>>>>>
>>>>>FYI, Figured out that running ./configure with --with-pydebug does
>>>NOT
>>>>>define Py_DEBUG in pyconfig.h.
>>>>>
>>>>>Defining it in pyconfig.h got me past the "undefined reference to
>>>>>`_PyUnicode_CheckConsistency'" error, but i'm now getting this:
>>>>>
>>>>>if test $? -ne 0 ; then \
>>>>>        echo "generate-posix-vars failed" ; \
>>>>>        rm -f ./pybuilddir.txt ; \
>>>>>        exit 1 ; \
>>>>>fi
>>>>>Debug memory block at address p=0x90b7b0: API ''
>>>>>    2416312320 bytes originally requested
>>>>>    The 3 pad bytes at p-3 are not all FORBIDDENBYTE (0xfb):
>>>>>        at p-3: 0x00 *** OUCH
>>>>>        at p-2: 0x00 *** OUCH
>>>>>        at p-1: 0x00 *** OUCH
>>>>> Because memory is corrupted at the start, the count of bytes
>>>>requested
>>>>>       may be bogus, and checking the trailing pad bytes may
>>>segfault.
>>>>>    The 4 pad bytes at tail=0x9096b7b0 are Segmentation fault
>>>>>generate-posix-vars failed
>>>>>make: *** [pybuilddir.txt] Error 1
>>>>>
>>>>>Will re-check patch application and try again
>>>>>
>>>>>----------
>>>>>
>>>>>_______________________________________
>>>>>Python tracker <report@bugs.python.org>
>>>>><http://bugs.python.org/issue23496>
>>>>>_______________________________________
>>>>
>>>>----------
>>>>
>>>>_______________________________________
>>>>Python tracker <report@bugs.python.org>
>>>><http://bugs.python.org/issue23496>
>>>>_______________________________________
>>>
>>>----------
>>>
>>>_______________________________________
>>>Python tracker <report@bugs.python.org>
>>><http://bugs.python.org/issue23496>
>>>_______________________________________
>>
>>----------
>>
>>_______________________________________
>>Python tracker <report@bugs.python.org>
>><http://bugs.python.org/issue23496>
>>_______________________________________
>
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue23496>
>_______________________________________
msg246938 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-07-19 10:40
UPDATE:
Haven't forgotten about this; I'm currently (thanks to Android's new mandatory PIE binaries requirement) rebuilding all of the necessary utilities (openssl, curl, git, etc) so that I can clone and test.

Between the above and a sharp increase in workload at the day job, expect a few weeks delay between now and continued work on this issue.
msg247118 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-07-22 13:51
UPDATE:
Build environment is up and running; cloning repo now.
msg247232 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-07-23 22:14
Build complete. Unfortunately while some of the tests complete successfully, the run ends in a segfault (see attached log)
msg247235 - (view) Author: Russell Keith-Magee (freakboy3742) * Date: 2015-07-24 00:10
What hardware architecture are you compiling for? If it's ARM64, and you're not using a trunk version of libffi, that segfault in test_ctypes is to be expected.
msg247269 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-07-24 10:41
I'm compiling for ARM, not ARM64, on an armv7 device.

On July 23, 2015 7:10:35 PM CDT, Russell Keith-Magee <report@bugs.python.org> wrote:
>
>Russell Keith-Magee added the comment:
>
>What hardware architecture are you compiling for? If it's ARM64, and
>you're not using a trunk version of libffi, that segfault in
>test_ctypes is to be expected.
>
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue23496>
>_______________________________________
msg247310 - (view) Author: Russell Keith-Magee (freakboy3742) * Date: 2015-07-25 03:18
Are you using the libffi sources vendored into the Python source tree, or a more recent version? I can verify that libffi v3.2 works on ARMv7 (on iOS, anyway), and there's been plenty of changes to the ARM source tree since the Python version was vendored in.
msg247346 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-07-25 11:56
I assume so; I'm using whatever is pulled frommy fork of the git repo.

I've actually run into a different error that occured when i rebuilt to include readline support; when I run ./python -m test I;m getting "ImportError: no module named _struct found."

Trying to figure that one out first...
msg247359 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-07-25 17:38
So...I have *no* clue why _struct can't be found.

Can you use gdb to get the segfault backtrace like you did before?

On July 25, 2015 6:56:10 AM CDT, Cyd Haselton <report@bugs.python.org> wrote:
>
>Cyd Haselton added the comment:
>
>I assume so; I'm using whatever is pulled frommy fork of the git repo.
>
>I've actually run into a different error that occured when i rebuilt to
>include readline support; when I run ./python -m test I;m getting
>"ImportError: no module named _struct found."
>
>Trying to figure that one out first...
>
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue23496>
>_______________________________________
msg247364 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-07-25 18:32
As soon as I build a PIE gdb I can.

And right now, running ./python -m test throws the ImportError...so I'll
need to resolve that first
On Jul 25, 2015 12:38 PM, "Ryan Gonzalez" <report@bugs.python.org> wrote:

>
> Ryan Gonzalez added the comment:
>
> So...I have *no* clue why _struct can't be found.
>
> Can you use gdb to get the segfault backtrace like you did before?
>
> On July 25, 2015 6:56:10 AM CDT, Cyd Haselton <report@bugs.python.org>
> wrote:
> >
> >Cyd Haselton added the comment:
> >
> >I assume so; I'm using whatever is pulled frommy fork of the git repo.
> >
> >I've actually run into a different error that occured when i rebuilt to
> >include readline support; when I run ./python -m test I;m getting
> >"ImportError: no module named _struct found."
> >
> >Trying to figure that one out first...
> >
> >----------
> >
> >_______________________________________
> >Python tracker <report@bugs.python.org>
> ><http://bugs.python.org/issue23496>
> >_______________________________________
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue23496>
> _______________________________________
>
msg247584 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-07-29 17:06
Finally found a hacky fix for the "no module named _struct found"; I copied all Modules/*.cpython-34m.so objects to Lib.

Unfortunately I still get a segfault error when test_ctypes is run:

[ 83/390/11] test_ctypes
Fatal Python error: Segmentation fault

Current thread 0xb6f2bec8 (most recent call first):
  File "/bld/pyt/cpython-android/Lib/ctypes/test/test_as_parameter.py", line 85 in test_callbacks
  File "/bld/pyt/cpython-android/Lib/unittest/case.py", line 577 in run
  File "/bld/pyt/cpython-android/Lib/unittest/case.py", line 625 in __call__
  File "/bld/pyt/cpython-android/Lib/unittest/suite.py", line 122 in run
  File "/bld/pyt/cpython-android/Lib/unittest/suite.py", line 84 in __call__
  File "/bld/pyt/cpython-android/Lib/unittest/suite.py", line 122 in run
  File "/bld/pyt/cpython-android/Lib/unittest/suite.py", line 84 in __call__
  File "/bld/pyt/cpython-android/Lib/unittest/suite.py", line 122 in run
  File "/bld/pyt/cpython-android/Lib/unittest/suite.py", line 84 in __call__
  File "/bld/pyt/cpython-android/Lib/unittest/suite.py", line 122 in run
  File "/bld/pyt/cpython-android/Lib/unittest/suite.py", line 84 in __call__
  File "/bld/pyt/cpython-android/Lib/test/support/__init__.py", line 1668 in run
  File "/bld/pyt/cpython-android/Lib/test/support/__init__.py", line 1769 in _run_suite
  File "/bld/pyt/cpython-android/Lib/test/support/__init__.py", line 1803 in run_unittest
  File "/bld/pyt/cpython-android/Lib/test/regrtest.py", line 1279 in test_runner
  File "/bld/pyt/cpython-android/Lib/test/regrtest.py", line 1280 in runtest_inner
  File "/bld/pyt/cpython-android/Lib/test/regrtest.py", line 978 in runtest
  File "/bld/pyt/cpython-android/Lib/test/regrtest.py", line 763 in main
  File "/bld/pyt/cpython-android/Lib/test/regrtest.py", line 1564 in main_in_temp_cwd  File "/bld/pyt/cpython-android/Lib/test/__main__.py", line 3 in <module>
  File "/bld/pyt/cpython-android/Lib/runpy.py", line 85 in _run_code
  File "/bld/pyt/cpython-android/Lib/runpy.py", line 170 in _run_module_as_main
Segmentation fault
msg247713 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-07-30 23:58
A nominal bump before starting to re-build a PIE gdb to debug the segfault above
msg248452 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-08-12 11:21
After struggling to get helpful output from gdb it is looking like it will not be possible due to the lack of debugging symbols in the libs on the android device.

Still investigating.
msg248458 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-08-12 14:27
Doesn't Python still have debug symbols? The system ones don't matter too much.

On August 12, 2015 6:21:23 AM CDT, Cyd Haselton <report@bugs.python.org> wrote:
>
>Cyd Haselton added the comment:
>
>After struggling to get helpful output from gdb it is looking like it
>will not be possible due to the lack of debugging symbols in the libs
>on the android device.
>
>Still investigating.
>
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue23496>
>_______________________________________
msg248671 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-08-16 00:32
I thought porting gdb would be the difficult part of getting debug info. I was so wrong.

Here is what I have so far, after lengthy consultation of the gdb manual.  Bear with me...it's basically a copy & paste of gdb session output:

(gdb) run
Starting program: /bld/pyt/cpython-android/python
setpgrp failed in child: No such process
warning: Unable to find dynamic linker breakpoint function.
GDB will be unable to debug shared library initializers
and track explicitly loaded dynamic code.

Program received signal SIGILL, Illegal instruction.
0xb6a63cc8 in ?? ()
(gdb) f
#0  0xb6a63cc8 in ?? ()
(gdb) info f
Stack level 0, frame at 0xbefff618:
 pc = 0xb6a63cc8; saved pc = 0xb6a5feb0
 called by frame at 0xbefff618
 Arglist at 0xbefff618, args:
 Locals at 0xbefff618, Previous frame's sp is 0xbefff618
(gdb) info args
No symbol table info available.
(gdb) info local
No symbol table info available.
(gdb) info source
Current source file is ./Modules/python.c
Compilation directory is /bld/pyt/cpython-android
Located in /bld/pyt/cpython-android/Modules/python.c
Contains 80 lines.
Source language is c.
Compiled with DWARF 2 debugging format.
Does not include preprocessor macro info.
msg248672 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-08-16 01:05
What if you run:

bt

?

On August 15, 2015 7:32:37 PM CDT, Cyd Haselton <report@bugs.python.org> wrote:
>
>Cyd Haselton added the comment:
>
>I thought porting gdb would be the difficult part of getting debug
>info. I was so wrong.
>
>Here is what I have so far, after lengthy consultation of the gdb
>manual.  Bear with me...it's basically a copy & paste of gdb session
>output:
>
>(gdb) run
>Starting program: /bld/pyt/cpython-android/python
>setpgrp failed in child: No such process
>warning: Unable to find dynamic linker breakpoint function.
>GDB will be unable to debug shared library initializers
>and track explicitly loaded dynamic code.
>
>Program received signal SIGILL, Illegal instruction.
>0xb6a63cc8 in ?? ()
>(gdb) f
>#0  0xb6a63cc8 in ?? ()
>(gdb) info f
>Stack level 0, frame at 0xbefff618:
> pc = 0xb6a63cc8; saved pc = 0xb6a5feb0
> called by frame at 0xbefff618
> Arglist at 0xbefff618, args:
> Locals at 0xbefff618, Previous frame's sp is 0xbefff618
>(gdb) info args
>No symbol table info available.
>(gdb) info local
>No symbol table info available.
>(gdb) info source
>Current source file is ./Modules/python.c
>Compilation directory is /bld/pyt/cpython-android
>Located in /bld/pyt/cpython-android/Modules/python.c
>Contains 80 lines.
>Source language is c.
>Compiled with DWARF 2 debugging format.
>Does not include preprocessor macro info.
>
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue23496>
>_______________________________________
msg248730 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-08-17 14:24
Result:

(gdb) bt
#0  0xb6a63cc8 in ?? ()
#1  0xb6a5feb0 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
msg248731 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-08-17 14:39
Wait, did you compile this with Clang?

On August 17, 2015 9:24:50 AM CDT, Cyd Haselton <report@bugs.python.org> wrote:
>
>Cyd Haselton added the comment:
>
>Result:
>
>(gdb) bt
>#0  0xb6a63cc8 in ?? ()
>#1  0xb6a5feb0 in ?? ()
>Backtrace stopped: previous frame identical to this frame (corrupt
>stack?)
>
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue23496>
>_______________________________________
msg248733 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-08-17 15:49
No...compiled with GCC 4.9.2

On August 17, 2015 9:39:55 AM CDT, Ryan Gonzalez <report@bugs.python.org> wrote:
>
>Ryan Gonzalez added the comment:
>
>Wait, did you compile this with Clang?
>
>On August 17, 2015 9:24:50 AM CDT, Cyd Haselton
><report@bugs.python.org> wrote:
>>
>>Cyd Haselton added the comment:
>>
>>Result:
>>
>>(gdb) bt
>>#0  0xb6a63cc8 in ?? ()
>>#1  0xb6a5feb0 in ?? ()
>>Backtrace stopped: previous frame identical to this frame (corrupt
>>stack?)
>>
>>----------
>>
>>_______________________________________
>>Python tracker <report@bugs.python.org>
>><http://bugs.python.org/issue23496>
>>_______________________________________
>
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue23496>
>_______________________________________
msg248909 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-08-20 18:09
A few questions for Russell:

>What hardware architecture are you compiling for? If it's ARM64, and you're not using a trunk version of libffi, that segfault in test_ctypes is to be expected.

Does this mean I can safely ignore the segfault?

>Are you using the libffi sources vendored into the Python source tree, or a more recent version? 

By 'vendored in' do you mean 'sources included in python source tree for building?'

>I can verify that libffi v3.2 works on ARMv7 (on iOS, anyway), and there's been plenty of changes to the ARM source tree since the Python version was vendored in.

Would your recommend downloading and building libffi from sources (on device) and then building python?

I'm asking the above questions because I've hit a fairly significant roadblock; I'm still getting the segfault when test_ctypes is run and I can't seem to get anything useful out of gdb.
msg248938 - (view) Author: Russell Keith-Magee (freakboy3742) * Date: 2015-08-21 01:03
>>What hardware architecture are you compiling for? If it's ARM64, and you're not using a trunk version of libffi, that segfault in test_ctypes is to be expected.

> Does this mean I can safely ignore the segfault?

Well, "safely" in the sense that everything except a very recent version of libffi is known to not work on ARM64 - so if it doesn't work, it's not because of something Python is doing wrong, it's a problem with a dependency.

>>Are you using the libffi sources vendored into the Python source tree, or a more recent version? 

>By 'vendored in' do you mean 'sources included in python source tree for building?'

Correct. The libffi source code that is in the Python source tree is quite old, and *definitely* doesn't work on ARM64. I'm not even sure that it works on ARMv7.

>>I can verify that libffi v3.2 works on ARMv7 (on iOS, anyway), and there's been plenty of changes to the ARM source tree since the Python version was vendored in.

> Would your recommend downloading and building libffi from sources (on device) and then building python?

Well, for starters - as I've said before, I'd recommend not compiling on device at all, but that's a separate issue. 

However, regardless of where you're compiling, you can either use an external libffi, or you can do what I've done in the iOS patch - update the Python source tree to include a newer version of libffi. If you update the source in the Python tree, you need to use 2 versions (or a merged version); you need v3.2 sources for ARMv7, and trunk sources for ARM64. This is because libffi v3.2 doesn't work for ARM64, and trunk doesn't even compile for ARMv7. See the iOS patch for details.

> I'm asking the above questions because I've hit a fairly significant roadblock; I'm still getting the segfault when test_ctypes is run and I can't seem to get anything useful out of gdb.

Personally, I've pretty much given up on CPython on Android. Even when I got it working, the performance of the JNI layer is *abysmal*, and severely crippled. If you're planning to actually interact with the device in any way (like, say, put a button on the screen), that's a big problem. 

I'm working on an approach that uses Java natively - think "Jython for Android". I'm still a way off having anything working, though.
msg248954 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-08-21 14:25
Question for Ryan Gonzalez:
Given this information...

On August 20, 2015 8:03:13 PM CDT, Russell Keith-Magee <report@bugs.python.org> wrote:
>
>Russell Keith-Magee added the comment:
>
>>>What hardware architecture are you compiling for? If it's ARM64, and
>you're not using a trunk version of libffi, that segfault in
>test_ctypes is to be expected.
>
>> Does this mean I can safely ignore the segfault?
>
>Well, "safely" in the sense that everything except a very recent
>version of libffi is known to not work on ARM64 - so if it doesn't
>work, it's not because of something Python is doing wrong, it's a
>problem with a dependency.
>

...do you know of a way to run all tests except _ctypes, so that Imcan verify everything else?

And to answer Russell's questions:

>>>Are you using the libffi sources vendored into the Python source
>tree, or a more recent version? 
>
>>By 'vendored in' do you mean 'sources included in python source tree
>for building?'
>
>Correct. The libffi source code that is in the Python source tree is
>quite old, and *definitely* doesn't work on ARM64. I'm not even sure
>that it works on ARMv7.
>
Definitely using the vendored in sources.

>
>> Would your recommend downloading and building libffi from sources (on
>device) and then building python?
>
>Well, for starters - as I've said before, I'd recommend not compiling
>on device at all, but that's a separate issue. 
>
Any particular reason why? The device I'm working with has a port of GCC which I've used to build other utilities that work well...given that I'm working on a tablet.

>However, regardless of where you're compiling, you can either use an
>external libffi, or you can do what I've done in the iOS patch - update
>the Python source tree to include a newer version of libffi. If you
>update the source in the Python tree, you need to use 2 versions (or a
>merged version); you need v3.2 sources for ARMv7, and trunk sources for
>ARM64. This is because libffi v3.2 doesn't work for ARM64, and trunk
>doesn't even compile for ARMv7. See the iOS patch for details.
>
Since I'm not compiling for ARM64...and have zero experience with hacking configure.ac files...would it be okay to include just the 3.2 sources if I note somewhere that this fork does not include ARM64 support?

>> I'm asking the above questions because I've hit a fairly significant
>roadblock; I'm still getting the segfault when test_ctypes is run and I
>can't seem to get anything useful out of gdb.
>
>Personally, I've pretty much given up on CPython on Android. Even when
>I got it working, the performance of the JNI layer is *abysmal*, and
>severely crippled. If you're planning to actually interact with the
>device in any way (like, say, put a button on the screen), that's a big
>problem. 
>
My goal is to build a port that operates on Android in a Linux-like environment; I'm currently using KBOX3.  I've no experience with the Python language so JNI interop is Greek to me.  For what i'm using the port for it functions fairly well.

>I'm working on an approach that uses Java natively - think "Jython for
>Android". I'm still a way off having anything working, though.
>
I'll keep an eye out for it once I've got Python basics under my belt.
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue23496>
>_______________________________________
msg248957 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-08-21 14:48
On August 21, 2015 9:25:10 AM CDT, Cyd Haselton <report@bugs.python.org> wrote:
>
>Cyd Haselton added the comment:
>
>Question for Ryan Gonzalez:
>Given this information...
>
>On August 20, 2015 8:03:13 PM CDT, Russell Keith-Magee
><report@bugs.python.org> wrote:
>>
>>Russell Keith-Magee added the comment:
>>
>>>>What hardware architecture are you compiling for? If it's ARM64, and
>>you're not using a trunk version of libffi, that segfault in
>>test_ctypes is to be expected.
>>
>>> Does this mean I can safely ignore the segfault?
>>
>>Well, "safely" in the sense that everything except a very recent
>>version of libffi is known to not work on ARM64 - so if it doesn't
>>work, it's not because of something Python is doing wrong, it's a
>>problem with a dependency.
>>
>
>...do you know of a way to run all tests except _ctypes, so that Imcan
>verify everything else?
>

You can temporarily change the last line of Lib/ctypes/test/__init__.py to:

return args[1]

>And to answer Russell's questions:
>
>>>>Are you using the libffi sources vendored into the Python source
>>tree, or a more recent version? 
>>
>>>By 'vendored in' do you mean 'sources included in python source tree
>>for building?'
>>
>>Correct. The libffi source code that is in the Python source tree is
>>quite old, and *definitely* doesn't work on ARM64. I'm not even sure
>>that it works on ARMv7.
>>
>Definitely using the vendored in sources.
>
>>
>>> Would your recommend downloading and building libffi from sources
>(on
>>device) and then building python?
>>
>>Well, for starters - as I've said before, I'd recommend not compiling
>>on device at all, but that's a separate issue. 
>>
>Any particular reason why? The device I'm working with has a port of
>GCC which I've used to build other utilities that work well...given
>that I'm working on a tablet.
>
>>However, regardless of where you're compiling, you can either use an
>>external libffi, or you can do what I've done in the iOS patch -
>update
>>the Python source tree to include a newer version of libffi. If you
>>update the source in the Python tree, you need to use 2 versions (or a
>>merged version); you need v3.2 sources for ARMv7, and trunk sources
>for
>>ARM64. This is because libffi v3.2 doesn't work for ARM64, and trunk
>>doesn't even compile for ARMv7. See the iOS patch for details.
>>
>Since I'm not compiling for ARM64...and have zero experience with
>hacking configure.ac files...would it be okay to include just the 3.2
>sources if I note somewhere that this fork does not include ARM64
>support?
>
>>> I'm asking the above questions because I've hit a fairly significant
>>roadblock; I'm still getting the segfault when test_ctypes is run and
>I
>>can't seem to get anything useful out of gdb.
>>
>>Personally, I've pretty much given up on CPython on Android. Even when
>>I got it working, the performance of the JNI layer is *abysmal*, and
>>severely crippled. If you're planning to actually interact with the
>>device in any way (like, say, put a button on the screen), that's a
>big
>>problem. 
>>
>My goal is to build a port that operates on Android in a Linux-like
>environment; I'm currently using KBOX3.  I've no experience with the
>Python language so JNI interop is Greek to me.  For what i'm using the
>port for it functions fairly well.
>
>>I'm working on an approach that uses Java natively - think "Jython for
>>Android". I'm still a way off having anything working, though.
>>
>I'll keep an eye out for it once I've got Python basics under my belt.
>>----------
>>
>>_______________________________________
>>Python tracker <report@bugs.python.org>
>><http://bugs.python.org/issue23496>
>>_______________________________________
>
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue23496>
>_______________________________________
msg248974 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-08-21 19:04
Thanks Ryan.

Running ./python -m test (with the edit to the __init__.py for ctypes) produces the following error:

[151/390/18] test_hash
Fatal Python error: Bus error

Current thread 0xb6f72ec8 (most recent call first):
  File "/bld/pyt/cpython-android/Lib/test/test_hash.py", line 89 in test_unaligned_buffers
  File "/bld/pyt/cpython-android/Lib/unittest/case.py", line 577 in run
  File "/bld/pyt/cpython-android/Lib/unittest/case.py", line 625 in __call__
  File "/bld/pyt/cpython-android/Lib/unittest/suite.py", line 122 in run
  File "/bld/pyt/cpython-android/Lib/unittest/suite.py", line 84 in __call__
  File "/bld/pyt/cpython-android/Lib/unittest/suite.py", line 122 in run
  File "/bld/pyt/cpython-android/Lib/unittest/suite.py", line 84 in __call__
  File "/bld/pyt/cpython-android/Lib/unittest/suite.py", line 122 in run
  File "/bld/pyt/cpython-android/Lib/unittest/suite.py", line 84 in __call__
  File "/bld/pyt/cpython-android/Lib/test/support/__init__.py", line 1668 in run
  File "/bld/pyt/cpython-android/Lib/test/support/__init__.py", line 1769 in _run_suite
  File "/bld/pyt/cpython-android/Lib/test/support/__init__.py", line 1803 in run_unittest
  File "/bld/pyt/cpython-android/Lib/test/regrtest.py", line 1279 in test_runner
  File "/bld/pyt/cpython-android/Lib/test/regrtest.py", line 1280 in runtest_inner
  File "/bld/pyt/cpython-android/Lib/test/regrtest.py", line 978 in runtest
  File "/bld/pyt/cpython-android/Lib/test/regrtest.py", line 763 in main
  File "/bld/pyt/cpython-android/Lib/test/regrtest.py", line 1564 in main_in_temp_cwd  File "/bld/pyt/cpython-android/Lib/test/__main__.py", line 3 in <module>
  File "/bld/pyt/cpython-android/Lib/runpy.py", line 85 in _run_code
  File "/bld/pyt/cpython-android/Lib/runpy.py", line 170 in _run_module_as_main
Bus error

Not sure what a bus error is...off to Google
msg248975 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-08-21 19:15
Bus error is basically unaligned memory access.

...

...do you feel like trying to get a backtrace from GDB again? :) (The last
time likely didn't end well because the machine stack got somehow
corrupted.)

On Fri, Aug 21, 2015 at 2:04 PM, Cyd Haselton <report@bugs.python.org>
wrote:

>
> Cyd Haselton added the comment:
>
> Thanks Ryan.
>
> Running ./python -m test (with the edit to the __init__.py for ctypes)
> produces the following error:
>
> [151/390/18] test_hash
> Fatal Python error: Bus error
>
> Current thread 0xb6f72ec8 (most recent call first):
>   File "/bld/pyt/cpython-android/Lib/test/test_hash.py", line 89 in
> test_unaligned_buffers
>   File "/bld/pyt/cpython-android/Lib/unittest/case.py", line 577 in run
>   File "/bld/pyt/cpython-android/Lib/unittest/case.py", line 625 in
> __call__
>   File "/bld/pyt/cpython-android/Lib/unittest/suite.py", line 122 in run
>   File "/bld/pyt/cpython-android/Lib/unittest/suite.py", line 84 in
> __call__
>   File "/bld/pyt/cpython-android/Lib/unittest/suite.py", line 122 in run
>   File "/bld/pyt/cpython-android/Lib/unittest/suite.py", line 84 in
> __call__
>   File "/bld/pyt/cpython-android/Lib/unittest/suite.py", line 122 in run
>   File "/bld/pyt/cpython-android/Lib/unittest/suite.py", line 84 in
> __call__
>   File "/bld/pyt/cpython-android/Lib/test/support/__init__.py", line 1668
> in run
>   File "/bld/pyt/cpython-android/Lib/test/support/__init__.py", line 1769
> in _run_suite
>   File "/bld/pyt/cpython-android/Lib/test/support/__init__.py", line 1803
> in run_unittest
>   File "/bld/pyt/cpython-android/Lib/test/regrtest.py", line 1279 in
> test_runner
>   File "/bld/pyt/cpython-android/Lib/test/regrtest.py", line 1280 in
> runtest_inner
>   File "/bld/pyt/cpython-android/Lib/test/regrtest.py", line 978 in runtest
>   File "/bld/pyt/cpython-android/Lib/test/regrtest.py", line 763 in main
>   File "/bld/pyt/cpython-android/Lib/test/regrtest.py", line 1564 in
> main_in_temp_cwd  File "/bld/pyt/cpython-android/Lib/test/__main__.py",
> line 3 in <module>
>   File "/bld/pyt/cpython-android/Lib/runpy.py", line 85 in _run_code
>   File "/bld/pyt/cpython-android/Lib/runpy.py", line 170 in
> _run_module_as_main
> Bus error
>
> Not sure what a bus error is...off to Google
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue23496>
> _______________________________________
>
msg248990 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-08-22 18:06
Update: 
I found this bug report (https://bugs.python.org/issue23786) and re-compiled python with -mno-unaligned-access as mentioned. test_hash still throws a bus error.

Back to drawing board....suggestions welcome.
msg248995 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-08-22 19:47
I'm assuming gdb still doesn't work?

On August 22, 2015 1:06:41 PM CDT, Cyd Haselton <report@bugs.python.org> wrote:
>
>Cyd Haselton added the comment:
>
>Update: 
>I found this bug report (https://bugs.python.org/issue23786) and
>re-compiled python with -mno-unaligned-access as mentioned. test_hash
>still throws a bus error.
>
>Back to drawing board....suggestions welcome.
>
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue23496>
>_______________________________________
msg248996 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-08-22 20:06
On August 22, 2015 2:47:42 PM CDT, Ryan Gonzalez <report@bugs.python.org> wrote:
>
>Ryan Gonzalez added the comment:
>
>I'm assuming gdb still doesn't work?

It does. I just don't know it well enough to know if the errors are something I'm doing wrong or something the program is doing wrong.

Basically I'm running
gdb
file ./python
set args -m test
set sysroot /path/to/sysroot
run

and examining/posting the results.  Which are confusing to me because, again, don't know it well enough.
>
>On August 22, 2015 1:06:41 PM CDT, Cyd Haselton
><report@bugs.python.org> wrote:
>>
>>Cyd Haselton added the comment:
>>
>>Update: 
>>I found this bug report (https://bugs.python.org/issue23786) and
>>re-compiled python with -mno-unaligned-access as mentioned. test_hash
>>still throws a bus error.
>>
>>Back to drawing board....suggestions welcome.
>>
>>----------
>>
>>_______________________________________
>>Python tracker <report@bugs.python.org>
>><http://bugs.python.org/issue23496>
>>_______________________________________
>
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue23496>
>_______________________________________
msg248997 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-08-22 20:28
After typing 'run', enter 'bt' and post the results.

On Sat, Aug 22, 2015 at 3:06 PM, Cyd Haselton <report@bugs.python.org>
wrote:

>
> Cyd Haselton added the comment:
>
> On August 22, 2015 2:47:42 PM CDT, Ryan Gonzalez <report@bugs.python.org>
> wrote:
> >
> >Ryan Gonzalez added the comment:
> >
> >I'm assuming gdb still doesn't work?
>
> It does. I just don't know it well enough to know if the errors are
> something I'm doing wrong or something the program is doing wrong.
>
> Basically I'm running
> gdb
> file ./python
> set args -m test
> set sysroot /path/to/sysroot
> run
>
> and examining/posting the results.  Which are confusing to me because,
> again, don't know it well enough.
> >
> >On August 22, 2015 1:06:41 PM CDT, Cyd Haselton
> ><report@bugs.python.org> wrote:
> >>
> >>Cyd Haselton added the comment:
> >>
> >>Update:
> >>I found this bug report (https://bugs.python.org/issue23786) and
> >>re-compiled python with -mno-unaligned-access as mentioned. test_hash
> >>still throws a bus error.
> >>
> >>Back to drawing board....suggestions welcome.
> >>
> >>----------
> >>
> >>_______________________________________
> >>Python tracker <report@bugs.python.org>
> >><http://bugs.python.org/issue23496>
> >>_______________________________________
> >
> >----------
> >
> >_______________________________________
> >Python tracker <report@bugs.python.org>
> ><http://bugs.python.org/issue23496>
> >_______________________________________
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue23496>
> _______________________________________
>
msg249199 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-08-26 14:34
UPDATE:
Still working on the test_hash issue; not making much progress

QUESTION
Does gdb have to be configured with the --with-python switch in order to debug the python binary correctly?
msg249208 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-08-26 16:01
I don't think so. I believe that switch is just to enable writing gdb extensions in Python.

On August 26, 2015 9:34:29 AM CDT, Cyd Haselton <report@bugs.python.org> wrote:
>
>Cyd Haselton added the comment:
>
>UPDATE:
>Still working on the test_hash issue; not making much progress
>
>QUESTION
>Does gdb have to be configured with the --with-python switch in order
>to debug the python binary correctly?
>
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue23496>
>_______________________________________
msg250794 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-09-15 18:53
Ryan,

Here's the results after the edit to the ctypes test. I also re-compiled gdb with python support, which is why this took so long

(gdb) file ./python

Load new symbol table from "./python"? (y or n) y
Reading symbols from ./python...done.
Traceback (most recent call last):
  File "/bld/pyt/cpython-android/python-gdb.py", line 59, in <module>
    _type_char_ptr = gdb.lookup_type('char').pointer() # char*
AttributeError: 'module' object has no attribute 'lookup_type'

(gdb) set args -m tests
(gdb) set sysroot /usr/gcc-4.9-pie/sysroot
(gdb) run

Starting program: /bld/pyt/cpython-android/python -m tests
setpgrp failed in child: No such process
warning: Unable to find dynamic linker breakpoint function.
GDB will be unable to debug shared library initializers
and track explicitly loaded dynamic code.

Program received signal SIGILL, Illegal instruction.
0xb6a63cc8 in ?? ()
(gdb) bt
Python Exception <class 'ImportError'> No module named 'gdb.frames':
#0  0xb6a63cc8 in ?? ()
#1  0xb6a5feb0 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
msg251598 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-09-25 19:43
Still battling this bus error issue.  Recompiling gdb --with-python did not help; I still cannot get any useful info out of it. I suspect it may be due to this error:

setpgrp failed in child: No such process

but I'm not sure.

At this point I'm going to try downloading the Google-patched version...hopefully it is patched such that the setpgrp bug is resolved.
msg251648 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-09-26 14:10
UPDATE: Before downloading/building Google gdb source I ran test_gdb.py, which failed completely.  Details of those tests are attached.
msg253800 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-10-31 17:02
UPDATE: Finslly rebuilding Python from github clone after having to factory reset my tablet for reasons (root access).  Hopefully build and tests will benefit from paving and starting over.
msg253817 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-11-01 00:12
Unfortunately, in spite of the other minor issues that resolved,I am still getting a bus error when the hash test runs.

Before I dive back into trying to get gdb working to debug this, is there any way to skip this test to see of the others run?
msg253847 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-11-01 14:34
GDB debugging isn't going to happen any time soon; all of the gdb tests fail.
I re-compiled with the --with-pydebug option, thinking that would help, but it won't build; I get the following error when the newly built python starts to build packages:

Debug memory block at address p=0xb6003070: API 'a'
    795107700 bytes originally requested
    The 3 pad bytes at p-3 are not all FORBIDDENBYTE (0xfb):
        at p-3: 0x2f *** OUCH
        at p-2: 0x00 *** OUCH
        at p-1: 0x00 *** OUCH
    Because memory is corrupted at the start, the count of bytes requested
       may be bogus, and checking the trailing pad bytes may segfault.
    The 4 pad bytes at tail=0xe56491e4 are Segmentation fault
generate-posix-vars failed

Additional things I've attemptedto try to resolve the test_hash failure, based on this  bug report (https://mail.python.org/pipermail/python-bugs-list/2015-March/267442.html)
1) compiling with -mno-unaligned-access
2) compiling with FNV as the default hash algorithm.

Neither worked. At this point I am at a loss and am beginning to think there may be a problem with my device.  Any suggestions are welcome.
msg253851 - (view) Author: Ryan Gonzalez (Ryan.Gonzalez) * Date: 2015-11-01 15:14
Are there no binaries?

Pretty sure it isn't your device. I'd think a compiler issue. What are you using, and what version?

You have insane determination! :D

On November 1, 2015 8:34:21 AM CST, Cyd Haselton <report@bugs.python.org> wrote:
>
>Cyd Haselton added the comment:
>
>GDB debugging isn't going to happen any time soon; all of the gdb tests
>fail.
>I re-compiled with the --with-pydebug option, thinking that would help,
>but it won't build; I get the following error when the newly built
>python starts to build packages:
>
>Debug memory block at address p=0xb6003070: API 'a'
>    795107700 bytes originally requested
>    The 3 pad bytes at p-3 are not all FORBIDDENBYTE (0xfb):
>        at p-3: 0x2f *** OUCH
>        at p-2: 0x00 *** OUCH
>        at p-1: 0x00 *** OUCH
> Because memory is corrupted at the start, the count of bytes requested
>       may be bogus, and checking the trailing pad bytes may segfault.
>    The 4 pad bytes at tail=0xe56491e4 are Segmentation fault
>generate-posix-vars failed
>
>Additional things I've attemptedto try to resolve the test_hash
>failure, based on this  bug report
>(https://mail.python.org/pipermail/python-bugs-list/2015-March/267442.html)
>1) compiling with -mno-unaligned-access
>2) compiling with FNV as the default hash algorithm.
>
>Neither worked. At this point I am at a loss and am beginning to think
>there may be a problem with my device.  Any suggestions are welcome.
>
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue23496>
>_______________________________________
msg253856 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-11-01 15:27
Pretty sure there are binaries, but the python binary can't build its own modules.

I'm using GCC 4.9.2 on Android L...which means it is a PIE and that Python is being built as a PIE
msg253862 - (view) Author: Cyd Haselton (chaselton) * Date: 2015-11-01 18:06
Key note: it looks like the config option --with-hash-algorithm does nothing; python is still built with the siphash hash algorithim even though I specified fnv and, in pyconfig.h:

/* Define hash algorithm for str, bytes and memoryview. SipHash24: 1, FNV: 2,
   externally defined: 0 */
#define Py_HASH_ALGORITHM 2

What's the correct way to build with fnv as the hash algorithm?
msg254248 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2015-11-07 01:17
Ryan, please sign the PSF Contributor Agreement
https://www.python.org/psf/contrib/
https://www.python.org/psf/contrib/contrib-form/

Also, when replying by email, please delete the previous message, except possibly for quoting a line or two.  When listed on the web page, big quotations are useless noise.
msg264176 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2016-04-25 14:35
FYI: https://mail.python.org/pipermail/python-dev/2016-April/144320.html
History
Date User Action Args
2016-06-07 17:47:40cvrebertsetnosy: + cvrebert
2016-04-25 14:35:26eric.snowsetnosy: + eric.snow
messages: + msg264176
2016-04-20 14:24:22Roman.Evstifeevsetnosy: + Roman.Evstifeev
2015-11-25 15:10:02Chi Hsuan Yensetnosy: + Chi Hsuan Yen
2015-11-07 01:17:21terry.reedysetnosy: + terry.reedy
messages: + msg254248
2015-11-01 18:06:17chaseltonsetmessages: + msg253862
2015-11-01 15:27:30chaseltonsetmessages: + msg253856
2015-11-01 15:14:17Ryan.Gonzalezsetmessages: + msg253851
2015-11-01 14:34:21chaseltonsetmessages: + msg253847
2015-11-01 00:12:00chaseltonsetmessages: + msg253817
2015-10-31 17:02:49chaseltonsetmessages: + msg253800
2015-09-26 14:10:50chaseltonsetfiles: + test_gdb.log

messages: + msg251648
2015-09-25 19:43:09chaseltonsetmessages: + msg251598
2015-09-17 06:51:13ethan.furmansetnosy: - ethan.furman
2015-09-15 18:53:07chaseltonsetmessages: + msg250794
2015-08-26 16:01:11Ryan.Gonzalezsetmessages: + msg249208
2015-08-26 14:34:29chaseltonsetmessages: + msg249199
2015-08-22 20:28:17Ryan.Gonzalezsetmessages: + msg248997
2015-08-22 20:06:53chaseltonsetmessages: + msg248996
2015-08-22 19:47:42Ryan.Gonzalezsetmessages: + msg248995
2015-08-22 18:06:40chaseltonsetmessages: + msg248990
2015-08-21 19:15:54Ryan.Gonzalezsetmessages: + msg248975
2015-08-21 19:04:40chaseltonsetmessages: + msg248974
2015-08-21 14:48:32Ryan.Gonzalezsetmessages: + msg248957
2015-08-21 14:25:10chaseltonsetmessages: + msg248954
2015-08-21 01:03:13freakboy3742setmessages: + msg248938
2015-08-20 18:09:24chaseltonsetmessages: + msg248909
2015-08-17 15:49:42chaseltonsetmessages: + msg248733
2015-08-17 14:39:55Ryan.Gonzalezsetmessages: + msg248731
2015-08-17 14:24:50chaseltonsetmessages: + msg248730
2015-08-16 01:05:45Ryan.Gonzalezsetmessages: + msg248672
2015-08-16 00:32:36chaseltonsetmessages: + msg248671
2015-08-12 14:27:23Ryan.Gonzalezsetmessages: + msg248458
2015-08-12 11:21:23chaseltonsetmessages: + msg248452
2015-07-30 23:58:16chaseltonsetmessages: + msg247713
2015-07-29 17:06:17chaseltonsetmessages: + msg247584
2015-07-25 18:32:18chaseltonsetmessages: + msg247364
2015-07-25 17:38:24Ryan.Gonzalezsetmessages: + msg247359
2015-07-25 17:10:41mpaolinisetnosy: + mpaolini
2015-07-25 11:56:10chaseltonsetmessages: + msg247346
2015-07-25 03:18:26freakboy3742setmessages: + msg247310
2015-07-24 10:41:57chaseltonsetmessages: + msg247269
2015-07-24 00:10:35freakboy3742setmessages: + msg247235
2015-07-23 22:14:56chaseltonsetfiles: + py_test_results.log

messages: + msg247232
2015-07-22 13:51:51chaseltonsetmessages: + msg247118
2015-07-20 03:50:34tritiumsetnosy: + tritium
2015-07-19 10:40:58chaseltonsetmessages: + msg246938
2015-06-30 15:09:08chaseltonsetmessages: + msg245993
2015-06-30 14:20:45Ryan.Gonzalezsetmessages: + msg245990
2015-06-30 13:14:34chaseltonsetmessages: + msg245988
2015-06-29 14:29:35chaseltonsetmessages: + msg245938
2015-06-29 14:00:01Ryan.Gonzalezsetmessages: + msg245937
2015-06-29 12:09:24chaseltonsetmessages: + msg245932
2015-06-27 22:51:29chaseltonsetmessages: + msg245894
2015-06-23 09:50:34chaseltonsetmessages: + msg245674
2015-06-17 20:15:59chaseltonsetmessages: + msg245445
2015-06-05 14:57:38Ryan.Gonzalezsetmessages: + msg244868
2015-06-05 11:22:07chaseltonsetmessages: + msg244859
2015-06-04 21:56:18Ryan.Gonzalezsetfiles: + lib_fixes.patch

messages: + msg244846
2015-06-04 21:52:58Ryan.Gonzalezsetfiles: + kbox_fix.patch

messages: + msg244845
2015-05-29 16:38:54chaseltonsetmessages: + msg244389
2015-05-29 15:52:23Ryan.Gonzalezsetmessages: + msg244387
2015-05-29 10:54:18chaseltonsetmessages: + msg244366
2015-04-26 17:12:58chaseltonsetmessages: + msg242062
2015-04-23 10:57:51chaseltonsetmessages: + msg241862
2015-04-22 14:02:20chaseltonsetmessages: + msg241805
2015-04-20 15:38:17chaseltonsetmessages: + msg241663
2015-04-20 14:54:06Ryan.Gonzalezsetmessages: + msg241657
2015-04-20 13:27:53chaseltonsetmessages: + msg241642
2015-04-20 11:33:42chaseltonsetmessages: + msg241634
2015-04-20 00:40:05Ryan.Gonzalezsetmessages: + msg241579
2015-04-20 00:39:19chaseltonsetmessages: + msg241578
2015-04-20 00:30:44chaseltonsetfiles: + python.patch

messages: + msg241577
2015-04-20 00:26:20Ryan.Gonzalezsetfiles: + kbox_fix.patch
2015-04-20 00:25:24Ryan.Gonzalezsetmessages: + msg241576
2015-04-20 00:16:11chaseltonsetmessages: + msg241574
2015-04-18 20:38:43chaseltonsetmessages: + msg241457
2015-04-18 18:08:17Ryan.Gonzalezsetmessages: + msg241437
2015-04-18 17:39:15chaseltonsetmessages: + msg241432
2015-04-18 16:59:39Ryan.Gonzalezsetmessages: + msg241427
2015-04-18 00:10:53chaseltonsetmessages: + msg241385
2015-04-15 20:23:28Ryan.Gonzalezsetmessages: + msg241154
2015-04-14 01:16:33chaseltonsetmessages: + msg240822
2015-04-11 18:01:42chaseltonsetmessages: + msg240489
2015-04-10 14:52:43chaseltonsetmessages: + msg240423
2015-04-10 01:10:22Ryan.Gonzalezsetmessages: + msg240406
2015-04-10 00:53:21chaseltonsetmessages: + msg240404
2015-04-10 00:37:09chaseltonsetmessages: + msg240403
2015-04-09 23:12:54chaseltonsetmessages: + msg240401
2015-04-09 19:13:08Ryan.Gonzalezsetmessages: + msg240366
2015-04-09 18:51:15chaseltonsetmessages: + msg240363
2015-04-09 17:34:35Ryan.Gonzalezsetmessages: + msg240357
2015-04-09 17:28:31chaseltonsetmessages: + msg240356
2015-04-09 17:07:28Ryan.Gonzalezsetfiles: + rjmatthews64_fixes2.patch

messages: + msg240352
2015-04-09 17:02:56chaseltonsetmessages: + msg240351
2015-04-09 16:56:06chaseltonsetmessages: + msg240350
2015-04-09 15:38:29Ryan.Gonzalezsetfiles: + rjmatthews64_fixes2.patch

messages: + msg240342
2015-04-09 15:33:02chaseltonsetmessages: + msg240339
2015-04-09 15:24:24Ryan.Gonzalezsetfiles: + rjmatthews64_fixes2.patch

messages: + msg240338
2015-04-09 14:23:52chaseltonsetmessages: + msg240329
2015-04-07 16:42:58chaseltonsetmessages: + msg240218
2015-04-07 13:20:39chaseltonsetmessages: + msg240208
2015-04-06 23:39:51hayposetnosy: - haypo
2015-04-06 20:22:44Ryan.Gonzalezsetfiles: - issue_20306.patch
2015-04-06 20:21:58Ryan.Gonzalezsetfiles: + issue_20306.patch

messages: + msg240178
2015-04-06 20:07:30chaseltonsetmessages: + msg240176
2015-04-06 16:19:23Ryan.Gonzalezsetfiles: + androidfn.patch

messages: + msg240157
2015-04-06 02:10:36chaseltonsetmessages: + msg240144
2015-04-06 01:30:17chaseltonsetmessages: + msg240143
2015-04-05 17:42:34chaseltonsetmessages: + msg240127
2015-04-05 13:25:49Ryan.Gonzalezsetmessages: + msg240119
2015-04-05 13:14:33chaseltonsetmessages: + msg240118
2015-04-05 13:08:08chaseltonsetmessages: + msg240117
2015-03-28 20:42:17chaseltonsetmessages: + msg239466
2015-03-25 15:41:20Ryan.Gonzalezsetmessages: + msg239264
2015-03-25 13:53:12chaseltonsetmessages: + msg239256
2015-03-22 20:10:58Ryan.Gonzalezsetmessages: + msg238934
2015-03-22 20:04:07chaseltonsetmessages: + msg238932
2015-03-21 00:44:34Ryan.Gonzalezsetmessages: + msg238744
2015-03-20 00:44:25Ryan.Gonzalezsetfiles: + rjmatthews62_fixes_tweaked.patch
2015-03-20 00:44:17Ryan.Gonzalezsetfiles: + unused_var.patch
2015-03-20 00:44:10Ryan.Gonzalezsetfiles: + issue_21668.patch
2015-03-20 00:43:45Ryan.Gonzalezsetfiles: + issue_20306.patch
2015-03-20 00:43:34Ryan.Gonzalezsetfiles: + issue_20305_tweaked.patch
2015-03-20 00:43:26Ryan.Gonzalezsetfiles: + issue_16353.patch
2015-03-20 00:43:15Ryan.Gonzalezsetfiles: + kbox_fix.patch
2015-03-20 00:43:04Ryan.Gonzalezsetfiles: + android_segfault_fix.patch

messages: + msg238592
2015-03-10 17:56:20skrahsetnosy: - skrah
2015-03-10 17:45:29chaseltonsetmessages: + msg237785
2015-03-10 15:29:26Ryan.Gonzalezsetmessages: + msg237774
2015-03-10 00:02:34Ryan.Gonzalezsetmessages: + msg237725
2015-03-09 23:50:56chaseltonsetmessages: + msg237724
2015-03-09 19:01:26skrahsetmessages: + msg237693
2015-03-09 17:56:05ethan.furmansetmessages: + msg237684
2015-03-09 17:52:19skrahsetmessages: + msg237683
2015-03-09 17:42:30Ryan.Gonzalezsetmessages: + msg237682
2015-03-09 17:38:05skrahsetmessages: + msg237681
2015-03-09 17:28:36Ryan.Gonzalezsetmessages: + msg237680
2015-03-09 14:45:42skrahsetmessages: + msg237654
2015-03-09 12:05:38chaseltonsetmessages: + msg237639
2015-03-08 22:03:10Ryan.Gonzalezsetmessages: + msg237574
2015-03-06 19:52:35chaseltonsetmessages: + msg237374
2015-03-05 20:33:40Ryan.Gonzalezsetmessages: + msg237285
2015-03-05 20:09:51chaseltonsetmessages: + msg237284
2015-03-05 17:57:37Ryan.Gonzalezsetmessages: + msg237280
2015-03-05 11:11:43chaseltonsetmessages: + msg237260
2015-03-05 00:05:11Ryan.Gonzalezsetmessages: + msg237222
2015-03-04 23:23:06chaseltonsetmessages: + msg237219
2015-03-04 22:42:10Ryan.Gonzalezsetmessages: + msg237216
2015-03-04 22:37:28chaseltonsetmessages: + msg237215
2015-03-03 10:45:45chaseltonsetmessages: + msg237118
2015-03-02 18:44:12Ryan.Gonzalezsetmessages: + msg237073
2015-03-02 03:23:18chaseltonsetmessages: + msg237009
2015-03-02 00:47:47Ryan.Gonzalezsetmessages: + msg236999
2015-03-01 00:17:58chaseltonsetmessages: + msg236926
2015-02-28 21:26:27Ryan.Gonzalezsetmessages: + msg236911
2015-02-28 15:07:25chaseltonsetmessages: + msg236890
2015-02-28 14:10:24skrahsetmessages: + msg236886
2015-02-28 14:04:27chaseltonsetmessages: + msg236884
2015-02-28 14:01:00skrahsetmessages: + msg236883
2015-02-27 23:36:28ethan.furmansetmessages: + msg236858
2015-02-27 23:26:51Ryan.Gonzalezsetmessages: + msg236857
2015-02-27 19:13:53ethan.furmansetmessages: + msg236820
2015-02-27 19:10:28chaseltonsetmessages: + msg236819
2015-02-27 18:25:12ethan.furmansetmessages: + msg236810
2015-02-27 17:28:35chaseltonsetmessages: + msg236801
2015-02-27 17:27:05ethan.furmansetmessages: + msg236800
2015-02-27 17:17:20Ryan.Gonzalezsetmessages: + msg236797
2015-02-27 16:25:57ethan.furmansetmessages: + msg236779
2015-02-27 16:05:04skrahsetmessages: + msg236773
2015-02-27 15:31:02skrahsetmessages: + msg236766
2015-02-27 15:23:59r.david.murraysetmessages: + msg236763
2015-02-27 15:11:01chaseltonsetmessages: + msg236761
2015-02-27 14:55:17skrahsetnosy: + skrah
messages: + msg236756
2015-02-27 13:35:16chaseltonsetmessages: + msg236746
2015-02-27 01:30:55ethan.furmansetmessages: + msg236722
2015-02-26 16:38:49hayposetmessages: + msg236685
2015-02-26 16:29:18chaseltonsetmessages: + msg236683
2015-02-26 12:45:07chaseltonsetmessages: + msg236667
2015-02-25 18:06:59Ryan.Gonzalezsetmessages: + msg236611
2015-02-25 15:31:37chaseltonsetmessages: + msg236584
2015-02-25 14:35:11r.david.murraysetnosy: + r.david.murray
messages: + msg236578
2015-02-25 11:38:37chaseltonsetmessages: + msg236571
2015-02-25 10:54:26hayposetmessages: + msg236567
2015-02-25 10:04:23chaseltonsetmessages: + msg236566
2015-02-24 19:08:09Ryan.Gonzalezsettype: enhancement
messages: + msg236530
2015-02-24 12:32:49chaseltonsetmessages: + msg236488
2015-02-23 23:07:37chaseltonsetmessages: + msg236469
2015-02-23 19:47:17Ryan.Gonzalezsetmessages: + msg236462
2015-02-23 15:28:41chaseltonsetfiles: - python-3.4.2-androidpatches-1.tar.gz
2015-02-23 15:26:16chaseltonsetmessages: + msg236447
2015-02-23 14:38:36hayposetnosy: + haypo
dependencies: + add function to os module for getting path to default shell, Android's incomplete locale.h implementation prevents cross-compilation, The select and time modules uses libm functions without linking against it
messages: + msg236445
2015-02-23 14:13:32chaseltonsetfiles: + python-3.4.2-androidpatches-1.tar.gz

messages: + msg236444
2015-02-23 03:57:45Ryan.Gonzalezsetnosy: + Ryan.Gonzalez
2015-02-22 00:19:50freakboy3742setnosy: + freakboy3742
2015-02-21 22:53:10ethan.furmansetnosy: + ethan.furman
2015-02-21 20:46:56chaseltoncreate