msg314295 - (view) |
Author: Steven Noonan (Steven Noonan) |
Date: 2018-03-23 01:03 |
The Windows 10 ARM64 release is out along with a bunch of ARM64 devices. This version of Windows has full support for building native Win32 applications (this isn't just some rehash of Windows RT). It also can run x86 (but not x86_64) apps under a transparent emulation layer.
I would like to see a native build of Python on Windows 10 ARM64.
I did some very basic work to get it compiling (add 10.0.16299.0 as DefaultWindowsSDKVersion, add WindowsSDKDesktopARM64Support property). But there's still a lot missing: ssl, tk, and ctypes don't build.
ssl/ctypes have some assembly that needs writing/porting.
tk has some kind of build failure with the newer Windows SDK: https://core.tcl.tk/tk/tktview?name=3d34589aa0
|
msg314296 - (view) |
Author: Steven Noonan (Steven Noonan) |
Date: 2018-03-23 01:07 |
Oh, another change I had to make was remove all the BaseAddress elements in the Link sections. The linker complains if these are used (the lower 4GB of memory are apparently reserved for the x86 emulation). Also, from what I was told by someone over at Microsoft, the BaseAddress options don't do anything useful on modern Windows versions unless you build with -fixed as well, because everything gets relocated anyway.
|
msg314308 - (view) |
Author: Steve Dower (steve.dower) *  |
Date: 2018-03-23 13:10 |
I'd like to see this as well, but just to set expectations, it can't be any earlier than 3.8 at this stage (for upstream support), and we need access to some real hardware to regularly run tests on.
That said, most of the changes you describe are on my list of things to do anyway. If you've done them and would like to submit a pull request (against master) then we'll happily take them now and include them in 3.7. But you'll still need to build the ARM versions yourself.
|
msg314309 - (view) |
Author: Steven Noonan (Steven Noonan) |
Date: 2018-03-23 13:26 |
I originally tagged this issue against 3.6 just because that's what I was attempting to build. But I'm not super concerned about what release these changes actually land in, I can always backport it to my own builds (and at my own risk).
Even though I'm super interested in seeing these changes done, I'm not sure I'm the right person to do the heavy lifting on this. I don't know if the seemingly undocumented (?) Windows ARM64 ABI matches the well-documented Linux AArch64 ABI, for example. If it does match, that would make porting the asm bits pretty straightforward (mostly just translating GNU assembler AT&T syntax to MASM, I suppose). I could do that. But if it doesn't match then I'd probably need to dig into ARM architecture manuals. Someone else could likely do it in their sleep.
|
msg314311 - (view) |
Author: Zachary Ware (zach.ware) *  |
Date: 2018-03-23 16:11 |
Ideally, all of the changes necessary in OpenSSL, libffi, and Tcl/Tk will happen upstream and we'll just update to new versions of them. We have PR 3806 in progress to extract libffi from our repo and treat it just as any other external dependency on Windows, but neither Steve nor I have had a chance to work on it in quite a long time. That will probably be a prerequisite for properly supporting ARM64 Windows.
|
msg323310 - (view) |
Author: Paul Monson (Paul Monson) * |
Date: 2018-08-09 02:19 |
I'm interested in getting python working on Windows running on Raspberry Pi (Thumb-2 instruction set). I can also contribute to ARM64 Windows. I've made some progress getting ARM32 working on a fork. The next roadblock for my investigation is libffi support for ARM32 on Windows. Can I help with that?
|
msg323368 - (view) |
Author: Steve Dower (steve.dower) *  |
Date: 2018-08-10 13:03 |
If libffi already has support and just needs to be updated in CPython, then sure. I thought we'd finished extracting it, but Zach posted aPR link above that apparently still needs finishing.
However, if libffi doesn't have the support yet then you'll have to approach them.
|
msg323382 - (view) |
Author: Paul Monson (Paul Monson) * |
Date: 2018-08-10 18:37 |
The PR hasn't changed since September. Is it still active?
Should I consider updating the libffi_msvc code to support ARM32 in Python? Completing the switch to libffi makes more sense to me.
I fetched the PR code to my fork and rebased it on the current master branch. When I run python -m test there are 2 tests that fail. Is that what's holding the PR up? or is it something more? I'm new to the Python codebase but I can try to help debugging if the failing tests are the issue.
|
msg350843 - (view) |
Author: Dan (dan) |
Date: 2019-08-30 00:06 |
Hi,
I've just noticed this but a couple of months ago I managed to compile Python 2.7 on win32 ARM64 including the said ssl, tk, ctypes modules.
You can download from here and test -
https://mega.nz/#F!PclhDIRB!-yhBZ6UM7S596ijNU3dx0A
You will need to install the MSVC 2015 runtime first and if you want ssl to work then also download the OpenSSL package from the same link.
For ctypes I used the (then most) latest version of libffi. There's an msvc_build directory in the master branch which allows ARM64 builds. With the library working the ctypes module builds just fine (BTW from looking at the libffi code it seems that the code is a straightforward port from the Linux ARM64 code, which is generally fine since Windows on ARM seems to use the standard ARMv8 ABI but who knows what end cases may occur).
For SSL, I managed to compile OpenSSL with a lot of headache. Also as noted I had to give-up on ASM optimizations so the ssl module may not be as fast as the native i686, however it will probably still be way faster than x86 emulation mode.
For TCL/TK I compiled the TCL/TK libraries from source as well. There's an issue with compiling that on MSVC 2015 because apparently the source uses some names which are now reserved keywords on the newer compiler so some search & replace is needed but once the module is built the Python modules compile just fine.
Overall this is a pretty much complete Python 2.7 package as it has all modules usually shipped built-in compiled and seemingly working (as far as I tested). The whole thing took several days but if you want to port it to Python 3.8 it shouldn't be too difficult and I'll gladly share my (very slightly) modified sources with you if you wish.
|
msg356617 - (view) |
Author: Steve Dower (steve.dower) *  |
Date: 2019-11-14 17:05 |
I just added a PR that will produce ARM64 releases (as embeddable, nuget and Microsoft Store packages).
The only missing feature is tkinter (and everything that depends on it). For users who require that, I'd suggest grabbing the x86 installer from python.org which works fine (apart from some minor arithmetic and process management bugs in the Windows emulation layer).
This is not saying we're going to release it just yet. Really it shouldn't be released until 3.9, as it's a new platform and we need to allow the 3rd party package ecosystem to catch up, but at least having the ability to build packages easily will enable those who are already trying to do it themselves.
Allegedly there's a lot of demand for Python on ARM64, but I haven't actually seen it other than this bug. And if that demand immediately extends to numpy or pywin32 on ARM64 then we're still a very long way from actually satisfying it.
I'll likely tweak the PR a bit more to prevent releasing ARM64 packages by default, but then I don't see any issue having it in the 3.8 and master branches for those who would be helped. Any thoughts/comments?
|
msg356724 - (view) |
Author: Dan (dan) |
Date: 2019-11-15 22:52 |
Steve, the 2.7 build I've posted includes native tk, do you need the necessary patches?
|
msg356884 - (view) |
Author: Steve Dower (steve.dower) *  |
Date: 2019-11-18 16:59 |
If the patches are for Tcl/Tk, then you should submit it to them.
I'm not a Tk maintainer, and I never want to be one, so frankly I'd be happier to say it's not supported on ARM :)
But if it builds fine with our PCbuild/prepare_tcltk.bat script and people want it then I'll keep it.
|
msg356999 - (view) |
Author: Dan (dan) |
Date: 2019-11-19 22:37 |
According to this -
https://wiki.tcl-lang.org/page/Building+with+Visual+Studio+2017
the issue will be fixed in the next version (8.6.10). TLDR explanation: Tcl/Tk has variables with names that are now reserved keywords in VS2017, meaning that it can't be build with recent compilers and since VS2017 is the first compiler that has official ARM64 support it means that it currently can't be built for that target.
Anyway when version 8.6.10 is eventually released then it should (hopefully) build fine with the current scripts.
Maybe we can add it later when that happens?
|
msg357090 - (view) |
Author: Steve Dower (steve.dower) *  |
Date: 2019-11-20 17:30 |
New changeset de148f263fba75cd10d2cb010fe9c495cee4ec83 by Steve Dower in branch 'master':
bpo-33125: Add support for building and releasing Windows ARM64 packages (GH-16828)
https://github.com/python/cpython/commit/de148f263fba75cd10d2cb010fe9c495cee4ec83
|
msg357091 - (view) |
Author: Steve Dower (steve.dower) *  |
Date: 2019-11-20 17:34 |
I've merged the change to master, but note that the release is not enabled, so we aren't actually releasing ARM64 builds yet (though they will be compiled [but not tested] in PRs - nothing should be different from other Windows builds though). There's no harm in giving the build scripts a bit of bake time.
I'm also not concerned about backporting to 3.8 to make it easier to build that version, which I've heard a little bit of demand for. But let's run the scripts in master for a little while first.
Deliberately leaving the stage as "commit review" rather than "backport needed" for now.
|
msg357094 - (view) |
Author: Steve Dower (steve.dower) *  |
Date: 2019-11-20 17:41 |
I also filed issue38867 specifically for enabling Tkinter on Windows ARM.
|
msg357931 - (view) |
Author: Steve Dower (steve.dower) *  |
Date: 2019-12-06 17:41 |
New changeset c9f480d2ccda9de46584cabe086f0acfa45f2faf by Steve Dower in branch '3.8':
bpo-33125: Add support for building and releasing Windows ARM64 packages (GH-17480)
https://github.com/python/cpython/commit/c9f480d2ccda9de46584cabe086f0acfa45f2faf
|
msg357934 - (view) |
Author: Steve Dower (steve.dower) *  |
Date: 2019-12-06 18:42 |
Closing this as the work we need for support is now added. Whether we start releasing binaries from python.org is a discussion to have on python-dev and with the broader community.
Right now, I don't think the ecosystem is ready to add support (mainly because there is no CI system that can build/test easily). And most of our tooling isn't up to the task either. Until that can be improved, it's not fair to expect third party packagers to provide wheels or support the platform.
|
msg357935 - (view) |
Author: Steve Dower (steve.dower) *  |
Date: 2019-12-06 18:45 |
This site seems like a pretty good aggregation of news around Windows on ARM64 devices: https://www.windows-arm.com/
Short story is, it's a real thing, but for (what I consider) obvious reasons the OSS developer experience isn't great yet.
|
|
Date |
User |
Action |
Args |
2019-12-06 18:45:01 | steve.dower | set | messages:
+ msg357935 |
2019-12-06 18:42:46 | steve.dower | set | status: open -> closed resolution: fixed messages:
+ msg357934
stage: patch review -> resolved |
2019-12-06 17:41:01 | steve.dower | set | messages:
+ msg357931 |
2019-12-06 16:33:18 | zach.ware | set | messages:
- msg357922 |
2019-12-06 16:09:44 | wayoman | set | nosy:
+ wayoman messages:
+ msg357922
|
2019-12-05 23:46:03 | steve.dower | set | stage: commit review -> patch review pull_requests:
+ pull_request16959 |
2019-11-20 17:41:52 | steve.dower | set | messages:
+ msg357094 |
2019-11-20 17:34:36 | steve.dower | set | messages:
+ msg357091 stage: patch review -> commit review |
2019-11-20 17:30:52 | steve.dower | set | messages:
+ msg357090 |
2019-11-19 22:37:07 | dan | set | messages:
+ msg356999 |
2019-11-18 16:59:50 | steve.dower | set | messages:
+ msg356884 |
2019-11-15 22:52:34 | dan | set | messages:
+ msg356724 |
2019-11-14 17:05:50 | steve.dower | set | assignee: steve.dower messages:
+ msg356617 versions:
+ Python 3.9 |
2019-11-14 16:58:31 | steve.dower | set | keywords:
+ patch stage: patch review pull_requests:
+ pull_request16665 |
2019-08-30 00:06:50 | dan | set | nosy:
+ dan messages:
+ msg350843
|
2018-08-10 18:37:04 | Paul Monson | set | messages:
+ msg323382 |
2018-08-10 13:03:58 | steve.dower | set | messages:
+ msg323368 |
2018-08-09 02:19:18 | Paul Monson | set | nosy:
+ Paul Monson messages:
+ msg323310
|
2018-03-23 16:21:29 | steven.downum | set | nosy:
+ steven.downum
|
2018-03-23 16:11:21 | zach.ware | set | messages:
+ msg314311 |
2018-03-23 13:26:14 | Steven Noonan | set | messages:
+ msg314309 |
2018-03-23 13:10:04 | steve.dower | set | messages:
+ msg314308 versions:
+ Python 3.8, - Python 3.6 |
2018-03-23 01:07:22 | Steven Noonan | set | messages:
+ msg314296 |
2018-03-23 01:03:31 | Steven Noonan | create | |