New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Better platform.processor support #80148
Comments
or: Unable to implement 'uname' on Python due to recursive call In this issue, I stumbled across a strange and somewhat unintuitive behavior. This project attempts to supply a What I found, however, was that because the stdlib Line 836 in 9db56fb
uname is implemented by Python (and on the path), will probably fail after a long delay due to infinite recursion.
Moreover, the only call that's currently invoking For example, on Windows, you get a detailed description from the hardware: 'Intel64 Family 6 Model 142 Stepping 9, GenuineIntel' On macOS, you get just 'i386'. And on Linux, I see 'x86_64' or sometimes just '' (in docker). To make matters even worse, this I suggest instead the platform module should (a) resolve processor information in a more uniform manner and (b) not ever call uname, maybe with something like this. At the very least, the After some period for comment, I'll draft an implementation. |
Your proposal sounds fine to me. You could fall back on platform.machine() instead of calling |
As the documentation says, the API is intended as fairly portable implementation of the Unix uname helper across platforms. It's fine to redirect this directly to e.g. /proc output instead of using the executable, but in whatever you do here, the output of platform.uname() needs to stay compatible to what the function returned prior to such a change, which usually means: to the output of the uname helper on a system. Could you please check that on most systems, the output remains the same ? Thanks. |
It won't be possible in general to emit what the function returned before, as What I might be able to do is find the implementation of "uname" and see if there's a way to get the value from the same source. I did find what I believe is the canonical source. I'll explore if those calls can be translated to Python. |
The first call I see in that routine is to "sysinfo", but the signature of that function doesn't match what I find in the man pages for that function. So that function must be coming from elsewhere. |
Thanks. It would be good to do some before/after tests on popular |
Aha! It seems the 'sysinfo' call is for Solaris: https://docs.oracle.com/cd/E23823_01/html/816-5167/sysinfo-2.html |
Best I can tell, neither sysinfo nor sysctl are exposed in any way to Python, so it may not be possible to accurately load the processor information from those system calls without writing a wrapper in C. What I might try is to experiment with ctypes to see if I can prove the concept. |
Reading further, the 'sysctl' call seems to only be for BSD (https://www.freebsd.org/cgi/man.cgi?sysctl(3)). I could find the man page for sysctl for BSD but not Linux. There is a _sysctl in Linux (http://man7.org/linux/man-pages/man2/sysctl.2.html), but it's use is discouraged and it doesn't provide the necessary information. Now I suspect that the aforementioned GNU coreutils 'uname' implementation is only for non-Linux systems, as none of the underlying system calls are relevant on Linux. I expect if one compiled that uname on Linux, 'uname -p' would emit 'unknown'. Meaning I still don't know how to get a 'uname -p' result on Linux (without invoking uname -p). |
Hmm. But if I go to the Linux man page for uname (https://linux.die.net/man/1/uname) and follow the links to the source code, I end up at the same repository. So maybe the BSD man page is suitable for Linux. I'll work from that assumption for now. |
After fussing with sysctl for a while, I'm fairly confident that one can't use sysctl on Linux reliably (https://stackoverflow.com/a/55066774/70170). I'll keep digging to see if I can find another implementation of |
This answer is extremely helpful. This lack of consistency means that |
Correction on last comment: s/Debian/Ubuntu/ |
Jason: StackExchange does have lots of good hints, but it's not always I started writing the module back in 1999 and even then, the support https://www.egenix.com/www2002/python/mxCGIPython.html The module was originally created to come up with a good name to Note that the processor is not always needed to determine whether Perhaps adding a more capable API to interface to /proc/cpuinfo |
Do we really wish to retain the output for this unreliable interface, especially when it is not standardized and is returning improper information? Is it valuable for Does maintaining compatibility for My instinct is it's impractical to attempt to maintain all of these forks of "uname -p", especially when the result is a largely unpredictable value, so I'm considering the only other viable option I can conceive now:
I was also considering this: instead of invoking "uname" anywhere on the path, invoke it from an explicit whitelist of paths, such as /bin and /usr/bin, so that it's never self-referential. Unfortunately, that wouldn't work if a Python-based implementation were put on one of those paths, so it would be brittle at best. Marc-Andre, I'd love your feedback in light of these challenges. |
The core concern I want to address is that it's not possible to use any function in the platform module without invoking "uname -p", and thus it's not possible to implement "uname" in Python. No amount of supplementary interfaces will help with that. |
On 08.03.2019 18:00, Jason R. Coombs wrote:
I don't know where you get that idea from. The uname family of APIs It's also easy to bypass that by simply seeding the global cache To be clear: I do not consider your use case to be particularly common |
In this commit, I demonstrate the alternative approach I was considering that avoids calling "uname -p" until it's required, but otherwise retains compatibility by using the same logic for resolving the processor on the various platforms. |
I don't think these options are possible in the general case. It was what I attempted to do in the first place, but could not. Consider the situation where a namespace package is present or where a script uses pkg_resources to bootstrap itself (a very common case), or any other case where Here's what happens:
The point of this utility is to supply "coreutils" using Python. It's derived from an abandoned project called "pycoreutils", one purpose of which is to provide the core utilities on a minimal Linux distribution that doesn't have uname. Another is to supply coreutils on Windows. Having an alternate name isn't really viable when the purpose is to supply that interface. I do think your considerations are reasonable, and I'm close to giving up. I look forward to your feedback on the 'resolved-late' branch. |
On 08.03.2019 18:50, Jason R. Coombs wrote:
I don't quite follow: since you are the author of the tool, you can of """ # Seed uname cache to avoid calling uname
platform._uname_cache = platform.uname_result(
system='Linux',
node='moon',
release='5.99.99',
version='#1 SMP 2020',
machine='x86_64',
processor='x86_64') print ('Hello from uname.py')
This is only true for the platform APIs which need information from
I don't have anything against making calling of uname lazy. Your PR is missing tests, though, to support that it actually |
I thought I'd tried that, but failed ref, which is why I committed this change. The problem is that, if
Yes, that sounds like a good plan. I'll add some tests that assert the values and then update the tests to match the current output, establish a baseline. |
In PR 12824 (#12824), I've developed a test that should assure the current output from uname().processor. I've merged those changes with PR 12239, which if the tests pass, should illustrate the values returned are unchanged. @lemburg, would you be willing to review these PRs to confirm they capture and address your concern? |
The aformentioned test broke tests in buildbots: https://buildbot.python.org/all/#builders/105/builds/779 |
raspbian failure https://buildbot.python.org/all/#/builders/645/builds/31 |
I'm hoping that PR 19544 fixes the issue. |
Reopening the ticket, since the implementation makes backwards incompatible changes to platform.uname(): see https://bugs.python.org/issue40570 for a discussion on a better approach to lazy evaluation of getting the processor information. Before we head on into implementation details, could you please point me to the motivation why only the processor detail of uname() needs lazy evaluation ? Thanks. |
My bad. I probably could have been more proactive about providing a reproducer. The problem, as described above (msg335220) and in the associated cmdix ticket, is that invocation of Here's a Dockerfile replicating the issue:
As you can see, this reproducer creates a very simple 'uname' implementation. All it does is print that it's about to patch the platform module (because maybe that will make things work). Unfortunately, that behavior is never reached because before that code has a chance to run, The
So ultimately, the same behavior is triggered before the user's code is ever executed. But more importantly, why should "uname -p" be invoked in a subprocess on Windows to get "platform.system()"? |
Thanks, Jason. I'll have a closer look at the issue and report back later this week. |
I'm going to close this issue again, as the implementation is now present in at least a couple of releases. May I suggest that if there are ongoing concerns or issues to open up a new issue and reference this one and loop me into the conversation? I'm also happy to re-open this one as well. |
uname -p
on Android #19577Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: