New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cannot build importlib.h on Windows #59636
Comments
On Windows, the _freeze_importlib tool is not built, so it's not possible to refresh the file Python/importlib.h, which makes development on the importlib very difficult on Windows. The Makefile contains the rules below, it's probably necessary to implement a similar behavior in a new build project: Modules/_freeze_importlib: Modules/_freeze_importlib.o $(LIBRARY_OBJS_OMIT_FROZEN) Python/importlib.h: $(srcdir)/Lib/importlib/_bootstrap.py Modules/_freeze_importlib.c |
shouldn't be too hard to add, I'll give it a go soon. |
Here's a patch. A new project, _freeze_importlib, has the necessary prerequisites and runs the executable, generating importlib.h, as a post-build step. I'm not sure if we should make this project a pre-requisite of any other projects, though |
IMO dependencies is not a big issue here. In the worst case, the developer can build a second time... I think your patch won't work on Unix machines: _freeze_importlib.c is linked with all Python files *except* frozen.c which defines PyImport_FrozenModules. #ifdef should be used IMO. Also, I noticed that the marshalled code is different. The first difference a 32bit vs. 64bit issue: 0xFFFFFFFF is marshalled as a TYPE_INT64 on 64bit platforms, but as a TYPE_LONG on 32bit platforms. See bpo-15466. |
Thank you. I see how I must change the linkage on PC Only, although I wonder why tuen unix version isn't simply doing something similar.. The marshaled code was indeed created using 64 bits. I was unsure whether to include that file as part of the patch, perhaps I shouldn't. I'll wait for an update for the 64 bit issue. |
(Sorry for the previous message, it was wrong, and I also pressed the wrong button...) A Unix Makefile cannot stand circular dependencies, so _freezeimportlib.c cannot depend on frozen.o. OTOH it's not simple to have almost-similar list of files on VS projects (here we need all-of-python-but-frozen.c), so your solution is not bad either. |
I don't think it's necessary, it's a pretty minor issue since the generated bytecode is portable anyway. |
New changeset fe29a657bde9 by Martin v. Löwis in branch 'default': |
Thanks for the patch, committed with slight modifications. |
Comments about PyImport_FrozenModules linkage have not been addressed. Now the build is failing on all Unix machines... |
To be clear, the build fails when importlib needs to be frozen again: $ touch Lib/importlib/_bootstrap.py
$ make
make Modules/_freeze_importlib
make[1] : on entre dans le répertoire « /home/antoine/cpython/default »
gcc -pthread -c -Wno-unused-result -g -O0 -Wall -Wstrict-prototypes -I. -I./Include -DPy_BUILD_CORE -o Modules/_freeze_importlib.o Modules/_freeze_importlib.c
gcc -pthread -o Modules/_freeze_importlib Modules/_freeze_importlib.o Modules/getbuildinfo.o Parser/acceler.o Parser/grammar1.o Parser/listnode.o Parser/node.o Parser/parser.o Parser/bitset.o Parser/metagrammar.o Parser/firstsets.o Parser/grammar.o Parser/pgen.o Parser/myreadline.o Parser/parsetok.o Parser/tokenizer.o Objects/abstract.o Objects/accu.o Objects/boolobject.o Objects/bytes_methods.o Objects/bytearrayobject.o Objects/bytesobject.o Objects/cellobject.o Objects/classobject.o Objects/codeobject.o Objects/complexobject.o Objects/descrobject.o Objects/enumobject.o Objects/exceptions.o Objects/genobject.o Objects/fileobject.o Objects/floatobject.o Objects/frameobject.o Objects/funcobject.o Objects/iterobject.o Objects/listobject.o Objects/longobject.o Objects/dictobject.o Objects/memoryobject.o Objects/methodobject.o Objects/moduleobject.o Objects/namespaceobject.o Objects/object.o Objects/obmalloc.o Objects/capsule.o Objects/rangeobject.o Objects/setobject.o Objects/sliceobject.o Objects/structseq.o Objects/tupleobject.o Objects/typeobject.o Objects/unicodeobject.o Objects/unicodectype.o Objects/weakrefobject.o Python/_warnings.o Python/Python-ast.o Python/asdl.o Python/ast.o Python/bltinmodule.o Python/ceval.o Python/compile.o Python/codecs.o Python/dynamic_annotations.o Python/errors.o Python/frozenmain.o Python/future.o Python/getargs.o Python/getcompiler.o Python/getcopyright.o Python/getplatform.o Python/getversion.o Python/graminit.o Python/import.o Python/importdl.o Python/marshal.o Python/modsupport.o Python/mystrtoul.o Python/mysnprintf.o Python/peephole.o Python/pyarena.o Python/pyctype.o Python/pyfpe.o Python/pymath.o Python/pystate.o Python/pythonrun.o Python/pytime.o Python/random.o Python/structmember.o Python/symtable.o Python/sysmodule.o Python/traceback.o Python/getopt.o Python/pystrcmp.o Python/pystrtod.o Python/dtoa.o Python/formatter_unicode.o Python/fileutils.o Python/dynload_shlib.o Python/thread.o Modules/config.o Modules/getpath.o Modules/main.o Modules/gcmodule.o Modules/_threadmodule.o Modules/signalmodule.o Modules/posixmodule.o Modules/errnomodule.o Modules/pwdmodule.o Modules/_sre.o Modules/_codecsmodule.o Modules/_weakref.o Modules/_functoolsmodule.o Modules/operator.o Modules/_collectionsmodule.o Modules/itertoolsmodule.o Modules/_localemodule.o Modules/_iomodule.o Modules/iobase.o Modules/fileio.o Modules/bytesio.o Modules/bufferedio.o Modules/textio.o Modules/stringio.o Modules/zipimport.o Modules/faulthandler.o Modules/symtablemodule.o Modules/xxsubtype.o -lpthread -ldl -lutil -lm
Modules/_freeze_importlib.o: In function `main':
/home/antoine/cpython/default/Modules/_freeze_importlib.c:37: undefined reference to `PyImport_FrozenModules'
Python/import.o: In function `find_frozen':
/home/antoine/cpython/default/Python/import.c:978: undefined reference to `PyImport_FrozenModules'
collect2: ld a retourné 1 code d'état d'exécution
make[1]: *** [Modules/_freeze_importlib] Erreur 1
make[1] : on quitte le répertoire « /home/antoine/cpython/default »
make: *** [Python/importlib.h] Erreur 2 |
New changeset 7967cb63e50e by Martin v. Löwis in branch 'default': |
New changeset deb421baf671 by Martin v. Löwis in branch 'default': |
The last commits should address Amaury's concerns. _freeze_importlib must now be built manually, followed by a build of pythoncore. Interestingly, the builds failed only once. When rebuilding, the builds succeeded as no attempt to build freeze_importlib was made. After the last change to freeze_importlib, they again built it, and that seems to work now. Tentatively closing the issue again. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: