classification
Title: Missing pyconfig.h when building from source and pgo flag is enabled
Type: behavior Stage: resolved
Components: Distutils, Windows Versions: Python 3.6
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: dstufft, eric.araujo, matrixise, neyuru, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2018-11-03 23:52 by neyuru, last changed 2019-01-11 06:02 by neyuru. This issue is now closed.

Messages (8)
msg329212 - (view) Author: Jorge Ramos (neyuru) * Date: 2018-11-03 23:52
When testing the builds when PGO is enabled, or by simply running rt.bat, the following error is (always) issued:

cpython\include\Python.h(8): fatal error C1083: Cannot include: 'pyconfig.h': No such file or directory

One example of steps taken is:

git clone git@github.com:python/cpython.git
cd cpython
git checkout 3.6
PCBuild\get_externals.bat
Tools\msi\get_externals.bat
PCBuild\build.bat -p x64


I have to manually copy a pyconfig.h file from a working Python's include directory to the repositorie's include directory to fix this error. By the way, I had PYTHONHOME environment variable pointing to 

...\Python36\DLLs; ...\Python36\Lib; ...\Python36\Lib\site-packages; ...\Python36\include

with no success.
msg329222 - (view) Author: Stéphane Wirtel (matrixise) * (Python committer) Date: 2018-11-04 11:19
I started a compilation with 
PCBuild\build.bat -d -p x64 -r --pgo  on a Windows machine but I am not the owner of this machine.
msg329228 - (view) Author: Stéphane Wirtel (matrixise) * (Python committer) Date: 2018-11-04 14:19
Hi Jorge,

I am really sorry but I don't have any issue for the compilation of Python 3.6 under Windows 8.1

C:\Users\pydev\cpython-bpo-35157>PCbuild\amd64\python.exe -m test.pythoninfo                                                                                                                                         
Python debug information                                                                                                                                                                                             
========================                                                                                                                                                                                             
                                                                                                                                                                                                                     
Py_DEBUG: No (sys.gettotalrefcount() missing)                                                                                                                                                                        
_decimal.__libmpdec_version__: 2.4.2                                                                                                                                                                                 
builtins.float.double_format: IEEE, little-endian                                                                                                                                                                    
builtins.float.float_format: IEEE, little-endian                                                                                                                                                                     
datetime.datetime.now: 2018-11-04 14:18:35.618798                                                                                                                                                                    
expat.EXPAT_VERSION: expat_2.2.6                                                                                                                                                                                     
locale.encoding: cp1252                                                                                                                                                                                              
os.cpu_count: 2                                                                                                                                                                                                      
os.cwd: C:\Users\pydev\cpython-bpo-35157                                                                                                                                                                             
os.environ[COMSPEC]: C:\Windows\system32\cmd.exe                                                                                                                                                                     
os.environ[HOMEDRIVE]: C:                                                                                                                                                                                            
os.environ[HOMEPATH]: \Users\pydev                                                                                                                                                                                   
os.environ[HOME]: C:\Users\pydev                                                                                                                                                                                     
os.environ[LANG]: fr_BE.UTF-8                                                                                                                                                                                        
os.environ[PATH]: C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\TortoiseHg\;C:\Program Files\TortoiseSVN\bin;C:\Program Files\Git\cmd;C:\Progr
am Files\Microsoft Windows Performance Toolkit\;C:\Program Files\Git\bin;C:\Program Files\Microsoft\Web Platform Installer\                                                                                          
os.environ[PYTHON]: py -3.6                                                                                                                                                                                          
os.environ[TEMP]: C:\Users\pydev\AppData\Local\Temp                                                                                                                                                                  
os.environ[TMP]: C:\Users\pydev\AppData\Local\Temp                                                                                                                                                                   
os.environ[USERPROFILE]: C:\Users\pydev                                                                                                                                                                              
os.environ[VS100COMNTOOLS]: C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\Tools\                                                                                                                       
os.environ[VS140COMNTOOLS]: C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\Tools\                                                                                                                       
os.environ[VS90COMNTOOLS]: C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\Tools\                                                                                                                         
os.login: pydev                                                                                                                                                                                                      
os.name: nt                                                                                                                                                                                                          
os.supports_bytes_environ: False                                                                                                                                                                                     
os.supports_effective_ids: []                                                                                                                                                                                        
os.supports_fd: ['stat', 'truncate']                                                                                                                                                                                 
os.supports_follow_symlinks: ['stat']                                                                                                                                                                                
os.umask: 000                                                                                                                                                                                                        
platform.architecture: 64bit WindowsPE                                                                                                                                                                               
platform.platform: Windows-8.1                                                                                                                                                                                       
platform.python_implementation: CPython                                                                                                                                                                              
socket.hostname: win81bb                                                                                                                                                                                             
sqlite3.sqlite_version: 3.21.0                                                                                                                                                                                       
sqlite3.version: 2.6.0                                                                                                                                                                                               
ssl.HAS_SNI: True                                                                                                                                                                                                    
ssl.OPENSSL_VERSION: OpenSSL 1.0.2p  14 Aug 2018                                                                                                                                                                     
ssl.OPENSSL_VERSION_INFO: (1, 0, 2, 16, 15)                                                                                                                                                                          
ssl.OP_ALL: -0x7ffffc01                                                                                                                                                                                              
ssl.OP_NO_TLSv1_1: 0x10000000                                                                                                                                                                                        
sys.api_version: 1013                                                                                                                                                                                                
sys.builtin_module_names: ('_ast', '_bisect', '_blake2', '_codecs', '_codecs_cn', '_codecs_hk', '_codecs_iso2022', '_codecs_jp', '_codecs_kr', '_codecs_tw', '_collections', '_csv', '_datetime', '_functools', '_hea
pq', '_imp', '_io', '_json', '_locale', '_lsprof', '_md5', '_multibytecodec', '_opcode', '_operator', '_pickle', '_random', '_sha1', '_sha256', '_sha3', '_sha512', '_signal', '_sre', '_stat', '_string', '_struct',
 '_symtable', '_thread', '_tracemalloc', '_warnings', '_weakref', '_winapi', 'array', 'atexit', 'audioop', 'binascii', 'builtins', 'cmath', 'errno', 'faulthandler', 'gc', 'itertools', 'marshal', 'math', 'mmap', 'm
svcrt', 'nt', 'parser', 'sys', 'time', 'winreg', 'xxsubtype', 'zipimport', 'zlib')                                                                                                                                   
sys.byteorder: little                                                                                                                                                                                                
sys.dont_write_bytecode: False                                                                                                                                                                                       
sys.executable: C:\Users\pydev\cpython-bpo-35157\PCbuild\amd64\python.exe                                                                                                                                            
sys.filesystem_encoding: utf-8/surrogatepass                                                                                                                                                                         
sys.flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)   
sys.float_info: sys.float_info(max=1.7976931348623157e+308, max_exp=1024, max_10_exp=308, min=2.2250738585072014e-308, min_exp=-1021, min_10_exp=-307, dig=15, mant_dig=53, epsilon=2.220446049250313e-16, radix=2, r
ounds=1)                                                                                                                                                                                                             
sys.float_repr_style: short                                                                                                                                                                                          
sys.hash_info: sys.hash_info(width=64, modulus=2305843009213693951, inf=314159, nan=0, imag=1000003, algorithm='siphash24', hash_bits=64, seed_bits=128, cutoff=0)                                                   
sys.hexversion: 50726896                                                                                                                                                                                             
sys.implementation: namespace(cache_tag='cpython-36', hexversion=50726896, name='cpython', version=sys.version_info(major=3, minor=6, micro=7, releaselevel='final', serial=0))                                      
sys.int_info: sys.int_info(bits_per_digit=30, sizeof_digit=4)                                                                                                                                                        
sys.maxsize: 9223372036854775807                                                                                                                                                                                     
sys.maxunicode: 1114111                                                                                                                                                                                              
sys.path: ['', 'C:\\Users\\pydev\\cpython-bpo-35157\\PCbuild\\amd64\\python36.zip', 'C:\\Users\\pydev\\cpython-bpo-35157\\DLLs', 'C:\\Users\\pydev\\cpython-bpo-35157\\lib', 'C:\\Users\\pydev\\cpython-bpo-35157\\PC
build\\amd64', 'C:\\Users\\pydev\\AppData\\Roaming\\Python\\Python36\\site-packages', 'C:\\Users\\pydev\\cpython-bpo-35157', 'C:\\Users\\pydev\\cpython-bpo-35157\\lib\\site-packages']                              
sys.platform: win32                                                                                                                                                                                                  
sys.prefix: C:\Users\pydev\cpython-bpo-35157                                                                                                                                                                         
sys.stderr.encoding: utf-8/backslashreplace                                                                                                                                                                          
sys.stdin.encoding: utf-8/surrogateescape                                                                                                                                                                            
sys.stdout.encoding: utf-8/surrogateescape                                                                                                                                                                           
sys.thread_info: sys.thread_info(name='nt', lock=None, version=None)                                                                                                                                                 
sys.version: 3.6.7+ (heads/3.6:4614b98, Nov  4 2018, 13:07:59) [MSC v.1900 64 bit (AMD64)]                                                                                                                           
sys.version_info: sys.version_info(major=3, minor=6, micro=7, releaselevel='final', serial=0)                                                                                                                        
sys.windowsversion: sys.getwindowsversion(major=6, minor=3, build=9600, platform=2, service_pack='')                                                                                                                 
sys.winver: 3.6                                                                                                                                                                                                      
sysconfig[prefix]: C:\Users\pydev\cpython-bpo-35157                                                                                                                                                                  
test_socket.HAVE_SOCKET_ALG: False                                                                                                                                                                                   
test_socket.HAVE_SOCKET_CAN: False                                                                                                                                                                                   
test_socket.HAVE_SOCKET_RDS: False                                                                                                                                                                                   
test_support.IPV6_ENABLED: True                                                                                                                                                                                      
test_support._is_gui_available: False                                                                                                                                                                                
test_support.python_is_optimized: False                                                                                                                                                                              
time.altzone: -3600                                                                                                                                                                                                  
time.daylight: 0                                                                                                                                                                                                     
time.get_clock_info(perf_counter): namespace(adjustable=False, implementation='QueryPerformanceCounter()', monotonic=True, resolution=1e-07)                                                                         
time.get_clock_info(time): namespace(adjustable=True, implementation='GetSystemTimeAsFileTime()', monotonic=False, resolution=0.015625)                                                                              
time.time: 1541341115.6187983                                                                                                                                                                                        
time.timezone: 0                                                                                                                                                                                                     
time.tzname: ('Coordinated Universal Time', 'Coordinated Universal Time')                                                                                                                                            
tkinter.TCL_VERSION: 8.6                                                                                                                                                                                             
tkinter.TK_VERSION: 8.6                                                                                                                                                                                              
tkinter.info_patchlevel: 8.6.6                                                                                                                                                                                       
zlib.ZLIB_RUNTIME_VERSION: 1.2.11                                                                                                                                                                                    
zlib.ZLIB_VERSION: 1.2.11
msg329247 - (view) Author: Jorge Ramos (neyuru) * Date: 2018-11-04 18:28
Hello: I followed the same command: PCBuild\build.bat -d -p x64 -r --pgo and in summary, it found 5 errors (the last part of this messages contains the PGInstrument|x64 interpreter verbose). I include a number between parenthesis to show the test number:


***********************************************************************************************
test_distutils failed in 43 sec 783 ms	(107)
test_httpservers failed			(171)
test_platform failed 			(246)
test_regrtest failed in 1 min 36 sec	(273)
test_sysconfig failed			(330)
***********************************************************************************************
 
And for the sake of completeness I also runh the command "python.exe -m test.pythoninfo" which basically gave the same results as yours. But my problem is not with the exixting python installation but at compile time when using PGO.



***********************************************************************************************
Running PGInstrument|x64 interpreter...
Run tests sequentially
0:00:00 [  1/407] test_grammar
0:00:00 [  2/407] test_opcodes
0:00:00 [  3/407] test_dict
0:00:01 [  4/407] test_builtin
0:00:02 [  5/407] test_exceptions
0:00:10 [  6/407] test_types
0:00:11 [  7/407] test_unittest
0:00:18 [  8/407] test_doctest
0:00:25 [  9/407] test_doctest2
0:00:26 [ 10/407] test_support
0:00:26 [ 11/407] test___all__
0:00:33 [ 12/407] test___future__
0:00:33 [ 13/407] test__locale
0:00:33 [ 14/407] test__opcode
0:00:33 [ 15/407] test__osx_support
0:00:33 [ 16/407] test_abc
0:00:34 [ 17/407] test_abstract_numbers
0:00:34 [ 18/407] test_aifc
0:00:35 [ 19/407] test_argparse
0:00:41 [ 20/407] test_array
0:00:46 [ 21/407] test_asdl_parser
0:00:46 [ 22/407] test_ast -- test_asdl_parser skipped
0:00:52 [ 23/407] test_asyncgen
0:00:53 [ 24/407] test_asynchat
0:00:56 [ 25/407] test_asyncio
Executing <Task finished coro=<CoroutineTests.test_async_def_wrapped.<locals>.start() done, defined at E:\RepoGiT\cpython\lib\test\test_asyncio\test_pep492.py:150> result=None created at E:\RepoGiT\cpython\lib\asyncio\base_events.py:452> took 0.141 seconds
0:02:28 [ 26/407] test_asyncore -- test_asyncio passed in 1 min 32 sec
0:02:28 [ 27/407] test_atexit -- test_asyncore skipped
0:02:29 [ 28/407] test_audioop
0:02:29 [ 29/407] test_augassign
0:02:29 [ 30/407] test_base64
0:02:32 [ 31/407] test_baseexception
0:02:32 [ 32/407] test_bdb
0:02:32 [ 33/407] test_bigaddrspace
0:02:32 [ 34/407] test_bigmem
0:02:33 [ 35/407] test_binascii
0:02:33 [ 36/407] test_binhex
0:02:33 [ 37/407] test_binop
0:02:34 [ 38/407] test_bisect
0:02:34 [ 39/407] test_bool
0:02:34 [ 40/407] test_buffer
0:03:18 [ 41/407] test_bufio -- test_buffer passed in 43 sec 970 ms
0:03:22 [ 42/407] test_bytes
0:03:31 [ 43/407] test_bz2
0:03:39 [ 44/407] test_calendar
0:03:58 [ 45/407] test_call
0:03:59 [ 46/407] test_capi
0:04:12 [ 47/407] test_cgi
0:04:12 [ 48/407] test_cgitb
0:04:14 [ 49/407] test_charmapcodec
0:04:15 [ 50/407] test_class
0:04:15 [ 51/407] test_cmath
0:04:15 [ 52/407] test_cmd
0:04:16 [ 53/407] test_cmd_line
0:04:38 [ 54/407] test_cmd_line_script
0:05:19 [ 55/407] test_code -- test_cmd_line_script passed in 41 sec 453 ms
0:05:19 [ 56/407] test_code_module
0:05:20 [ 57/407] test_codeccallbacks
0:05:20 [ 58/407] test_codecencodings_cn
0:05:21 [ 59/407] test_codecencodings_hk
0:05:22 [ 60/407] test_codecencodings_iso2022
0:05:23 [ 61/407] test_codecencodings_jp
0:05:26 [ 62/407] test_codecencodings_kr
0:05:27 [ 63/407] test_codecencodings_tw
0:05:27 [ 64/407] test_codecmaps_cn
0:05:27 [ 65/407] test_codecmaps_hk
0:05:28 [ 66/407] test_codecmaps_jp
0:05:28 [ 67/407] test_codecmaps_kr
0:05:28 [ 68/407] test_codecmaps_tw
0:05:28 [ 69/407] test_codecs
0:05:35 [ 70/407] test_codeop
0:05:35 [ 71/407] test_collections
0:05:39 [ 72/407] test_colorsys
0:05:39 [ 73/407] test_compare
0:05:39 [ 74/407] test_compile
0:05:40 [ 75/407] test_compileall
0:06:27 [ 76/407] test_complex -- test_compileall passed in 46 sec 78 ms
0:06:28 [ 77/407] test_concurrent_futures
0:08:34 [ 78/407] test_configparser -- test_concurrent_futures passed in 2 min 6 sec
0:08:36 [ 79/407] test_contains
0:08:37 [ 80/407] test_contextlib
0:08:37 [ 81/407] test_copy
0:08:37 [ 82/407] test_copyreg
0:08:38 [ 83/407] test_coroutines
0:08:39 [ 84/407] test_cprofile
0:08:40 [ 85/407] test_crashers
0:08:40 [ 86/407] test_crypt
0:08:40 [ 87/407] test_csv -- test_crypt skipped
0:08:41 [ 88/407] test_ctypes
0:08:44 [ 89/407] test_curses
0:08:44 [ 90/407] test_datetime -- test_curses skipped (resource denied)
0:08:50 [ 91/407] test_dbm
0:08:51 [ 92/407] test_dbm_dumb
0:08:52 [ 93/407] test_dbm_gnu
0:08:52 [ 94/407] test_dbm_ndbm -- test_dbm_gnu skipped
0:08:52 [ 95/407] test_decimal -- test_dbm_ndbm skipped
0:09:14 [ 96/407] test_decorators
0:09:14 [ 97/407] test_defaultdict
0:09:14 [ 98/407] test_deque
0:09:25 [ 99/407] test_descr
0:09:30 [100/407] test_descrtut
0:09:31 [101/407] test_devpoll
0:09:31 [102/407] test_dict_version -- test_devpoll skipped
0:09:31 [103/407] test_dictcomps
0:09:31 [104/407] test_dictviews
0:09:32 [105/407] test_difflib
0:09:37 [106/407] test_dis
0:09:38 [107/407] test_distutils
xxmodule.c
E:\RepoGiT\cpython\include\Python.h(8): fatal error C1083: No se puede abrir el archivo incluir: 'pyconfig.h': No such file or directory
foo.c
   Creando biblioteca D:\Users\yorch\AppData\Local\Temp\tmpjxcggbzn\tempt\Users\yorch\AppData\Local\Temp\tmpi66qfp2r\foo.cp36-win_amd64.lib y objeto D:\Users\yorch\AppData\Local\Temp\tmpjxcggbzn\tempt\Users\yorch\AppData\Local\Temp\tmpi66qfp2r\foo.cp36-win_amd64.exp
Generando código
Generación de código finalizada
foo.c
   Creando biblioteca D:\Users\yorch\AppData\Local\Temp\tmpjxcggbzn\tempt\Users\yorch\AppData\Local\Temp\tmpi66qfp2r\foo.cp36-win_amd64.lib y objeto D:\Users\yorch\AppData\Local\Temp\tmpjxcggbzn\tempt\Users\yorch\AppData\Local\Temp\tmpi66qfp2r\foo.cp36-win_amd64.exp
Generando código
Generación de código finalizada
xxmodule.c
E:\RepoGiT\cpython\include\Python.h(8): fatal error C1083: No se puede abrir el archivo incluir: 'pyconfig.h': No such file or directory
foo.c
   Creando biblioteca D:\Users\yorch\AppData\Local\Temp\tmpc9c13nr5\tempt\Users\yorch\AppData\Local\Temp\tmpb0jw9032\foo.cp36-win_amd64.lib y objeto D:\Users\yorch\AppData\Local\Temp\tmpc9c13nr5\tempt\Users\yorch\AppData\Local\Temp\tmpb0jw9032\foo.cp36-win_amd64.exp
Generando código
Generación de código finalizada
foo.c
   Creando biblioteca D:\Users\yorch\AppData\Local\Temp\tmpc9c13nr5\tempt\Users\yorch\AppData\Local\Temp\tmpb0jw9032\foo.cp36-win_amd64.lib y objeto D:\Users\yorch\AppData\Local\Temp\tmpc9c13nr5\tempt\Users\yorch\AppData\Local\Temp\tmpb0jw9032\foo.cp36-win_amd64.exp
Generando código
Generación de código finalizada
xxmodule.c
E:\RepoGiT\cpython\include\Python.h(8): fatal error C1083: No se puede abrir el archivo incluir: 'pyconfig.h': No such file or directory

D:\Users\yorch\AppData\Local\Temp\test_python_17544>exit 1

D:\Users\yorch\AppData\Local\Temp\test_python_17544>exit 0
0:10:22 [108/407] test_docxmlrpc -- test_distutils failed in 43 sec 783 ms
0:10:32 [109/407] test_dtrace
0:10:32 [110/407] test_dummy_thread
0:10:32 [111/407] test_dummy_threading
0:10:32 [112/407] test_dynamic
0:10:33 [113/407] test_dynamicclassattribute
0:10:33 [114/407] test_eintr
0:10:33 [115/407] test_email
0:10:46 [116/407] test_ensurepip
0:10:47 [117/407] test_enum
0:10:48 [118/407] test_enumerate
0:10:49 [119/407] test_eof
0:10:49 [120/407] test_epoll
0:10:49 [121/407] test_errno -- test_epoll skipped
0:10:50 [122/407] test_exception_hierarchy
0:10:50 [123/407] test_exception_variations
0:10:50 [124/407] test_extcall
0:10:51 [125/407] test_faulthandler
0:11:19 [126/407] test_fcntl
0:11:19 [127/407] test_file -- test_fcntl skipped
0:11:20 [128/407] test_file_eintr
0:11:20 [129/407] test_filecmp
0:11:21 [130/407] test_fileinput
0:11:21 [131/407] test_fileio
0:11:22 [132/407] test_finalization
0:11:26 [133/407] test_float
0:11:27 [134/407] test_flufl
0:11:27 [135/407] test_fnmatch
0:11:28 [136/407] test_fork1
0:11:28 [137/407] test_format -- test_fork1 skipped
0:11:28 [138/407] test_fractions
0:11:29 [139/407] test_frame
0:11:29 [140/407] test_fstring
0:11:31 [141/407] test_ftplib
0:11:40 [142/407] test_funcattrs
0:11:40 [143/407] test_functools
0:11:41 [144/407] test_future
0:11:42 [145/407] test_future3
0:11:42 [146/407] test_future4
0:11:42 [147/407] test_future5
0:11:43 [148/407] test_gc
0:11:52 [149/407] test_gdb
0:11:53 [150/407] test_generator_stop -- test_gdb skipped
0:11:53 [151/407] test_generators
0:11:54 [152/407] test_genericpath
0:11:54 [153/407] test_genexps
0:11:55 [154/407] test_getargs2
0:11:55 [155/407] test_getopt
0:11:55 [156/407] test_getpass
0:11:56 [157/407] test_gettext
0:11:56 [158/407] test_glob
0:12:02 [159/407] test_global
0:12:02 [160/407] test_grp
0:12:02 [161/407] test_gzip -- test_grp skipped
0:12:05 [162/407] test_hash
0:12:14 [163/407] test_hashlib
0:12:16 [164/407] test_heapq
0:12:20 [165/407] test_hmac
0:12:21 [166/407] test_html
0:12:21 [167/407] test_htmlparser
0:12:22 [168/407] test_http_cookiejar
0:12:22 [169/407] test_http_cookies
0:12:23 [170/407] test_httplib
0:12:31 [171/407] test_httpservers
0:12:38 [172/407] test_idle -- test_httpservers failed
0:12:41 [173/407] test_imaplib
0:12:50 [174/407] test_imghdr
0:12:50 [175/407] test_imp
0:12:50 [176/407] test_import
0:12:55 [177/407] test_importlib
0:13:05 [178/407] test_index
0:13:05 [179/407] test_inspect
0:13:09 [180/407] test_int
0:13:10 [181/407] test_int_literal
0:13:10 [182/407] test_io
0:13:55 [183/407] test_ioctl -- test_io passed in 45 sec 312 ms
0:13:56 [184/407] test_ipaddress -- test_ioctl skipped
0:13:57 [185/407] test_isinstance
0:13:57 [186/407] test_iter
0:13:58 [187/407] test_iterlen
0:13:59 [188/407] test_itertools
0:14:22 [189/407] test_json
0:14:32 [190/407] test_keyword
0:14:35 [191/407] test_keywordonlyarg
0:14:35 [192/407] test_kqueue
0:14:36 [193/407] test_largefile -- test_kqueue skipped
0:14:36 [194/407] test_lib2to3
0:15:42 [195/407] test_linecache -- test_lib2to3 passed in 1 min 5 sec
0:15:42 [196/407] test_list
0:15:44 [197/407] test_listcomps
0:15:44 [198/407] test_locale
0:15:45 [199/407] test_logging
0:16:03 [200/407] test_long
0:16:18 [201/407] test_longexp
0:16:19 [202/407] test_lzma
0:16:25 [203/407] test_macpath
0:16:26 [204/407] test_macurl2path
0:16:26 [205/407] test_mailbox
0:16:31 [206/407] test_mailcap
0:16:31 [207/407] test_marshal
0:16:33 [208/407] test_math
0:16:42 [209/407] test_memoryio
0:16:43 [210/407] test_memoryview
0:16:47 [211/407] test_metaclass
0:16:47 [212/407] test_mimetypes
0:16:48 [213/407] test_minidom
0:16:49 [214/407] test_mmap
0:16:50 [215/407] test_module
0:16:52 [216/407] test_modulefinder
0:16:54 [217/407] test_msilib
0:16:54 [218/407] test_multibytecodec
0:17:01 [219/407] test_multiprocessing_fork
0:17:01 [220/407] test_multiprocessing_forkserver -- test_multiprocessing_fork skipped
0:17:01 [221/407] test_multiprocessing_main_handling -- test_multiprocessing_forkserver skipped
0:17:02 [222/407] test_multiprocessing_spawn -- test_multiprocessing_main_handling skipped
0:17:02 [223/407] test_netrc -- test_multiprocessing_spawn skipped
0:17:03 [224/407] test_nis
0:17:03 [225/407] test_nntplib -- test_nis skipped
0:17:04 [226/407] test_normalization
0:17:04 [227/407] test_ntpath
0:17:05 [228/407] test_numeric_tower
0:17:07 [229/407] test_openpty
0:17:07 [230/407] test_operator -- test_openpty skipped
0:17:08 [231/407] test_optparse
0:17:08 [232/407] test_ordered_dict
0:17:16 [233/407] test_os
0:17:33 [234/407] test_ossaudiodev
0:17:34 [235/407] test_osx_env -- test_ossaudiodev skipped (resource denied)
0:17:34 [236/407] test_parser
0:17:35 [237/407] test_pathlib
0:17:38 [238/407] test_pdb
0:17:41 [239/407] test_peepholer
0:17:42 [240/407] test_pickle
0:18:17 [241/407] test_pickletools -- test_pickle passed in 35 sec 298 ms
0:18:25 [242/407] test_pipes
0:18:26 [243/407] test_pkg -- test_pipes skipped
0:18:26 [244/407] test_pkgimport
0:18:27 [245/407] test_pkgutil
0:18:27 [246/407] test_platform
b''
b"Fatal Python error: Py_Initialize: unable to load the file system codec\nModuleNotFoundError: No module named 'encodings'\n\nCurrent thread 0x000036ac (most recent call first):\n"
0:18:30 [247/407] test_plistlib -- test_platform failed
0:18:30 [248/407] test_poll
0:18:31 [249/407] test_popen -- test_poll skipped
0:18:32 [250/407] test_poplib
0:18:39 [251/407] test_posix
0:18:40 [252/407] test_posixpath -- test_posix skipped
0:18:41 [253/407] test_pow
0:18:42 [254/407] test_pprint
0:18:42 [255/407] test_print
0:18:43 [256/407] test_profile
0:18:43 [257/407] test_property
0:18:44 [258/407] test_pstats
0:18:44 [259/407] test_pty
0:18:45 [260/407] test_pulldom -- test_pty skipped
0:18:45 [261/407] test_pwd
0:18:46 [262/407] test_py_compile -- test_pwd skipped
0:18:46 [263/407] test_pyclbr
0:18:57 [264/407] test_pydoc
0:19:08 [265/407] test_pyexpat
0:19:09 [266/407] test_queue
0:19:13 [267/407] test_quopri
0:19:14 [268/407] test_raise
0:19:14 [269/407] test_random
0:19:19 [270/407] test_range
0:19:23 [271/407] test_re
0:19:29 [272/407] test_readline
0:19:29 [273/407] test_regrtest -- test_readline skipped
0:21:06 [274/407] test_repl -- test_regrtest failed in 1 min 36 sec
0:21:07 [275/407] test_reprlib
0:21:07 [276/407] test_resource
0:21:08 [277/407] test_richcmp -- test_resource skipped
0:21:08 [278/407] test_rlcompleter
0:21:09 [279/407] test_robotparser
0:21:09 [280/407] test_runpy
0:21:20 [281/407] test_sax
0:21:21 [282/407] test_sched
0:21:22 [283/407] test_scope
0:21:23 [284/407] test_script_helper
0:21:25 [285/407] test_secrets
0:21:25 [286/407] test_select
0:21:26 [287/407] test_selectors
0:21:28 [288/407] test_set
0:21:36 [289/407] test_setcomps
0:21:37 [290/407] test_shelve
0:21:38 [291/407] test_shlex
0:21:38 [292/407] test_shutil
0:21:40 [293/407] test_signal
0:21:43 [294/407] test_site
0:21:52 [295/407] test_slice
0:21:53 [296/407] test_smtpd
0:21:54 [297/407] test_smtplib
0:21:55 [298/407] test_smtpnet
0:21:56 [299/407] test_sndhdr -- test_smtpnet skipped (resource denied)
0:21:56 [300/407] test_socket
0:22:20 [301/407] test_socketserver
0:22:20 [302/407] test_sort -- test_socketserver skipped (resource denied)
0:22:21 [303/407] test_source_encoding
0:22:26 [304/407] test_spwd
0:22:26 [305/407] test_sqlite -- test_spwd skipped
0:22:28 [306/407] test_ssl
0:22:37 [307/407] test_startfile
0:22:38 [308/407] test_stat
0:22:39 [309/407] test_statistics
0:22:43 [310/407] test_strftime
0:22:44 [311/407] test_string
0:22:45 [312/407] test_string_literals
0:22:47 [313/407] test_stringprep
0:22:47 [314/407] test_strptime
0:22:48 [315/407] test_strtod
0:22:53 [316/407] test_struct
0:22:56 [317/407] test_structmembers
0:22:57 [318/407] test_structseq
0:22:57 [319/407] test_subclassinit
0:22:58 [320/407] test_subprocess
0:22:58 [321/407] test_sunau -- test_subprocess skipped
0:22:59 [322/407] test_sundry
0:22:59 [323/407] test_super
0:23:00 [324/407] test_symbol
0:23:01 [325/407] test_symtable
0:23:01 [326/407] test_syntax
0:23:02 [327/407] test_sys
0:23:13 [328/407] test_sys_setprofile
0:23:14 [329/407] test_sys_settrace
0:23:15 [330/407] test_sysconfig
(b'', b"Fatal Python error: Py_Initialize: unable to load the file system codec\nModuleNotFoundError: No module named 'encodings'\n\nCurrent thread 0x00001ff4 (most recent call first):\n")
0:23:16 [331/407] test_syslog -- test_sysconfig failed
0:23:16 [332/407] test_tarfile -- test_syslog skipped
0:24:10 [333/407] test_tcl -- test_tarfile passed in 53 sec 625 ms
0:24:13 [334/407] test_telnetlib
0:24:13 [335/407] test_tempfile
0:24:24 [336/407] test_textwrap
0:24:24 [337/407] test_thread
0:24:26 [338/407] test_threaded_import
0:24:29 [339/407] test_threadedtempfile
0:24:30 [340/407] test_threading
0:24:45 [341/407] test_threading_local
0:24:47 [342/407] test_threadsignals
0:24:48 [343/407] test_time -- test_threadsignals skipped
0:24:50 [344/407] test_timeit
0:24:51 [345/407] test_timeout
0:24:51 [346/407] test_tix -- test_timeout skipped (resource denied)
0:24:52 [347/407] test_tk -- test_tix skipped (resource denied)
0:24:52 [348/407] test_tokenize -- test_tk skipped (resource denied)
0:25:01 [349/407] test_tools
0:25:34 [350/407] test_trace -- test_tools passed in 33 sec 298 ms
0:26:01 [351/407] test_traceback
0:26:10 [352/407] test_tracemalloc
0:26:17 [353/407] test_ttk_guionly
0:26:59 [354/407] test_ttk_textonly -- test_ttk_guionly skipped (resource denied) in 42 sec 625 ms
0:27:00 [355/407] test_tuple
0:27:18 [356/407] test_turtle
0:27:18 [357/407] test_typechecks
0:27:19 [358/407] test_typing
0:27:20 [359/407] test_ucn
0:27:21 [360/407] test_unary
0:27:22 [361/407] test_unicode
0:27:28 [362/407] test_unicode_file
0:27:28 [363/407] test_unicode_file_functions
0:27:30 [364/407] test_unicode_identifiers
0:27:30 [365/407] test_unicodedata
0:27:44 [366/407] test_univnewlines
0:27:45 [367/407] test_unpack
0:27:45 [368/407] test_unpack_ex
0:27:46 [369/407] test_urllib
0:27:48 [370/407] test_urllib2
0:27:50 [371/407] test_urllib2_localnet
0:28:10 [372/407] test_urllib2net
0:28:11 [373/407] test_urllib_response -- test_urllib2net skipped (resource denied)
0:28:11 [374/407] test_urllibnet
0:28:12 [375/407] test_urlparse -- test_urllibnet skipped (resource denied)
0:28:12 [376/407] test_userdict
0:28:13 [377/407] test_userlist
0:28:14 [378/407] test_userstring
0:28:21 [379/407] test_utf8source
0:28:21 [380/407] test_uu
0:28:22 [381/407] test_uuid
0:28:22 [382/407] test_venv
0:29:19 [383/407] test_wait3 -- test_venv passed in 56 sec 531 ms
0:29:19 [384/407] test_wait4 -- test_wait3 skipped
0:29:20 [385/407] test_warnings -- test_wait4 skipped
0:29:28 [386/407] test_wave
0:29:29 [387/407] test_weakref
0:30:48 [388/407] test_weakset -- test_weakref passed in 1 min 18 sec
0:30:50 [389/407] test_webbrowser
0:30:50 [390/407] test_winconsoleio
Ä^Z
abc123
ϼўТλФЙ
A͏B ﬖ̳AA̝
􀀀􏿿􏿽
ϼўТλФЙ
ϼўТλФЙ
ϼўТλФЙ
ϼўТλФЙ
ϼўТλФЙ
ϼўТλФЙ
ϼўТλФЙ
ϼўТλФЙ
ϼўТλФЙ
ϼўТλФЙ
ϼўТλФЙ
ϼўТλФЙ
ϼўТλФЙ
ϼўТλФЙ
ϼўТλФЙ
􁿿􁀁
􁿿􁀁
􁿿􁀁
􁿿􁀁
􁿿􁀁
􁿿􁀁
􁿿􁀁
􁿿􁀁
􁿿􁀁
􁿿􁀁
􁿿􁀁
􁿿􁀁
􁿿􁀁
􁿿􁀁
􁿿􁀁
0:30:51 [391/407] test_winreg
0:30:54 [392/407] test_winsound
0:30:54 [393/407] test_with -- test_winsound skipped (resource denied)
0:30:55 [394/407] test_wsgiref
0:30:56 [395/407] test_xdrlib
0:30:56 [396/407] test_xml_dom_minicompat
0:30:57 [397/407] test_xml_etree
0:31:00 [398/407] test_xml_etree_c
0:31:04 [399/407] test_xmlrpc
0:31:11 [400/407] test_xmlrpc_net
0:31:11 [401/407] test_yield_from -- test_xmlrpc_net skipped (resource denied)
0:31:12 [402/407] test_zipapp
0:31:13 [403/407] test_zipfile
0:31:38 [404/407] test_zipfile64
0:31:39 [405/407] test_zipimport -- test_zipfile64 skipped (resource denied)
0:31:40 [406/407] test_zipimport_support
0:31:44 [407/407] test_zlib

Total duration: 31 min 46 sec
Tests result: FAILURE

E:\repogit\cpython>"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\\MSBuild\15.0\Bin\msbuild.exe" "E:\RepoGiT\cpython\PCbuild\\pythoncore.vcxproj" /t:KillPython /nologo /v:m /p:Configuration=PGInstrument /p:Platform=x64 /p:KillPython=true
  Killing any running python.exe instances...

E:\repogit\cpython>"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\\MSBuild\15.0\Bin\msbuild.exe" "E:\RepoGiT\cpython\PCbuild\pcbuild.proj" /t:Build /m /nologo /v:m /p:Configuration=PGUpdate /p:Platform=x64 /p:IncludeExternals=true /p:IncludeSSL=true /p:IncludeTkinter=true /p:UseTestMarker= /p:GIT="C:\Program Files\Git\cmd\git.exe"
  Killing any running python.exe instances...
E:\RepoGiT\cpython\PCbuild\pyproject.props(173,5): error : PGO run did not succeed (no python36!*.pgc files) and there is no data to merge [E:\RepoGiT\cpython\PCbuild\pythoncore.vcxproj]
msg329298 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2018-11-05 14:16
Those are just the distutils tests failing, which probably means the extra environment variables or paths involved in the PGO profiling run are confusing the logic for detecting when it is running in a source tree.

We should fix it, but it won't affect either PGO or to resulting build.
msg329305 - (view) Author: Jorge Ramos (neyuru) * Date: 2018-11-05 16:25
Hi:

Hmm.. but the installation does fail ("PGO run did not succeed (no python36!*.pgc files) and there is no data to merge"):

*************************************************************************
Total duration: 35 min 2 sec
Tests result: FAILURE

E:\repogit\cpython>"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\\MSBuild\15.0\Bin\msbuild.exe" "E:\RepoGiT\cpython\PCbuild\\pythoncore.vcxproj" /t:KillPython /nologo /v:m /p:Configuration=PGInstrument /p:Platform=x64 /p:KillPython=true
  Killing any running python.exe instances...

E:\repogit\cpython>"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\\MSBuild\15.0\Bin\msbuild.exe" "E:\RepoGiT\cpython\PCbuild\pcbuild.proj" /t:Build /m /nologo /v:m /p:Configuration=PGUpdate /p:Platform=x64 /p:IncludeExternals=true /p:IncludeSSL=true /p:IncludeTkinter=true /p:UseTestMarker= /p:GIT="C:\Program Files\Git\cmd\git.exe"
  Killing any running python.exe instances...
E:\RepoGiT\cpython\PCbuild\pyproject.props(173,5): error : PGO run did not succeed (no python36!*.pgc files) and there is no data to merge [E:\RepoGiT\cpython\PCbuild\pythoncore.vcxproj]

************************************************************************


So I can´t omit these issues and call it a day. =(
msg329333 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2018-11-06 03:13
How much have you edited the error messages you are posting? Those two build commands shouldn't be next to each other (the tests should be in between), so perhaps something is wrong in that script? Or maybe you trimmed the output before pasting it?
msg329343 - (view) Author: Jorge Ramos (neyuru) * Date: 2018-11-06 04:59
Yes, my last message is a trimmed version of my next to last message (which itself is a trimmed version of all the verbose of the build, showing just after the PGO interpreter wakes up). Although, obviously they where run at different times (one run completed in 31 minutes while the other was in 35 minutes), because I no longer had the output of my next to last message available.

Other than that, the messages are 'identical'. The commands followed to get to this are:

git clone git@github.com:neyuru/cpython.git
git remote add upstream git@github.com:python/cpython.git
git remote -v
cd cpython
git checkout 3.6
PCBuild\get_externals.bat
Tools\msi\get_externals.bat
PCBuild\build.bat -d -p x64
PCBuild\build.bat --pgo -p x64
History
Date User Action Args
2019-01-11 06:02:34neyurusetstatus: open -> closed
stage: resolved
2018-11-06 04:59:21neyurusetmessages: + msg329343
2018-11-06 03:13:17steve.dowersetmessages: + msg329333
2018-11-05 16:25:36neyurusetmessages: + msg329305
2018-11-05 14:16:56steve.dowersetnosy: + eric.araujo, dstufft
messages: + msg329298

components: + Distutils, - Build
type: compile error -> behavior
2018-11-04 18:28:41neyurusetmessages: + msg329247
2018-11-04 14:19:17matrixisesetmessages: + msg329228
2018-11-04 11:19:29matrixisesetnosy: + paul.moore, tim.golden, steve.dower
components: + Windows
2018-11-04 11:19:02matrixisesetnosy: + matrixise, zach.ware

messages: + msg329222
title: Missing pyconfig.h when building from source -> Missing pyconfig.h when building from source and pgo flag is enabled
2018-11-03 23:52:48neyurucreate