This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: ./Programs/_freeze_module fails with MSAN: Uninitialized value was created by an allocation of 'stat.i'
Type: Stage: patch review
Components: Build Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: vstinner
Priority: normal Keywords: patch

Created on 2022-03-01 09:38 by vstinner, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 31633 open vstinner, 2022-03-01 14:52
Messages (6)
msg414249 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-01 09:38
"./Programs/_freeze_module zipimport" fails with MSAN:
---
$ make SHELL="bash -x"
(...)
+ ./Programs/_freeze_module zipimport ./Lib/zipimport.py Python/frozen_modules/zipimport.h
==110524==WARNING: MemorySanitizer: use-of-uninitialized-value
==110523==WARNING: MemorySanitizer: use-of-uninitialized-value
==110526==WARNING: MemorySanitizer: use-of-uninitialized-value
==110525==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x4b8cfc in read_text /home/vstinner/python/main/Programs/_freeze_module.c:109:9
    #1 0x4b8cfc in main /home/vstinner/python/main/Programs/_freeze_module.c:224:24
    #2 0x7fe8acb8555f in __libc_start_call_main (/lib64/libc.so.6+0x2d55f)
    #3 0x7fe8acb8560b in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x2d60b)
    #4 0x435f44 in _start (/home/vstinner/python/main/Programs/_freeze_module+0x435f44)

  Uninitialized value was created by an allocation of 'stat.i' in the stack frame of function 'main'
    #0 0x4b72e0 in main /home/vstinner/python/main/Programs/_freeze_module.c:205

SUMMARY: MemorySanitizer: use-of-uninitialized-value /home/vstinner/python/main/Programs/_freeze_module.c:109:9 in read_text
Exiting
make: *** [Makefile:1184: Python/frozen_modules/importlib._bootstrap_external.h] Error 1
make: *** Waiting for unfinished jobs....
    #0 0x4b8cfc in read_text /home/vstinner/python/main/Programs/_freeze_module.c:109:9
    #1 0x4b8cfc in main /home/vstinner/python/main/Programs/_freeze_module.c:224:24
    #2 0x7f64ecd1355f in __libc_start_call_main (/lib64/libc.so.6+0x2d55f)
    #3 0x7f64ecd1360b in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x2d60b)
    #4 0x435f44 in _start (/home/vstinner/python/main/Programs/_freeze_module+0x435f44)

  Uninitialized value was created by an allocation of 'stat.i' in the stack frame of function 'main'
    #0 0x4b72e0 in main /home/vstinner/python/main/Programs/_freeze_module.c:205

SUMMARY: MemorySanitizer: use-of-uninitialized-value /home/vstinner/python/main/Programs/_freeze_module.c:109:9 in read_text
Exiting
make: *** [Makefile:1176: Python/frozen_modules/getpath.h] Error 1
    #0 0x4b8cfc in read_text /home/vstinner/python/main/Programs/_freeze_module.c:109:9
    #1 0x4b8cfc in main /home/vstinner/python/main/Programs/_freeze_module.c:224:24
    #2 0x7fb2a178d55f in __libc_start_call_main (/lib64/libc.so.6+0x2d55f)
    #3 0x7fb2a178d60b in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x2d60b)
    #4 0x435f44 in _start (/home/vstinner/python/main/Programs/_freeze_module+0x435f44)

    #0 0x4b8cfc in read_text /home/vstinner/python/main/Programs/_freeze_module.c:109:9
    #1 0x4b8cfc in main /home/vstinner/python/main/Programs/_freeze_module.c:224:24
    #2 0x7f4d471ca55f in __libc_start_call_main (/lib64/libc.so.6+0x2d55f)
    #3 0x7f4d471ca60b in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x2d60b)
    #4 0x435f44 in _start (/home/vstinner/python/main/Programs/_freeze_module+0x435f44)

  Uninitialized value was created by an allocation of 'stat.i' in the stack frame of function 'main'
  Uninitialized value was created by an allocation of 'stat.i' in the stack frame of function 'main'
    #0 0x4b72e0 in main /home/vstinner/python/main/Programs/_freeze_module.c:205

    #0 0x4b72e0 in main /home/vstinner/python/main/Programs/_freeze_module.c:205

SUMMARY: MemorySanitizer: use-of-uninitialized-value /home/vstinner/python/main/Programs/_freeze_module.c:109:9 in read_text
SUMMARY: MemorySanitizer: use-of-uninitialized-value /home/vstinner/python/main/Programs/_freeze_module.c:109:9 in read_text
Exiting
Exiting
make: *** [Makefile:1181: Python/frozen_modules/importlib._bootstrap.h] Error 1
make: *** [Makefile:1187: Python/frozen_modules/zipimport.h] Error 1
---

Build Python with clang MSAN:
---
./configure --with-pydebug --with-memory-sanitizer CC=clang LD=clang
make
---
msg414264 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-01 14:32
It looks like a bug in clang MSAN:
https://github.com/llvm/llvm-project/issues/54131
msg414267 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-01 14:49
Hum, later "import ssl" in setup.py also fails. Simplified code:
---
import _ssl
print(_ssl.txt2obj('1.3.6.1.5.5.7.3.1', name=False)) # server OID
---

Error:
---
$ ./python x.py 
Uninitialized bytes in MemcmpInterceptorCommon at offset 0 inside [0x701000000032, 8)
==135651==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x49467c in memcmp (/home/vstinner/python/main/python+0x49467c)
    #1 0x7f5546825adc in OBJ_bsearch_ex_ (/lib64/libcrypto.so.1.1+0x14eadc)
    #2 0x7f5546826ff1 in OBJ_obj2nid (/lib64/libcrypto.so.1.1+0x14fff1)
    #3 0x7f554675b434  (/lib64/libcrypto.so.1.1+0x84434)
    #4 0x7f554675b6af in d2i_ASN1_OBJECT (/lib64/libcrypto.so.1.1+0x846af)
    #5 0x7f5546827702 in OBJ_txt2obj (/lib64/libcrypto.so.1.1+0x150702)
    #6 0x7f5546a78c31 in _ssl_txt2obj_impl /home/vstinner/python/main/Modules/_ssl.c:5276:11
    #7 0x7f5546a78c31 in _ssl_txt2obj /home/vstinner/python/main/Modules/clinic/_ssl.c.h:1216:20
    (...)

  Uninitialized value was created by a heap allocation
    #0 0x466642 in __interceptor_malloc (/home/vstinner/python/main/python+0x466642)
    #1 0x7f55468276b3 in OBJ_txt2obj (/lib64/libcrypto.so.1.1+0x1506b3)

SUMMARY: MemorySanitizer: use-of-uninitialized-value (/home/vstinner/python/main/python+0x49467c) in memcmp
Exiting
---

The Python code calls OBJ_txt2obj("1.3.6.1.5.5.7.3.1", 0) in C: the OpenSSL function.
msg414268 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-01 14:53
> It looks like a bug in clang MSAN:
> https://github.com/llvm/llvm-project/issues/54131

I wrote GH-31633 to work around the false alarm on stat() and fstat().
msg414269 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-01 14:53
> The Python code calls OBJ_txt2obj("1.3.6.1.5.5.7.3.1", 0) in C: the OpenSSL function.

This error is unrelated to Python, but comes from OpenSSL.
msg414271 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-01 15:01
> The Python code calls OBJ_txt2obj("1.3.6.1.5.5.7.3.1", 0) in C: the OpenSSL function.

I reported this issue to OpenSSL: https://github.com/openssl/openssl/issues/17784
History
Date User Action Args
2022-04-11 14:59:56adminsetgithub: 91043
2022-03-01 15:01:07vstinnersetmessages: + msg414271
2022-03-01 14:53:57vstinnersetmessages: + msg414269
2022-03-01 14:53:32vstinnersetmessages: + msg414268
2022-03-01 14:52:47vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request29755
2022-03-01 14:49:16vstinnersetmessages: + msg414267
2022-03-01 14:32:04vstinnersetmessages: + msg414264
2022-03-01 09:38:30vstinnercreate