This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Valgrind reports "Conditional jump or move depends on uninitialised value(s)" in `PyUnicode_AsEncodedString` and `PyUnicode_Decode`
Type: compile error Stage: resolved
Components: Interpreter Core Versions: Python 3.9, Python 3.8
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: bstaletic, serhiy.storchaka
Priority: normal Keywords:

Created on 2020-10-27 18:52 by bstaletic, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
test.c bstaletic, 2020-10-27 18:52 Example that repros the valgrind detecter error.
Messages (4)
msg379790 - (view) Author: Boris Staletic (bstaletic) Date: 2020-10-27 18:52
When running valgrind on a C code that calls `PyUnicode_AsEncodedString` and `PyUnicode_Decode`, valgrind reports that there's a conditional jump based on uninitialized variable, if the encoding is "latin1".

I am able to replicate the error 100% of the time, on Ubuntu 20.04, with python 3.9.0 installed with pyenv. I also have repro'd the error in my CI (link below). Steps to repro:

1. docker run -it ubuntu:20.04 /bin/bash
2. apt update
3. apt install valgrind gcc build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev xz-utils tk-dev libffi-dev liblzma-dev python-openssl git
4. curl https://pyenv.run | bash
5. export PATH="/root/.pyenv/bin:$PATH"
6. eval "$(pyenv init -)"
7. PYTHON_CONFIGURE_OPTS="--enable-shared" pyenv install 3.9.0
8. Take the attached C file.
9. gcc -ggdb3 -I/root/.pyenv/versions/3.9.0/include/python3.9 -L/root/.pyenv/versions/3.9.0/lib test2.c -lpython3.9
10. LD_LIBRARY_PATH=/root/.pyenv/versions/3.9.0/lib/ PYTHONMALLOC=malloc valgrind ./a.out

Valgrind output:

==22783== Conditional jump or move depends on uninitialised value(s)
==22783==    at 0x49ABE64: PyUnicode_Decode (unicodeobject.c:3443)
==22783==    by 0x49ABE64: PyUnicode_Decode (unicodeobject.c:3398)
==22783==    by 0x109251: main (test2.c:5)
==22783==
==22783== Conditional jump or move depends on uninitialised value(s)
==22783==    at 0x499A294: PyUnicode_AsEncodedString (unicodeobject.c:3732)
==22783==    by 0x499A294: PyUnicode_AsEncodedString (unicodeobject.c:3688)
==22783==    by 0x10926D: main (test2.c:6)


CI log: https://dev.azure.com/borisstaletic/3ce92110-caa5-4c49-b8c3-44a433da676b/_apis/build/builds/1338/logs/6
Repository for testing the bug: https://github.com/bstaletic/ycmd/tree/python-error
msg379808 - (view) Author: Boris Staletic (bstaletic) Date: 2020-10-27 21:45
I can also reproduce the same problem with the ubuntu packaged python3, which is 3.8.5 on Ubuntu 20.04. The only problem is that, with a stripped library, you don't get line numbers in valgrind's output. Steps to repro:

1. apt install valgrind gcc python3-config
2. Save the same attached file from the first comment as test.c.
3. gcc $(python3-config --includes) $(python3-config --ldflags) -lpython3.8 -o python-error
4. PYTHONMALLOC=malloc valgrind ./python-error

Valgrind output:

==1200== Conditional jump or move depends on uninitialised value(s)
==1200==    at 0x4A7B37B: PyUnicode_Decode (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==1200==    by 0x109264: main (in /python-error)
==1200==
==1200== Conditional jump or move depends on uninitialised value(s)
==1200==    at 0x4A7AE57: PyUnicode_AsEncodedString (in /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0)
==1200==    by 0x109280: main (in /python-error)

I have not checked earlier versions of python.
msg379846 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-10-28 22:10
It points on strcmp(lower, "us_ascii") == 0.

Seems that the compiler optimizes calling strcmp() with compile-time constant "us_ascii" by reading and comparing first 8 bytes as single word. But if lower contains "latin1" it has only 7 bytes initialized, and the 8-th is not initialized. It does not affect the result, but valgrind complains. Looks like a bug in valgrind.
msg379849 - (view) Author: Boris Staletic (bstaletic) Date: 2020-10-29 00:26
Thanks for looking into this.

> Looks like a bug in valgrind.

That actually explains why I wasn't able to reproduce this problem on my local machine. Ubuntu 20.04 comes with valgrind 3.15.0, while my local machine has 3.16.1. Upgrading valgrind on Ubuntu 20.04 does fix the issue.

This is good enough for me and I guess this can be closed as "not a bug".
History
Date User Action Args
2022-04-11 14:59:37adminsetgithub: 86342
2020-10-29 05:51:27methanesetstatus: open -> closed
resolution: not a bug
stage: resolved
2020-10-29 00:26:54bstaleticsetmessages: + msg379849
2020-10-28 22:10:49serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg379846
2020-10-27 21:45:26bstaleticsetmessages: + msg379808
versions: + Python 3.8
2020-10-27 18:52:25bstaleticcreate