Title: Compiling against master branch fails; error: expected expression SET_DECODE_ERROR
Type: compile error Stage: resolved
Components: Versions: Python 3.7
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: CuriousLearner, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2017-12-02 12:50 by CuriousLearner, last changed 2017-12-04 13:43 by CuriousLearner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 4679 closed serhiy.storchaka, 2017-12-02 18:18
PR 4681 merged serhiy.storchaka, 2017-12-02 18:50
PR 4694 merged vstinner, 2017-12-04 09:40
Messages (9)
msg307427 - (view) Author: Sanyam Khurana (CuriousLearner) * (Python triager) Date: 2017-12-02 12:50
The current master branch at commit af5a895073c24637c094772b27526b94a12ec897

fails while building the interpreter.

The following is the traceback while running `make` after `./configure --with-pydebug`

gcc -c -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes    -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration   -I. -I./Include    -DPy_BUILD_CORE -o Modules/main.o Modules/main.c
Modules/main.c:904:20: error: expected expression
            return SET_DECODE_ERROR("PYTHONEXECUTABLE environment "
Modules/main.c:46:5: note: expanded from macro 'SET_DECODE_ERROR'
    do { \
1 error generated.
make: *** [Modules/main.o] Error 1
msg307429 - (view) Author: Sanyam Khurana (CuriousLearner) * (Python triager) Date: 2017-12-02 13:14
I used git bisect to track down the commit that broke the build.

Seems like this is the SHA: ebac19dad6263141d5db0a2c923efe049dba99d2

ebac19dad6263141d5db0a2c923efe049dba99d2 is the first bad commit
commit ebac19dad6263141d5db0a2c923efe049dba99d2
Author: Victor Stinner <>
Date:   Fri Dec 1 20:09:52 2017 +0100

    bpo-32030: Don't call _PyPathConfig_Fini() in Py_FinalizeEx() (#4667)


    * _PyPathConfig_Fini() cannot be called in Py_FinalizeEx().
      Py_Initialize() and Py_Finalize() can be called multiple times, but
      it must not "forget" parameters set by Py_SetProgramName(),
      Py_SetPath() or Py_SetPythonHome(), whereas _PyPathConfig_Fini()
      clear all these parameters.
    * config_get_program_name() and calculate_program_full_path() now
      also decode paths using Py_DecodeLocale() to use the
      surrogateescape error handler, rather than decoding using
      mbstowcs() which is strict.
    * Change _Py_CheckPython3() prototype: () => (void)
    * Truncate a few lines which were too long

:040000 040000 de4ec929ff3fcd3c6455d8dae4d16e47fdd32ad5 8b289341f1b5e68757ac3b111a7f11808f5de1ec M	Include
:040000 040000 ecddc93b37f74818d85f6d8e8ab0e0c57e893c03 c51971a3af2b6f163e625505265277410bb9eedd M	Modules
:040000 040000 b92d30ac6375b7dde865a878e701684dd400e742 7d5d8810407060f32c9cbce20dcfc6f9d425524f M	PC
:040000 040000 537f9e0e6a27b41178fe7b828a8da99c63cc2d9b b14940c66180023a3704ad3949367db3e82ca832 M	Python
msg307443 - (view) Author: Sanyam Khurana (CuriousLearner) * (Python triager) Date: 2017-12-02 17:49
I tried to resolve the issue by trying to figure out the syntax for the SET_DECODE_ERROR macro.

I tried to call a function from the macro, and eventually do the checking in that function.

Second way I tried was to replace statement for macro instead of using do-while for placing the if-else clause.

I was stuck at the issue that at the point when the compiler tries to replace the macro with actual code, `pymain` wasn't defined.

If someone could provide a pointer on this, it would be really helpful and I'll try to work on a patch.

One simplest possible solution would be not use macro and simply use the code at the 3 places where it is being used. But I'll defer this anti-DRY approach if we can have a better solution.
msg307444 - (view) Author: Sanyam Khurana (CuriousLearner) * (Python triager) Date: 2017-12-02 17:58
Even a simple use of ternary operator instead of using if-else inside do-while to handle the macro like:

    pymain->err = ((LEN) == (size_t)-2) \
    ? _Py_INIT_USER_ERR("cannot decode " #NAME) \
    : _Py_INIT_NO_MEMORY()

, throws the error `undeclared identifier pymain`:

gcc -c -Wno-unused-result -Wsign-compare -g -O0 -Wall -Wstrict-prototypes    -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration   -I. -I./Include    -DPy_BUILD_CORE -o Modules/main.o Modules/main.c
Modules/main.c:899:20: error: use of undeclared identifier 'pymain'
            return SET_DECODE_ERROR("PYTHONEXECUTABLE environment "
Modules/main.c:46:5: note: expanded from macro 'SET_DECODE_ERROR'
    pymain->err = ((LEN) == (size_t)-2) \
1 error generated.
make: *** [Modules/main.o] Error 1

I'm still trying to figure out a solution for this.
msg307452 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-12-02 19:36
New changeset 13badcbc60cdbfae1dba1683fd2fae9d70717143 by Serhiy Storchaka in branch 'master':
bpo-32197: Try to fix a compiler error on OS X introduced in bpo-32030. (#4681)
msg307559 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-12-04 12:39
New changeset 31a8393cf6a74c870c3484dd68500619f6232c6d by Victor Stinner in branch 'master':
Revert "bpo-32197: Try to fix a compiler error on OS X introduced in bpo-32030. (#4681)" (#4694)
msg307562 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-12-04 13:06
Oh sorry, I missed the macOS failure and I wasn't available last week-end for quickly fix it. So thanks Serhiy for the revert ;-) I reapplied my changes with the following fix. test_embed randomly failed, but it's unrelated (I was able to reproduce the bug on master, with the revert):

The fix:

diff --git a/Modules/main.c b/Modules/main.c
index 84706e1e290..4095259b88c 100644
--- a/Modules/main.c
+++ b/Modules/main.c
@@ -901,8 +901,8 @@ config_get_program_name(_PyMainInterpreterConfig *config)
         size_t len;
         wchar_t* program_name = Py_DecodeLocale(p, &len);
         if (program_name == NULL) {
-            return SET_DECODE_ERROR("PYTHONEXECUTABLE environment "
-                                    "variable", len);
+            return DECODE_LOCALE_ERR("PYTHONEXECUTABLE environment "
+                                     "variable", (Py_ssize_t)len);
         config->program_name = program_name;
@@ -916,8 +916,8 @@ config_get_program_name(_PyMainInterpreterConfig *config)
             size_t len;
             wchar_t* program_name = Py_DecodeLocale(pyvenv_launcher, &len);
             if (program_name == NULL) {
-                return SET_DECODE_ERROR("__PYVENV_LAUNCHER__ environment "
-                                        "variable", len);
+                return DECODE_LOCALE_ERR("__PYVENV_LAUNCHER__ environment "
+                                         "variable", (Py_ssize_t)len);
             config->program_name = program_name;
msg307563 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-12-04 13:06
I tested manually that compilation and tests succeed on macOS.
msg307566 - (view) Author: Sanyam Khurana (CuriousLearner) * (Python triager) Date: 2017-12-04 13:43
Hey Victor, thanks for the update.

I was confused about how the macro was replaced in the previous version.
Date User Action Args
2017-12-04 13:43:34CuriousLearnersetmessages: + msg307566
2017-12-04 13:06:28vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg307563

stage: patch review -> resolved
2017-12-04 13:06:01vstinnersetmessages: + msg307562
2017-12-04 12:39:17vstinnersetmessages: + msg307559
2017-12-04 09:40:10vstinnersetpull_requests: + pull_request4606
2017-12-02 19:36:02serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg307452
2017-12-02 18:50:57serhiy.storchakasetpull_requests: + pull_request4592
2017-12-02 18:18:43serhiy.storchakasetkeywords: + patch
stage: patch review
pull_requests: + pull_request4590
2017-12-02 17:58:53CuriousLearnersetmessages: + msg307444
2017-12-02 17:49:42CuriousLearnersetmessages: + msg307443
2017-12-02 13:14:17CuriousLearnersetmessages: + msg307429
2017-12-02 12:55:30serhiy.storchakasetnosy: + vstinner
type: crash -> compile error
2017-12-02 12:50:00CuriousLearnercreate