building python 2.7.15 on archlinux32 from source succeeds, but running the testsuite segfaults with:

test_writelines (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... /usr/bin/xvfb-run: line 181:  3617 Segmentation fault      (core dumped) DISPLAY=:$SERVERNUM XAUTHORITY=$AUTHFILE "$@" 2>&1

the executed commands are:

  cd Python-${pkgver}

  # Temporary workaround for FS#22322
  # See http://bugs.python.org/issue10835 for upstream report
  sed -i "/progname =/s/python/python${_pybasever}/" Python/pythonrun.c

  # Enable built-in SQLite module to load extensions (fix FS#22122)
  sed -i "/SQLITE_OMIT_LOAD_EXTENSION/d" setup.py

  # FS#23997
  sed -i -e "s|^#.* /usr/local/bin/python|#!/usr/bin/python2|" Lib/cgi.py

  sed -i "s/python2.3/python2/g" Lib/distutils/tests/test_build_scripts.py \
    Lib/distutils/tests/test_install_scripts.py

  # Ensure that we are using the system copy of various libraries (expat, zlib and libffi),
  # rather than copies shipped in the tarball
  rm -r Modules/expat
  rm -r Modules/zlib
  rm -r Modules/_ctypes/{darwin,libffi}*

  # clean up #!s
  find . -name '*.py' | \
    xargs sed -i "s|#[ ]*![ ]*/usr/bin/env python$|#!/usr/bin/env python2|"

  # Workaround asdl_c.py/makeopcodetargets.py errors after we touched the shebangs
  touch Include/Python-ast.h Python/Python-ast.c Python/opcode_targets.h

  # FS#48761
  # http://bugs.python.org/issue25750
  patch -Np1 -i ../descr_ref.patch

  export OPT="${CFLAGS}"
  ./configure --prefix=/usr \
              --enable-shared \
              --with-threads \
              --enable-optimizations \
              --with-lto \
              --enable-ipv6 \
              --enable-unicode=ucs4 \
              --with-system-expat \
              --with-system-ffi \
              --with-dbmliborder=gdbm:ndbm \
              --without-ensurepip

  make

  LD_LIBRARY_PATH="${srcdir}/Python-${pkgver}":${LD_LIBRARY_PATH} \
    xvfb-run "${srcdir}/Python-${pkgver}/python" -m test.regrtest -v -uall 2>&1 | tee log.all

I don't see how test_idle runs the /usr/bin/xvfb-run command. First, it would be interesting to see what is the argument of the xvfb-run command and check if it does crash alone.

Can you please try to trace system calls using the following command and then attached the created trace file to this issue?

strace -f -o trace ./python -m test test_idle

> test_writelines (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... /usr/bin/xvfb-run: line 181:  3617 Segmentation fault      (core dumped) DISPLAY=:$SERVERNUM XAUTHORITY=$AUTHFILE "$@" 2>&1

What are values of DISPLAY, SERVERNUM and AUTHFILE environment variables?

Note the last line, which shows that the testsuite itself is executed using xvfb-run.

The archlinux32 build script is based off the original PKGBUILD for archlinux: https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/python2

xvfb-run is only involved, because the testsuite depends on being run in a GUI environment in order to test GUI components...

Ah, I see. Can you try to get the Python traceback and/or C traceback using gdb?

I think we should first determine whether xvfb is required to get the segfault, and second, where it fails.

Does
<python2> -m test.regrtest -v -m test_io test_idle
fail and fail in the same place, the 11th and last test run in the module? If not, what about

xvfb-run <python2> -m ...?

In either case, inserting prints in 'test_writelines', which has multiple asserts, should determine where it fails.

Test.test_idle checks that tkinter can imported, which in turn imports _tkinter, which tries to connect to a windowing system. Running with xvfb satisfies this.  Test_idle finds and run the test modules in idlelib.idle_test.  Individual modules and classes somewhat redundantly import test.support and check "requires('gui')".  So IDLE gui tests, and, I  am fairly sure, the tkinter gui tests, also require -ugui or -uall to run.  

Test_io is not a gui test, so does not need or run "requires('gui')", and hence does not need -ugui.  On the other hand, it imports PyShell, which imports tkinter and _tkinter, and hence would need xvfb-run on a headless server, as opposed to a workstation with screen.  Erich, I assume you are running on the latter.

> Can you try to get the Python traceback and/or C traceback using gdb?
Sry, I can't - I don't know how to use gdb :-(
Is there some one-line command which makes gdb start xvfb-run and attach to it? Or do I need to start xvfb-run and separately attach gdb to it? (how do I do this?)

> ... and hence would need xvfb-run on a headless server, as opposed to a workstation with screen.  Erich, I assume you are running on the latter.

I'm running this on a headless box (packaging for archlinux32). Setting up a desktop on the test box would be some effort, but should be possible in principle.

In the original PKGBUILD which Eli posted, there is a comment about when the tests started failing - which suggests, the problem is connected to xvfb-run.

ok, some update:

1st: running the complete test suite with xvfb fails (as already reported):
> LD_LIBRARY_PATH="${srcdir}/Python-${pkgver}":${LD_LIBRARY_PATH} \
    xvfb-run "${srcdir}/Python-${pkgver}/python" -m test.regrtest -v -uall
...
0:02:14 load avg: 0.90 [174/403/1] test_idle
test_close (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_misc (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_read (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_readline (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_readlines (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_unsupported (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_close (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
test_misc (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
test_unsupported (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
test_write (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
test_writelines (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
/usr/bin/xvfb-run: line 181: 13491 Segmentation fault      (core dumped) DISPLAY=:$SERVERNUM XAUTHORITY=$AUTHFILE "$@" 2>&1


running w/o xvfb-run succeeds:
> LD_LIBRARY_PATH="${srcdir}/Python-${pkgver}":${LD_LIBRARY_PATH} ${srcdir}/Python-${pkgver}/python -m test.regrtest -v -m test_io test_idle
== CPython 2.7.15 (default, Jul 16 2018, 12:22:58) [GCC 8.1.1 20180531]
==   Linux-4.17.4-1.0-ARCH-i686-with-glibc2.1.3 little-endian
==   /home/vagrant/python2/src/Python-2.7.15/build/test_python_14208
== CPU count: 2
Run tests sequentially
0:00:00 load avg: 0.26 [1/1] test_idle
test_close (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_misc (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_read (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_readline (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_readlines (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_unsupported (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_close (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
test_misc (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
test_unsupported (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
test_write (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
test_writelines (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok

----------------------------------------------------------------------
Ran 11 tests in 0.003s

OK
1 test OK.

Total duration: 150 ms
Tests result: SUCCESS


running only these tests with xvfb-run _succeeds_:
> LD_LIBRARY_PATH="${srcdir}/Python-${pkgver}":${LD_LIBRARY_PATH} xvfb-run ${srcdir}/Python-${pkgver}/python -m test.regrtest -v -m test_io test_idle
== CPython 2.7.15 (default, Jul 16 2018, 12:22:58) [GCC 8.1.1 20180531]
==   Linux-4.17.4-1.0-ARCH-i686-with-glibc2.1.3 little-endian
==   /home/vagrant/python2/src/Python-2.7.15/build/test_python_14226
== CPU count: 2
Run tests sequentially
0:00:00 load avg: 0.15 [1/1] test_idle
test_close (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_misc (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_read (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_readline (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_readlines (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_unsupported (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_close (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
test_misc (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
test_unsupported (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
test_write (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
test_writelines (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok

----------------------------------------------------------------------
Ran 11 tests in 0.004s

OK
1 test OK.

Total duration: 144 ms
Tests result: SUCCESS

I got a similar error and managed to fix it by installing the noto-fonts package [1]. ttf-dejavu [2] works, too. I guess at least one font should be installed to allow those tests to run.

[1] https://www.archlinux.org/packages/extra/any/noto-fonts/
[2] https://www.archlinux.org/packages/extra/any/ttf-dejavu/

Here is partial backtrace from commit 1d26c72e6a9c5b28b27c158f2f196217707dbb0f:

#0  0x00007ffff6763197 in GetTkFontAttributes (faPtr=faPtr@entry=0x555555cd91e8, ftFont=<optimized out>)
    at /usr/src/debug/tk8.6.8/unix/../unix/tkUnixRFont.c:167
#1  0x00007ffff67639fa in InitFont (tkwin=tkwin@entry=0x555555ee94a0, pattern=pattern@entry=0x555555cfc690, fontPtr=0x555555cd91b0, 
    fontPtr@entry=0x0) at /usr/src/debug/tk8.6.8/unix/../unix/tkUnixRFont.c:313
#2  0x00007ffff6763c4e in TkpGetFontFromAttributes (tkFontPtr=tkFontPtr@entry=0x0, tkwin=tkwin@entry=0x555555ee94a0, 
    faPtr=0x555555d1e8b8) at /usr/src/debug/tk8.6.8/unix/../unix/tkUnixRFont.c:482
#3  0x00007ffff66b5ee2 in Tk_AllocFontFromObj (interp=0x555555c4b900, tkwin=tkwin@entry=0x555555ee94a0, objPtr=0x555555e39000)
    at /usr/src/debug/tk8.6.8/unix/../generic/tkFont.c:1158
#4  0x00007ffff66ac053 in DoObjConfig (interp=interp@entry=0x555555c4b900, recordPtr=recordPtr@entry=0x555555e53390 "0r\354UUU", 
    optionPtr=optionPtr@entry=0x555555f4e010, valuePtr=<optimized out>, valuePtr@entry=0x555555e39000, 
    tkwin=tkwin@entry=0x555555ee94a0, savedOptionPtr=savedOptionPtr@entry=0x0) at /usr/src/debug/tk8.6.8/unix/../generic/tkConfig.c:717
#5  0x00007ffff66ace92 in Tk_InitOptions (interp=interp@entry=0x555555c4b900, recordPtr=recordPtr@entry=0x555555e53390 "0r\354UUU", 
    optionTable=optionTable@entry=0x555555f4de10, tkwin=0x555555ee94a0) at /usr/src/debug/tk8.6.8/unix/../generic/tkConfig.c:496
#6  0x00007ffff67250b2 in CreateWidget (sharedPtr=0x555555ec7230, tkwin=<optimized out>, interp=0x555555c4b900, 
    parent=<optimized out>, objc=<optimized out>, objv=<optimized out>) at /usr/src/debug/tk8.6.8/unix/../generic/tkText.c:664
#7  0x00007ffff62efe16 in TclNRRunCallbacks () from /usr/lib/libtcl8.6.so
#8  0x00007ffff67eba0c in Tkapp_Call (selfptr=0x7ffff40242b0, args=<optimized out>)
    at /home/yen/Projects/cpython/Modules/_tkinter.c:1508

Looks more like a Tk issue than a CPython one.

For Arch Linux packagers: adding checkdepends=('ttf-font') to PKGBUILD can fix testing errors. Here's an example: https://github.com/archlinuxcn/repo/commit/e4de4598bccd44ba38c37854dc5edfb490dbe09a

Additional notes:

I am puzzled that test_idle output starts with test_io, as that is alphabetically in the middle of the files in idle_test/.

There are 2 independent 'test_io' modules: test/test_io, which tests the io module, and idlelib/test_idle/test_io, which tests idlelib.PyShell.PseudoFile and subclasses (which are all derived from io.TextIOBase).  Only the latter is an issue here.  The former is an irrelevant distraction and should not have been included with test_idle in msg321737.

Since log.all ends with "test_writelines (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... /usr/bin/xvfb-run: line 181: ...", I initially thought that test_writelines, in particular, stimulated the failure.  But in the msg321737 output, test_writelines ran 'OK'.

The fact that xvfv ... test_idle runs by itself but not when alphabetically preceding tests are run says that some preceding test alters something.   The failure of test_audioop seems an unlikely cause.  I suspect test__all__, by importing _tkinter either directly or via Tkinter or some IDLE module.  This could be tested with xvfb ... test___all__ test_idle.

Since the tests are mostly run alphabetically and the tkinter tests, test_tcl, test_tk, test_ttk_guionly, and test_ttk_textonly alphabetically follow test_idle, they should not be the cause of the crash during test_idle.  On the other hand, if the test above crashes, and so does 
xvbf ... -ugui test___all__ test_tcl test_tk test_ttk_guionly test_ttk_textonly
then test_idle would be eliminated as the direct culprit.

Sorry, after reading the original issue carefully, I realize that my issue is slightly different. In my case running test_idle only crashes, and it crashes before the first test in test_idle, not in test_writelines.

I copied the Arch Linux packaging script mentioned by Eli [2], modified it so that test_idle, test_tk and test_ttk_guionly are not skipped, and added a dependency ttf-font. With the modified PKGBUILD, the only failing test is test_ssl. I guess the issue has been fixed.

Erich Eckner, could you still reproduce the original crash?

[1] https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/python2

Yes, I get much further in the test suite, currently (40k lines logged vs. 23k). Now, it succeeds until:

0:08:26 load avg: 0.72 [358/403/3] test_tuple
test_addmul (test.test_tuple.TupleTest) ... ok
test_bigrepeat (test.test_tuple.TupleTest) ... /usr/bin/xvfb-run: line 181:  231
8 Segmentation fault      (core dumped) DISPLAY=:$SERVERNUM XAUTHORITY=$AUTHFILE
 "$@" 2>&1

The new crashing test seems the same as https://bugs.python.org/issue33153. I can't reproduce it on 64-bit Arch Linux, so that might be a 32-bit only issue.

It is not clear what you did differently to get a different result. I changed the title since it seems that idlelib code is not the direct problem.

yes, I'm also not sure what's different now - I executed the very same commands on the very same source tree :-/

2.7 is dead.