issue41069 introduces tests for paths/files containing non-ascii characters.

On AIX - since the merge of PR21035 and PR21156 - the bots have been broken, i.e., returning test failed.

commit 700cfa8c90a90016638bac13c4efd03786b2b2a0
Author: Serhiy Storchaka <storchaka@gmail.com>
Date:   Thu Jun 25 17:56:31 2020 +0300

    bpo-41069: Make TESTFN and the CWD for tests containing non-ascii characters. (GH-21035)

commit f925407a19eeb9bf5f7640143979638adce2c677
Author: Serhiy Storchaka <storchaka@gmail.com>
Date:   Thu Jun 25 20:39:12 2020 +0300

    [3.9] bpo-41069: Make TESTFN and the CWD for tests containing non-ascii characters. (GH-21035). (GH-21156)

    (cherry picked from commit 700cfa8c90a90016638bac13c4efd03786b2b2a0)

++++++++

Sadly, I cannot determine - exactly - where it is going wrong as the verbose results ends (says SUCCESS, but there is an ENV change, so bot says FAILED) as:

----------------------------------------------------------------------

Ran 614 tests in 59.122s

OK (skipped=8)
Warning -- files was modified by test_io
  Before: []
  After:  ['@test_23134518_tmpæ']
test_io failed (env changed) in 59.7 sec

== Tests result: SUCCESS ==

1 test altered the execution environment:
    test_io

Total duration: 59.8 sec

Thank you for your report (and for all other heroic work for maintaining AIX). Breaking only one test on AIX looks like a good news.

Do you able to run tests manually? Could you please run the following command?

    ./python -m test --list-cases -uall test_io | xargs -n 1 ./python -m test -vuall test_io -m

It may help to determine what exactly the test was broken.

Neat! extra arguments!!

The warnings - extracted:

== CPython 3.10.0a0 (heads/master-dirty:b1a8730, Jul 26 2020, 14:00:34) [GCC 7.2.0]
== AIX-2-00F9C1964C00-powerpc-32bit big-endian
== cwd: /home/aixtools/cpython/cpython-master/build/test_python_27984450▒
== CPU count: 32
== encodings: locale=ISO8859-1, FS=iso8859-1
0:00:00 Run tests sequentially
0:00:00 [1/1] test_io
test_fspath_support (test.test_io.CIOTest) ... ok

----------------------------------------------------------------------

Ran 1 test in 0.026s

OK
Warning -- files was modified by test_io
  Before: []
  After:  ['@test_27984450_tmpæ']
test_io failed (env changed)

+++++
== CPython 3.10.0a0 (heads/master-dirty:b1a8730, Jul 26 2020, 14:00:34) [GCC 7.2.0]
== AIX-2-00F9C1964C00-powerpc-32bit big-endian
== cwd: /home/aixtools/cpython/cpython-master/build/test_python_28246552▒
== CPU count: 32
== encodings: locale=ISO8859-1, FS=iso8859-1
0:00:00 Run tests sequentially
0:00:00 [1/1] test_io
test_fspath_support (test.test_io.PyIOTest) ... ok

----------------------------------------------------------------------

Ran 1 test in 0.026s

OK
Warning -- files was modified by test_io
  Before: []
  After:  ['@test_28246552_tmpæ']
test_io failed (env changed)

== Tests result: SUCCESS ==

1 test altered the execution environment:
    test_io

++++

== CPython 3.10.0a0 (heads/master-dirty:b1a8730, Jul 26 2020, 14:00:34) [GCC 7.2.0]
== AIX-2-00F9C1964C00-powerpc-32bit big-endian
== cwd: /home/aixtools/cpython/cpython-master/build/test_python_33620404▒
== CPU count: 32
== encodings: locale=ISO8859-1, FS=iso8859-1
0:00:00 Run tests sequentially
0:00:00 [1/1] test_io
test_reader_writer_close_error_on_close (test.test_io.CBufferedRWPairTest) ... ok

----------------------------------------------------------------------

Ran 1 test in 0.001s

OK
Warning -- Unraisable exception
Exception ignored in: <_io.BufferedRWPair object at 0x30689750>
Traceback (most recent call last):
  File "/home/aixtools/cpython/cpython-master/Lib/test/support/__init__.py", line 613, in gc_collect
    gc.collect()
ValueError: flush of closed file
Warning -- Unraisable exception
Exception ignored in: <_io.BufferedWriter>
Traceback (most recent call last):
  File "/home/aixtools/cpython/cpython-master/Lib/test/support/__init__.py", line 613, in gc_collect
    gc.collect()
ValueError: flush of closed file
test_io failed (env changed)

== Tests result: SUCCESS ==

1 test altered the execution environment:
    test_io

Hope this helps!

There are two errors in test_io, both are not related to AIX.

The first one is in test_fspath_support. It is exposed on non-UTF-8 locales. AIX by accident use ISO8859-1 locale by default.

The second one is in test_reader_writer_close_error_on_close and related to garbage collector. Seems it is only exposed when run this test separately.

Excellent!!

aixtools@gcc119:[/home/aixtools/cpython/cpython-master]git pr 21640
remote: Enumerating objects: 50, done.
remote: Counting objects: 100% (50/50), done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 58 (delta 46), reused 48 (delta 46), pack-reused 8
Unpacking objects: 100% (58/58), done.
From https://github.com/python/cpython
 * [new ref]         refs/pull/21640/head -> pr_21640

aixtools@gcc119:[/home/aixtools/cpython/cpython-master]./python -m test
test_io
0:00:00 Run tests sequentially
0:00:00 [1/1] test_io
test_io passed in 1 min

== Tests result: SUCCESS ==

1 test OK.

Total duration: 1 min
Tests result: SUCCESS

On 27/07/2020 16:16, Serhiy Storchaka wrote:
> Change by Serhiy Storchaka <storchaka+cpython@gmail.com>:
>
>
> ----------
> keywords: +patch
> pull_requests: +20781
> stage:  -> patch review
> pull_request: https://github.com/python/cpython/pull/21640
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue41401>
> _______________________________________
>

New changeset 67987acd5dc9776f55f4e139e2b3d9e7a6434d9f by Serhiy Storchaka in branch 'master':
bpo-41401: Fix test_fspath_support in test_io. (GH-21640)
https://github.com/python/cpython/commit/67987acd5dc9776f55f4e139e2b3d9e7a6434d9f

Automatic backport does not work due to changes in the test.support module. Victor, do you mind to backport PR 21640 to 3.9 and 3.8?

The 'master' branch bot is working again, the 3.9 branch is still broken, and the 3.8 branch seems, as yet, unaffected.

As much as I wish I had the skills to do the cherry picking - I am not going to touch this.

The AIX bots for 3.9 branch continue to report broken for test_io (ENV change) - as they still wait for the backport for that branch!

> Automatic backport does not work due to changes in the test.support module. Victor, do you mind to backport PR 21640 to 3.9 and 3.8?

Why don't you want to do backport yourself? It's a trivial change, it only changes a single line.

Anyway, I created PR 22133.

New changeset c73ee5acc96b4bbd6885156883b224b8cc3e470c by Victor Stinner in branch '3.9':
bpo-41401: Fix test_fspath_support in test_io. (GH-21640) (GH-22133)
https://github.com/python/cpython/commit/c73ee5acc96b4bbd6885156883b224b8cc3e470c

New changeset 84105cbaa395923e3584a87d67ccce72c8420bb4 by Victor Stinner in branch '3.8':
bpo-41401: Fix test_fspath_support in test_io. (GH-21640) (GH-22133) (GH-22135)
https://github.com/python/cpython/commit/84105cbaa395923e3584a87d67ccce72c8420bb4

It should now be fixed. Thanks.