classification
Title: test_os: test_copy_file_range_offset fails on FreeBSD CURRENT
Type: Stage: resolved
Components: Versions: Python 3.10, Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: koobs, lukasz.langa, pablogsal, vstinner
Priority: normal Keywords:

Created on 2021-02-15 19:26 by pablogsal, last changed 2021-03-08 19:36 by vstinner. This issue is now closed.

Messages (11)
msg387046 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-02-15 19:26
======================================================================
FAIL: test_copy_file_range_offset (test.test_os.FileTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/test_os.py", line 365, in test_copy_file_range_offset
    self.assertIn(i, range(0, bytes_to_copy+1));
AssertionError: 7 not found in range(0, 7)
----------------------------------------------------------------------

https://buildbot.python.org/all/#/builders/151/builds/300/steps/5/logs/stdio
msg387047 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-02-15 19:26
koobs, can you give us access to the buildbot? The latest IP you provided do not work anymore
msg387065 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-02-15 22:20
> https://buildbot.python.org/all/#/builders/151/builds/300/steps/5/logs/stdio

This is the CURRENT version of FreeBSD (latest kernel and latest libc). test.pythoninfo says:

os.uname: posix.uname_result(sysname='FreeBSD', nodename='140-CURRENT-amd64', release='14.0-CURRENT', version='FreeBSD 14.0-CURRENT #13 main-n244626-cb7cc72c546: Fri Feb  5 02:20:29 UTC 2021     root@130-CURRENT-amd64:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG', machine='amd64')

On FreeBSD 12.2 (stable), test_os pass because os.copy_file_range() is missing:

$ grep copy_file_range pyconfig.h -A1
/* Define to 1 if you have the `copy_file_range' function. */
/* #undef HAVE_COPY_FILE_RANGE */

$ ./python -m test test_os -v -m test_copy_file_range_offset
== CPython 3.9.1+ (heads/3.9:c9f07813ab, Feb 15 2021, 22:00:04) [Clang 10.0.1 (git@github.com:llvm/llvm-project.git llvmorg-10.0.1-0-gef32c611a
== FreeBSD-12.2-RELEASE-p3-amd64-64bit-ELF little-endian
== cwd: /usr/home/vstinner/python/3.9/build/test_python_5324æ
== CPU count: 8
== encodings: locale=UTF-8, FS=utf-8
0:00:00 load avg: 11.77 Run tests sequentially
0:00:00 load avg: 11.77 [1/1] test_os
test_copy_file_range_offset (test.test_os.FileTests) ... skipped 'test needs os.copy_file_range()'

----------------------------------------------------------------------

Ran 1 test in 0.002s

OK (skipped=1)

== Tests result: SUCCESS ==

1 test OK.

Total duration: 674 ms
Tests result: SUCCESS
msg387066 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-02-15 22:23
Hummmm. The last time it succeeded was with platform.platform: FreeBSD-13.0-CURRENT-amd64-64bit-ELF:

https://buildbot.python.org/all/#/builders/151/builds/280
msg387067 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-02-15 22:25
This seems a problem in FreeBSD 14.0 CURRENT. Koobs, could you report this upstream?
msg387128 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-02-16 20:55
AMD64 FreeBSD Shared 3.x is also affected:
https://buildbot.python.org/all/#/builders/483/builds/813

FAIL: test_copy_file_range_offset (test.test_os.FileTests)
msg387757 - (view) Author: Kubilay Kocak (koobs) (Python triager) Date: 2021-02-27 02:05
I'll see if I can isolate
msg387890 - (view) Author: Kubilay Kocak (koobs) (Python triager) Date: 2021-03-02 01:04
Fixed upstream in https://cgit.freebsd.org/src/commit/?id=a5f9fe2bab789f49e8b53da3a62dbd34725e23ea via https://reviews.freebsd.org/D28981

I'll update the worker to past this revision and restart the worker
msg387900 - (view) Author: Kubilay Kocak (koobs) (Python triager) Date: 2021-03-02 07:00
Updated worker to:

FreeBSD 14.0-CURRENT #14 main-n245176-8742817ba62: Tue Mar  2 03:48:10 UTC 2021

Next builds should pass (all else being equal)
msg387928 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-03-02 13:40
Buildbot is green! 
https://buildbot.python.org/all/#/builders/483/builds/841
msg388290 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-03-08 19:36
It was a FreeBSD kernel bug. Kubilay Kocak reported it to Rick Macklem who fixed the FreeBSD kernel:
https://github.com/freebsd/freebsd-src/commit/a5f9fe2bab789f49e8b53da3a62dbd34725e23ea
History
Date User Action Args
2021-03-08 19:36:37vstinnersetmessages: + msg388290
2021-03-02 13:40:49pablogsalsetstatus: open -> closed
resolution: fixed
messages: + msg387928

stage: resolved
2021-03-02 07:00:25koobssetmessages: + msg387900
2021-03-02 01:04:59koobssetmessages: + msg387890
2021-02-27 02:05:04koobssetmessages: + msg387757
2021-02-16 20:55:13vstinnersettitle: test_copy_file_range_offset fails on AMD64 FreeBSD Shared 3.9 -> test_os: test_copy_file_range_offset fails on FreeBSD CURRENT
messages: + msg387128
versions: + Python 3.10
2021-02-15 22:25:54pablogsalsetmessages: + msg387067
2021-02-15 22:23:29pablogsalsetmessages: + msg387066
2021-02-15 22:20:41vstinnersetmessages: + msg387065
2021-02-15 19:26:33pablogsalsetmessages: + msg387047
2021-02-15 19:26:10pablogsalcreate