classification
Title: test_loading.py - false positive result for "def test_find" when find_library() is not functional or the (shared) library does not exist
Type: behavior Stage: patch review
Components: ctypes, Tests Versions: Python 3.9, Python 3.8, Python 3.7, Python 3.6, Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Michael.Felt, aixtools@gmail.com, martin.panter, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2016-09-26 11:04 by Michael.Felt, last changed 2021-03-31 16:44 by Michael.Felt.

Pull Requests
URL Status Linked Edit
PR 18312 open Michael.Felt, 2020-02-02 12:23
Messages (11)
msg277411 - (view) Author: Michael Felt (Michael.Felt) * Date: 2016-09-26 11:04
in Lib/ctypes/test/test_loading.py there is the following test

    def test_find(self):
        for name in ("c", "m"):
            lib = find_library(name)
            if lib:
                cdll.LoadLibrary(lib)
                CDLL(lib)


1. on AIX, the test is "broken" because lib is None for both names because:
a1) find_library is basically broken for AIX (needs ldconfig, gcc, and/or other tools not generally available on AIX (production) servers
a2) "m" will always fail because there is no equivalent to libm.so - AIX libm.a only has static members - none are shared.
2. IMHO - the test is misnamed "test_find" - as None is an acceptable answer - which is why it 'appears' to PASS on AIX

As far as library names to look for I suggest switching "m" to "ssl" - as something that should be everywhere.

And the test should be something more like:

    self.assertEqual(lib not None)

OR is the test actually testing LoadLibrary and CDLL (as well)

Back to my humble opinion: assuming the real test is to test LoadLibrary and CDLL by accepting None as a valid result from find_library - false POSITIVE is the result when find_library() is not working.
msg277415 - (view) Author: Michael Felt (Michael.Felt) * Date: 2016-09-26 12:45
The assert to be added is much more simple:

i.e., self.assertTrue(lib)

And then you get something like:

root@x064:[/data/prj/python/python-3.6.0.177/Lib/ctypes/test]../../../python -m unittest test_loading.py
libc_name is libc.a(shr.o)
sslib is None
F.sss
======================================================================
FAIL: test_find (test_loading.LoaderTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/prj/python/python-3.6.0.177/Lib/ctypes/test/test_loading.py", line 55, in test_find
    self.assertTrue(lib)
AssertionError: None is not true

----------------------------------------------------------------------
Ran 7 tests in 0.126s

FAILED (failures=1, skipped=5)
msg277480 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-09-27 02:02
The purpose of the test seems to be to check that finding and loading works for widely-available libraries. However I suspect find_library("c") can fail on other platforms as well. Presumably that is why there is the special case for Cygwin at the top of the file.

Open SSL is also widely available, but not universal. I think it is even possible to build Python without SSL support, and the tests should not fail in this case.

Perhaps it would be okay to remove "m" and just run the test on find_library("c"). If so, we can adjust the test to call self.skipTest("C library not found"), which will warn that the test could not be run.
msg277543 - (view) Author: Michael Felt (Michael.Felt) * Date: 2016-09-27 19:17
Actually, what may be needed are more "@skip" blocks - as ctypes has always been platform dependent.

OR - have a counter or boolean that starts as zero or false and increase each time find_library() returns a value - and the test fails if the counter is still zero, or boolean is still false.

Further, perhaps a separate test_load() that is platform specific - with the correct - read expected - result from find_library(). I say expected because python has always been testing for libc.so.6 while I see in cloud-init that that are calls hard-coded to CDLL("libc.so.7") - but maybe that is something specific for Canonical aka ubuntu.

However, it seems that expecting libc.so.6 to ALWAYS be present is not safe.

FYI:

michael@x071:[/data/prj/python/cloud-init]grep CDLL cloud-init-0*/cloudinit/*.py
cloud-init-0.7.5/cloudinit/util.py:            libc = ctypes.CDLL('/lib/libc.so.7')
cloud-init-0.7.8/cloudinit/util.py:            libc = ctypes.CDLL('/lib/libc.so.7')
msg277609 - (view) Author: Michael Felt (Michael.Felt) * Date: 2016-09-28 12:39
Is this suitable?

    def test_find(self):
        # to track that at least one call to find_library() found something
        found = false
        for name in ("c", "m"):
            lib = find_library(name)
            if lib:
                found = true
                cdll.LoadLibrary(lib)
                CDLL(lib)
        # test is FAILED if nothing was found
        self.assertTrue(found)
msg277610 - (view) Author: Michael Felt (Michael.Felt) * Date: 2016-09-28 12:42
ok - false needs to be False and true needs to be True

seemed so simple. Sigh.
msg277793 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-10-01 03:57
Other tests in this file skip the test if libc_name is None. So I think it would make more sense to skip the test rather than fail in test_find(). I.e.

if not found:
    self.skipTest("Could not find c and m libraries")

If you are confident that find_library() will always find libc on AIX, perhaps you can suggest an extra test (or add to an existing test), to first check for the AIX platform, and only then fail if find_library() returned None.
msg278091 - (view) Author: Michael Felt (aixtools@gmail.com) Date: 2016-10-04 20:54
On 01-Oct-16 05:57, Martin Panter wrote:
> Martin Panter added the comment:
>
> Other tests in this file skip the test if libc_name is None. So I think it would make more sense to skip the test rather than fail in test_find(). I.e.
>
> if not found:
>      self.skipTest("Could not find c and m libraries")
>
> If you are confident that find_library() will always find libc on AIX, perhaps you can suggest an extra test (or add to an existing test), to first check for the AIX platform, and only then fail if find_library() returned None.
Yes, I am confident - it is part of the core run-time environment:
  # lslpp -w | grep libc.a
   /usr/ccs/lib/libc.a                         bos.rte.libc File
   /usr/lib/libc.a                             bos.rte.libc Symlink
   /usr/lib/threads/libc.a                     bos.rte.libc Symlink

And, bos.rte.libc cannot be uninstalled:

   SELECTED FILESETS:  The following is a list of filesets that you asked to
   remove.  They cannot be removed until all of their dependent filesets
   are also removed.  See subsequent lists for details of dependents.

     bos.rte.libc 5.3.7.0                      # Base Level Fileset

   NON-DEINSTALLABLE DEPENDENTS:  The following filesets depend upon one or
   more of the selected filesets listed above.  These dependents should
   be removed before the selected filesets.  Deinstallability checks 
indicate
   that these dependents should not be removed from the system; 
therefore, the
   selected filesets cannot be removed.

     bos.mp64 5.3.7.0                          # Base Operating System 
64-bit...
     bos.rte 5.3.7.0                           # Base Level Fileset
     devices.chrp.IBM.lhea.rte 5.3.7.0         # Host Ethernet Adapter 
(HEA) ...
     devices.chrp.base.rte 5.3.7.0             # RISC PC Base System 
Device S...
     devices.chrp.vdevice.rte 5.3.7.0          # Virtual I/O Bus Support
     devices.vdevice.IBM.v-scsi.rte 5.3.7.0    # Virtual SCSI Client Support
     ifor_ls.base.cli 5.3.7.0                  # License Use Management 
Runti...

I gave up counting the number of dependents that "officially" are 
dependents on bos.rte.libc after the count went above 200 - on a vanilla 
system install.

I'll make a different test (skip if not aix) that find_library("c") must 
succeed. The load has already been tested - with libc_name already 
hard-coded.

>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue28276>
> _______________________________________
msg361228 - (view) Author: Michael Felt (Michael.Felt) * Date: 2020-02-02 11:05
This is something from long long ago - time to get it completed.

The (remaining) issue is: "c" and "m" may not be shared libraries - so nothing is ever found and the test is "skipped" but reports itself as PASSED.

The original issue (fixed for AIX in Python3.7) would be if find_library itself is broken and always returns NULL/None.

Again, the tests says "PASSED" when actually it was skipped.

Following this - Martin's suggestion seems a viable solution.
msg389854 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-03-30 21:20
Concur. Also you can add "ssl" to the list of tested names if both "c" and "m" fail on AIX.
msg389918 - (view) Author: Michael Felt (Michael.Felt) * Date: 2021-03-31 16:44
Sure. Probably have to rebase first.
History
Date User Action Args
2021-03-31 16:44:25Michael.Feltsetmessages: + msg389918
2021-03-30 21:20:55serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg389854
2020-02-02 12:24:19Michael.Feltsetversions: + Python 3.8, Python 3.9, - Python 2.7
2020-02-02 12:23:50Michael.Feltsetkeywords: + patch
stage: patch review
pull_requests: + pull_request17688
2020-02-02 11:05:06Michael.Feltsettype: behavior
messages: + msg361228
2016-10-04 20:54:22aixtools@gmail.comsetnosy: + aixtools@gmail.com
messages: + msg278091
2016-10-01 03:57:15martin.pantersetmessages: + msg277793
2016-09-28 12:42:10Michael.Feltsetmessages: + msg277610
2016-09-28 12:39:39Michael.Feltsetmessages: + msg277609
2016-09-27 19:17:34Michael.Feltsetmessages: + msg277543
2016-09-27 02:02:53martin.pantersetversions: - Python 3.3, Python 3.4
nosy: + martin.panter

messages: + msg277480

components: + Tests
2016-09-26 12:45:14Michael.Feltsetmessages: + msg277415
2016-09-26 11:04:57Michael.Feltsetcomponents: + ctypes
2016-09-26 11:04:43Michael.Feltcreate