classification
Title: uuid.getnode() MAC address on AIX
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.5, Python 3.4, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: BreamoreBoy, David.Edelsohn, aivarsk, haypo, koobs, neologix, python-dev, serhiy.storchaka, xnox
Priority: normal Keywords: patch

Created on 2013-02-25 07:29 by aivarsk, last changed 2015-01-05 17:55 by xnox. This issue is now closed.

Files
File name Uploaded Description Edit
aix_mac.patch aivarsk, 2013-02-25 07:29 review
uuid_netstat_getnode.patch serhiy.storchaka, 2014-10-01 19:33 review
uuid_netstat_getnode-3.5.patch serhiy.storchaka, 2014-10-22 19:22 review
uuid_netstat_getnode-3.5_2.patch serhiy.storchaka, 2014-10-23 10:32 review
uuid_netstat_getnode-3.4.patch serhiy.storchaka, 2014-10-23 10:32 review
koobs-freebsd10-python27-build758.log koobs, 2014-11-14 02:01
uuid_arp_getnode_freebsd.patch serhiy.storchaka, 2014-11-14 19:40 review
Messages (27)
msg182925 - (view) Author: Aivars Kalvāns (aivarsk) Date: 2013-02-25 07:29
uuid.getnode() on AIX returned random integer. This patch finds MAC in output of `netstat -ia`.
Tested on AIX 5.2
msg223390 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-17 23:46
@Aivars sorry about the delay in getting back to you.
msg223693 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2014-07-22 20:35
Thanks for the patch, but I'm not even sure AIX is an officially supported platform, so I'm not sure what to do with this patch.
msg223705 - (view) Author: David Edelsohn (David.Edelsohn) * Date: 2014-07-22 22:26
Huh?  What does officially supported platform mean? CPython builds and runs on AIX.
msg223724 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2014-07-23 05:37
> Huh?  What does officially supported platform mean? CPython builds and runs on AIX.

It means a platform for which we have regular contributors committed
to support the port, and ideallly with a stable buildbot.

We apparently have an unstable buildbot with mainy tests failing (and
last time I checked it didn't even build IIRC))
http://buildbot.python.org/all/builders/PPC64%20AIX%203.x/

So it would be nice to have the full regtest pass (with skips if applicable).
msg223738 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-23 13:43
David is listed against AIX on the experts list https://docs.python.org/devguide/experts.html.  That alone suggests to me that AIX is an officially supported platform.
msg228108 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-10-01 19:33
Thank you Aivars for your patch. I have verified and confirm that this method works on AIX and True64 UNIX (it should also work on IRIX, but I can't login in Snakebite's i6).

Here is modified patch. MAC address is now searched only in column with the "Address" header. Added try/except around converting to int for the case if candidate word contains non-heximal digits. Synchronized _netstat_getnode() with current code of _find_mac and extracted common code in separate function. Split _ifconfig_getnode() into separate functions which use different commands: ifconfig, arp and lanscan.
msg229151 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-10-12 14:36
If there are no objections, I'll commit the patch soon.
msg229153 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2014-10-12 14:39
My only comment would be to use subprocess instead of os.popen().
msg229154 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-10-12 14:44
This is different issue and can be applied only to 3.5.
msg229160 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2014-10-12 15:07
Why is that a different issue?
The code you *add in this patch* uses os.popen, why not use subprocess instead?

Furthermore, the code catches OSError when calling popen(), but
popen() doesn't raise an exception.
msg229162 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-10-12 15:10
FYI os.popen() now calls subprocess... So it's safe to call directly subprocess.
msg229163 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-10-12 15:13
+    cmd = 'LC_ALL=C %s %s 2>/dev/null' % (executable, '-ia')

It's safer to use the subprocess module instead of using a shell (see the recent Shellshock story) to change the environment variables and to redirect stderr. subprocess now has a convinient subprocess.DEVNULL. So it's something like:

env = os.environ.copy()
env['LC_ALL'] = 'C'
process = subprocess.Popen([executable, '-ia'], stderr=subprocess.DEVNULL)
...
msg229172 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-10-12 16:17
> The code you *add in this patch* uses os.popen, why not use subprocess instead?

Added code is just modified copy of existing code.

> Furthermore, the code catches OSError when calling popen(), but
popen() doesn't raise an exception.

It can raise an exception in rare cases. The manual of pipe mentions this. The code of posix_popen() contains error handling.

> FYI os.popen() now calls subprocess...

Not in 2.7.

> It's safer to use the subprocess module instead of using a shell (see the recent Shellshock story) to change the environment variables and to redirect stderr.

May be, but this is different issue, which is not related to this issue. If os.popen should be replaced with subprocess, it should be done even without applying the patch of this issue. Please open new issue.

However the recent Shellshock story affects only users of platforms which use bash or zsh as system shell (are there any?).

> subprocess now has a convinient subprocess.DEVNULL.

Not in 2.7.
msg229831 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-10-22 18:57
Here is a patch for 3.5 which uses subprocess.Popen() (subprocess.Popen() used in _find_mac() since issue22637).
msg229832 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-10-22 19:04
> Here is a patch for 3.5 which uses subprocess.Popen() (subprocess.Popen() used in _find_mac() since issue22637).

You probably forgot to attach the patch...
msg229833 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-10-22 19:22
> You probably forgot to attach the patch...

Indeed. :-(
msg229866 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-10-23 10:32
Here are updated patches for 3.5 (using subprocess) and 3.4 (using os.popen) 
which addresses Victor's comments.
msg230784 - (view) Author: Roundup Robot (python-dev) Date: 2014-11-07 10:27
New changeset e80cb046e764 by Serhiy Storchaka in branch '2.7':
Issue #17293: uuid.getnode() now determines MAC address on AIX using netstat.
https://hg.python.org/cpython/rev/e80cb046e764

New changeset ba4b31ed2952 by Serhiy Storchaka in branch '3.4':
Issue #17293: uuid.getnode() now determines MAC address on AIX using netstat.
https://hg.python.org/cpython/rev/ba4b31ed2952

New changeset 3e4f3cc4f1f9 by Serhiy Storchaka in branch 'default':
Issue #17293: uuid.getnode() now determines MAC address on AIX using netstat.
https://hg.python.org/cpython/rev/3e4f3cc4f1f9
msg230817 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-11-07 16:58
Looks as this hasn't broke buildbots.

Thank you Aivars for your patch. Thank you Natali and Victor for your suggestions and reviews.
msg231148 - (view) Author: Kubilay Kocak (koobs) Date: 2014-11-14 02:01
koobs-freebsd10 buildbot broken on all branches since:

2.7: e80cb046e7641fb8a71dda8254d2e619cdd64480
3.4: ba4b31ed2952b65ca447f57fbd6d540ebc4b749c
3.x: 3e4f3cc4f1f9dbee8e0ed5df47f77baae2ad310c

Full (2.7) log attached.

======================================================================
ERROR: test_arp_getnode (test.test_uuid.TestUUID)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/2.7.koobs-freebsd10/build/Lib/test/test_uuid.py", line 312, in test_arp_getnode
    node = uuid._arp_getnode()
  File "/usr/home/buildbot/python/2.7.koobs-freebsd10/build/Lib/uuid.py", line 348, in _arp_getnode
    ip_addr = socket.gethostbyname(socket.gethostname())
gaierror: [Errno 8] hostname nor servname provided, or not known
msg231176 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-11-14 19:40
This issue had added new tests. Here is a patch against 2.7 (3.4+ should use OSError instead of socket.gaierror) which fixes _arp_getnode().
msg231225 - (view) Author: Kubilay Kocak (koobs) Date: 2014-11-15 22:33
I only attached the 2.7 build log because the failures from 3.4 and 3.x are identical, copying them here for completeness:

From 3.4: 

======================================================================
ERROR: test_arp_getnode (test.test_uuid.TestUUID)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.4.koobs-freebsd10/build/Lib/test/test_uuid.py", line 324, in test_arp_getnode
    node = uuid._arp_getnode()
  File "/usr/home/buildbot/python/3.4.koobs-freebsd10/build/Lib/uuid.py", line 364, in _arp_getnode
    ip_addr = socket.gethostbyname(socket.gethostname())
socket.gaierror: [Errno 8] hostname nor servname provided, or not known

----------------------------------------------------------------------
Ran 18 tests in 0.209s

From 3.x:


======================================================================
ERROR: test_arp_getnode (test.test_uuid.TestUUID)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/test/test_uuid.py", line 325, in test_arp_getnode
    node = uuid._arp_getnode()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/uuid.py", line 362, in _arp_getnode
    ip_addr = socket.gethostbyname(socket.gethostname())
socket.gaierror: [Errno 8] hostname nor servname provided, or not known

----------------------------------------------------------------------
msg231242 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-11-16 10:21
Oh, socket.gaierror was not made an alias of OSError, only subclass of it. Well, we can apply the same patch to all releases. Does it fix tests?
msg231264 - (view) Author: Kubilay Kocak (koobs) Date: 2014-11-17 02:51
I don't have the environment to test here. Can you run a custom build on the buildbots?
msg231494 - (view) Author: Roundup Robot (python-dev) Date: 2014-11-21 20:00
New changeset 301d62ef5c0b by Serhiy Storchaka in branch '2.7':
Issue #17293: socket.gethostbyname() can raise an exception of FreeBSD.
https://hg.python.org/cpython/rev/301d62ef5c0b

New changeset 97ceab0bd6f8 by Serhiy Storchaka in branch '3.4':
Issue #17293: socket.gethostbyname() can raise an exception of FreeBSD.
https://hg.python.org/cpython/rev/97ceab0bd6f8
msg233473 - (view) Author: Dimitri John Ledkov (xnox) * Date: 2015-01-05 17:55
I'm getting socket.gaierror from test_ifconfig_getnode / uuid._ifconfig_getnode() on python 3.4.2 on Linux, in a no network environment. Thus i'd like to see these try:/excepts: to be ported back to 3.4 branch, if they haven't been already.

I filed http://bugs.python.org/issue23170 to track my issue.

Feel free to close that one as a (related) dupe of this one.
History
Date User Action Args
2015-01-05 17:55:00xnoxsetnosy: + xnox
messages: + msg233473
2014-11-27 22:52:52serhiy.storchakasetstatus: open -> closed
resolution: fixed
2014-11-21 20:00:16python-devsetmessages: + msg231494
2014-11-17 02:51:25koobssetmessages: + msg231264
2014-11-16 10:21:13serhiy.storchakasetmessages: + msg231242
2014-11-15 22:33:22koobssetmessages: + msg231225
2014-11-14 19:40:16serhiy.storchakasetfiles: + uuid_arp_getnode_freebsd.patch

messages: + msg231176
2014-11-14 02:02:01koobssetstatus: closed -> open
files: + koobs-freebsd10-python27-build758.log

nosy: + koobs
messages: + msg231148

resolution: fixed -> (no value)
2014-11-07 16:58:38serhiy.storchakasetstatus: open -> closed
resolution: fixed
messages: + msg230817

stage: patch review -> resolved
2014-11-07 10:27:23python-devsetnosy: + python-dev
messages: + msg230784
2014-10-23 10:32:35serhiy.storchakasetfiles: + uuid_netstat_getnode-3.5_2.patch, uuid_netstat_getnode-3.4.patch

messages: + msg229866
2014-10-22 19:22:37serhiy.storchakasetfiles: + uuid_netstat_getnode-3.5.patch

messages: + msg229833
2014-10-22 19:04:57hayposetmessages: + msg229832
2014-10-22 18:57:13serhiy.storchakasetmessages: + msg229831
2014-10-12 16:17:12serhiy.storchakasetmessages: + msg229172
2014-10-12 15:13:37hayposetmessages: + msg229163
2014-10-12 15:10:41hayposetnosy: + haypo
messages: + msg229162
2014-10-12 15:07:59neologixsetmessages: + msg229160
2014-10-12 14:44:56serhiy.storchakasetmessages: + msg229154
2014-10-12 14:39:05neologixsetmessages: + msg229153
2014-10-12 14:36:41serhiy.storchakasetmessages: + msg229151
2014-10-01 19:34:00serhiy.storchakasetfiles: + uuid_netstat_getnode.patch

nosy: + serhiy.storchaka
messages: + msg228108

assignee: serhiy.storchaka
stage: patch review
2014-07-23 13:43:51BreamoreBoysetmessages: + msg223738
2014-07-23 05:37:53neologixsetmessages: + msg223724
2014-07-22 22:26:37David.Edelsohnsetmessages: + msg223705
2014-07-22 20:35:05neologixsetnosy: + neologix
messages: + msg223693
2014-07-17 23:46:45BreamoreBoysetversions: + Python 3.4, Python 3.5, - Python 2.6
nosy: + BreamoreBoy, David.Edelsohn

messages: + msg223390

type: behavior
2013-02-25 07:29:08aivarskcreate