This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: socket.getfqdn and socket.gethostbyname fail on MacOS
Type: behavior Stage:
Components: macOS Versions: Python 3.10, Python 3.9, Python 3.8, Python 3.7, Python 3.6, Python 3.4, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: James Crowther, ghickman, ned.deily, ronaldoussoren, ssbarnea
Priority: normal Keywords:

Created on 2018-11-04 16:50 by ssbarnea, last changed 2022-04-11 14:59 by admin.

Messages (11)
msg329237 - (view) Author: Sorin Sbarnea (ssbarnea) * Date: 2018-11-04 16:50
It seems that when the MacOS machine does not have a FQDN name manually configured using `scutil`, python will be fail to resolve domain names and timeout after ~30 seconds.

I mention that the DNS resolution works correctly on the machine and that the DHCP servers provides one ore more domains to be useles for resolution. Tools like nslookup or ping are not affected by this, only python (tried py27,34-36 and got same behavior).

Usually python user encounters errors like:

Traceback (most recent call last):
    ...
    socket.gethostbyname(socket.gethostname())
gaierror: [Errno 8] nodename nor servname provided, or not known

One these machines `hostname` cli command returns a valid FQDN name like `foo.lan` but scutil will not return one:
```
sudo scutil --get HostName
HostName: not set
```

One temporary workaround is to manually run:
```
sudo scutil --set HostName `hostname`
```

This will set the hostname and python sockets functions will start to work (after few minutes as some caching is involved).

Still, we cannot expect users to run this command themselves. Even worse, they would have to re-run this each time the roam to another network that may have a different set of domains.

See: https://stackoverflow.com/questions/52602107
msg329238 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-11-04 17:00
Thanks for the report but I don't think the problem you are seeing is that simple.  On my 10.14.1 system, I don't see that problem:

$ scutil --get HostName
HostName: not set
$ /usr/local/bin/python3.7
Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 03:13:28)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> socket.gethostname()
'harj.local'
>>> socket.gethostbyname(socket.gethostname())
'10.0.1.8'
>>>

So there's likely some other configuration issue here.
msg329240 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-11-04 17:26
See also Issue29705.
msg329295 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2018-11-05 12:29
I agree with Ned. The code fragment works for me as well on 10.14.1 and I've used simular code in the past on different versions of macOS as well.

@ssbarnea: 
- What is the hostname as shown in the "Sharing" preference pane? Does the name include special characters (anything but ASCII letters and digits)?
- What version of macOS do you use?
- How did you install Python? (Python.org installer, homebrew, self built, ...)
msg329605 - (view) Author: James Crowther (James Crowther) Date: 2018-11-10 08:37
Hi guys, I am the OP for the stack overflow issue question that sparked this bug report. 


In response to Ned's question about configuration of the system:

MacOS 10.14.1
Installed x-code and command line tools
python versions used, varies, 3.5.3, 3.6 and 3.7.0 (sorry can't be more specific about 3.6 since I've used a lot. These python configurations have been both a bundled python env in blender, and also MacOS's native python 2.7.

So far they've all had the same issue, that being;

socket.gethostname() 

returns a value that seems to be set by the router/switch if 

scutil --set hostname somehostname 

has not been run. What Sorin mentioned on the stack overflow question that he didn't here (and I feel is important) is that ping and nslookup work with the expected hostname, i.e. the one the user can actually see by going to the system preferences and viewing it under the sharing section. Here it is called "Computer name", analogous to the same used on windows (which I might add has no such issues with hostnames being set to something other than the computer name unless a certain command is run).

So the real issue for us, is how come ping and nslookup will work fine with the macOS "computer name" but the socket module uses something else, seemingly the hostname and computer name are not or loosely connected on macOS perhaps. 

Ideally what we need here is a way to ensure that the user can input the computer name of a target machine on another system and that system resolve the ip address for the target machine. We can't currently use socket.gethostname for this on mac since it seems to require that the scutil --set hostname command has been run. Otherwise the hostname that is returned doesn't work with socket.gethostbyname(hostname).

Hope this helps, happy to clarify.

J.
msg331102 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2018-12-05 12:10
This issue does not have enough information to determine what is wrong. It is however fairly clear that it is not related to "scutil --get HostName" mentioning that there is no hostname set as that is the normal state of systems, and I have never run into the problem mentioned here (and have used macOS exclusively from one of the earliest public beta's of macOS 10.0).

My gut feeling is that this is some issue with your system or environment.

Some more questions:

- Is this is a consistent problem, or only on some networks?
- Do you have any custom settings in the Networking preference pane related to name resolution (extra search domains, DNS servers, ...)?
- Do the domains returned by the DHCP server actually exist?
- Likewise for the DNS servers
- Does the result of socket.gethostname() look sane? In particular, does this include a domain name?
- The original message mentions: "and that the DHCP servers provides one ore more domains to be useles for resolution.". What is meant by that? AFAIK the DHCP server can only return 1 domain name, not a full DNS search list (or at least not one that is used by most major platforms).
msg331751 - (view) Author: James Crowther (James Crowther) Date: 2018-12-13 09:21
Hi Ronald, I've replied to your comments below



>My gut feeling is that this is some issue with your system or environment.

I Doubt it, happens with another MacBook air which is literally out of the box, no mods.

>Some more questions:

>- Is this is a consistent problem, or only on some networks? 
All networks
>- Do you have any custom settings in the Networking preference pane related to name resolution (extra search domains, DNS servers, ...)? 
No
>- Do the domains returned by the DHCP server actually exist?
Not sure
>- Likewise for the DNS servers
likewise not sure
>- Does the result of socket.gethostname() look sane? In particular, does this include a domain name?
it is given a name that includes the ip address and the name of the router. 
>- The original message mentions: "and that the DHCP servers provides one ore more domains to be useles for resolution.". What is meant by that? AFAIK the DHCP server can only return 1 domain name, not a full DNS search list (or at least not one that is used by most major platforms).
Not sure what was meant by this comment.
msg331753 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2018-12-13 09:54
>>My gut feeling is that this is some issue with your system or environment.

> I Doubt it, happens with another MacBook air which is literally out of the box, no mods.

But on the other hand most other users (including myself and Ned) don't have this problem. 

> >- Does the result of socket.gethostname() look sane? In particular, does this include a domain name?
> it is given a name that includes the ip address and the name of the router. 

Does resolving that name work? If it doesn't that points to a problem with your environment. 

What's the result of resolving the IP address of your system? That name gets used by the system when available, and that could explain the issue your seeing.
msg359718 - (view) Author: George Hickman (ghickman) Date: 2020-01-10 11:28
I came across this today with the same timeout behaviour on macOS 10.14.6.

After some digging I tracked it down to the Search Domains setting of my local network, this was set to "local", removing that immediately fixed the issue.
msg414834 - (view) Author: Sorin Sbarnea (ssbarnea) * Date: 2022-03-10 09:15
I am able to reproduce this bug with Python 3.10 on MacOS 12.2 too, and I happen to have some extra insights about what happens, see https://github.com/tox-dev/tox/issues/2375

Basically the culprit is a reverse-dns query (PTR) that takes 30s to return, causing that very long delay. It appears that mDNSResponder is able to cache the result for some time but not very long, so it will happen again.

In my particular case that PTR query is an .ip6.arpa one and there is no ipv4 query made at all.

I was not able to identify any similar delay while calling `scutil --get HostName` or `hostname`, but `socket.getfqdn()` does reproduce it reliably.

Interestingly, it seems that the delay is at least sometimes 15s, maybe the delay of 30s I observed on tox was caused by two calls.

Anyway, I am more than interested in finding a solution for this issue. Such a delays can be a real dealbreaker even for cli tools.
msg414835 - (view) Author: Sorin Sbarnea (ssbarnea) * Date: 2022-03-10 09:27
Maybe I should mention that my networking is configured only with values received from the DHCP server, which includes 2 DNS servers, one ipv4 an done ipv6, both of them being the local router and *1* search domain, which is a real domain (not local).
History
Date User Action Args
2022-04-11 14:59:07adminsetgithub: 79345
2022-03-10 09:27:18ssbarneasetmessages: + msg414835
2022-03-10 09:15:02ssbarneasetmessages: + msg414834
versions: + Python 3.8, Python 3.9, Python 3.10
2020-01-10 11:28:23ghickmansetnosy: + ghickman
messages: + msg359718
2018-12-13 09:54:44ronaldoussorensetmessages: + msg331753
2018-12-13 09:21:37James Crowthersetmessages: + msg331751
2018-12-05 12:10:47ronaldoussorensetmessages: + msg331102
2018-11-10 08:38:00James Crowthersetnosy: + James Crowther
messages: + msg329605
2018-11-05 12:29:01ronaldoussorensetmessages: + msg329295
2018-11-04 17:26:51ned.deilysetmessages: + msg329240
2018-11-04 17:00:57ned.deilysetmessages: + msg329238
2018-11-04 16:50:41ssbarneacreate