Somehow, it turns out that `TCP_NOTSENT_LOWAT` that's available since 3.7.x is not available in the official macOS builds 🙀:

> python3.7
Python 3.7.4 (v3.7.4:e09359112e, Jul  8 2019, 14:54:52)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> socket.TCP_NOTSENT_LOWAT
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'socket' has no attribute 'TCP_NOTSENT_LOWAT'

> python3.8
Python 3.8.2 (v3.8.2:7b3ab5921f, Feb 24 2020, 17:52:18)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> socket.TCP_NOTSENT_LOWAT
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'socket' has no attribute 'TCP_NOTSENT_LOWAT'

> python3.9
Python 3.9.0a4 (v3.9.0a4:6e02691f30, Feb 25 2020, 18:14:13)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> socket.TCP_NOTSENT_LOWAT
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'socket' has no attribute 'TCP_NOTSENT_LOWAT'

And my local build has it 😺:

> ~/cpython/python.exe
Python 3.9.0a4+ (heads/master:be501ca241, Mar  4 2020, 15:16:49)
[Clang 10.0.1 (clang-1001.0.46.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> socket.TCP_NOTSENT_LOWAT
513

So... my guess is official builds are using old SDK or header files. 🤔

My system has it e.g. here:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/Kernel.framework/Versions/A/Headers/netinet/tcp.h
230:#define TCP_NOTSENT_LOWAT       0x201   /* Low water mark for TCP unsent data */

And in fact it's present in every `netinet/tcp.h` on my system: CommandLineTools 10.14 and 10.5 sdks; MacOSX dev sdk, {AppleTV,Watch,iPhone}{OS,Simulator} sdks.

+macos team, because I can't for the life of me figure out how official builds are made ☹️

In short: my local build has socket.TCP_NOTSENT_LOWAT but the official build does not.

AFAIK the macOS builds are still build on the oldest macOS release supported by the installer (that is, a macOS 10.9 system). This means the build won't use macOS APIs introduced in macOS 10.10 or later.

----

It would be nice to build the installer using the latest compiler and SDK (more APIs available, better compiler, ...), but that requires work to explicitly avoid using APIs that aren't available on the system the binary is running on.

Thank you for the explanation, Ronald.

`socket.TCP_NOTSENT_LOWAT` is just a constant though, to be passed to `setsockopt`. 

What do you think of `ifndef ... define ...` work-around, akin to a few other constants in socket module?
https://github.com/python/cpython/blob/799d7d61a91eb0ad3256ef9a45a90029cef93b7c/Modules/socketmodule.h#L162-L168

I don't like that workaround. I'm also not sure if the value of these constants is the same on macOS and iOS (I do know that some constants higher up in the stack are not the same).

This issue is a duplicate of an earlier bug about missing functions that I can't find at the moment. That issue can only be solved by building on a recent enough version of macOS.

The constant value is the same for macOS and iOS: iphone, watch, tv:

~ > locate netinet/tcp.h | xargs grep LOWAT
/Applications/Xcode.app/Contents/Developer/Platforms/AppleTVOS.platform/Developer/SDKs/AppleTVOS.sdk/usr/include/netinet/tcp.h:#define TCP_NOTSENT_LOWAT       0x201   /* Low water mark for TCP unsent data */
/Applications/Xcode.app/Contents/Developer/Platforms/AppleTVSimulator.platform/Developer/SDKs/AppleTVSimulator.sdk/usr/include/netinet/tcp.h:#define TCP_NOTSENT_LOWAT       0x201   /* Low water mark for TCP unsent data */
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/Kernel.framework/Versions/A/Headers/netinet/tcp.h:#define TCP_NOTSENT_LOWAT       0x201   /* Low water mark for TCP unsent data */
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/netinet/tcp.h:#define TCP_NOTSENT_LOWAT       0x201   /* Low water mark for TCP unsent data */
/Applications/Xcode.app/Contents/Developer/Platforms/WatchOS.platform/Developer/SDKs/WatchOS.sdk/usr/include/netinet/tcp.h:#define TCP_NOTSENT_LOWAT       0x201   /* Low water mark for TCP unsent data */
/Applications/Xcode.app/Contents/Developer/Platforms/WatchSimulator.platform/Developer/SDKs/WatchSimulator.sdk/usr/include/netinet/tcp.h:#define TCP_NOTSENT_LOWAT       0x201   /* Low water mark for TCP unsent data */
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk/usr/include/netinet/tcp.h:#define TCP_NOTSENT_LOWAT       0x201   /* Low water mark for TCP unsent data */
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator.sdk/usr/include/netinet/tcp.h:#define TCP_NOTSENT_LOWAT       0x201   /* Low water mark for TCP unsent data */
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Kernel.framework/Versions/A/Headers/netinet/tcp.h:#define TCP_NOTSENT_LOWAT       0x201   /* Low water mark for TCP unsent data */
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/netinet/tcp.h:#define TCP_NOTSENT_LOWAT       0x201   /* Low water mark for TCP unsent data */
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Kernel.framework/Versions/A/Headers/netinet/tcp.h:#define TCP_NOTSENT_LOWAT       0x201   /* Low water mark for TCP unsent data */
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/netinet/tcp.h:#define TCP_NOTSENT_LOWAT       0x201   /* Low water mark for TCP unsent data */

Windows build encountered a similar problem, see issue32394.
The solution is to check the runtime system version when importing socket module, if it is an older system, delete the constants. [1]

issue32394 has a small script (winsdk_watchdog.py) to help find such constants, usage:
1, build a CPython build with old SDK.
2, use winsdk_watchdog.py, dump possible affected constants to a file `winsdk_dump.json`.
3, build a CPython build with new SDK.
4, use winsdk_watchdog.py, compare constants between two builds .

If a new constant is introduced by new SDK/API, we remove it on older system during runtime.
Otherwise we can ignore this new constant, this means it has nothing to do with the new SDK.
(msg311858 is a demo.)

We don't need to use winsdk_watchdog.py routinely, just use it when updating the building SDK, this process only takes about 10~20 minutes.

I think macOS build can also uses this process.

[1]
The commit:
https://github.com/python/cpython/commit/19e7d48ce89422091f9af93038b9fee075d46e9e

Note that there was a minor fix later:
https://github.com/python/cpython/commit/8905fcc85a6fc3ac394bc89b0bbf40897e9497a#diff-a47fd74731aeb547ad780900bb8e6953

Wow, very curious.

Yes, it's very much like `socket.TCP_KEEPCNT` in that respect, though, admittedly I don't have a very old Mac to test this right now. I think there were VMs for that maybe? 🤔



I wonder, what failure would be best for a naive code below, a NameError or OSError(errno=42)?

sock.setsockopt(socket.SOL_TCP, socket.TCP_NOTSENT_LOWAT, 42)

I guess the question is, at what level should the users catch exceptions...
After all, we don't delete this constant on Linux, and surely someone somewhere runs a very old kernel...



Oddly according to https://github.com/search?l=Python&q=%22socket.TCP_NOTSENT_LOWAT%22&type=Code none (in the OSS community) appears to be using this feature yet?

The search without `socket.` prefix yields a bunch of vendored mypy pyi's, but no actual code either.

And some even work around the constant being optional: https://github.com/python-trio/trio/blob/5b91edb2a860d024ab057e2be55fb34f311bf8ed/trio/socket.py#L170-L178



So, would this be a "not a regression" if none appears to use this constant yet?

Or do we take "don't break existing code" so seriously, that in this case too, we ought to assume that there's someone out there who has private code like below which we must not break?:

if code := getattr(socket, "TCP_NOTSENT_LOWAT", None):
    sock.setsockopt(socket.SOL_TCP, code, 42)



P.S.
If someone wants to take https://github.com/python/cpython/pull/19402 forward, by all means :)
Or I can try to hack up delete-at-runtime...

It seems that people usually use the socket module like this, I think it's safe to respect this habit:

    if hasattr(socket, "FLAG_NAME"):
        do_something

If use PR19402, your program will have problem on the older version system, not only "don't break existing code".

So I think delete-at-runtime is a suitable way.

The active PR on bpo-41100 will make it possible to build the installer on the latest version of macOS (which will be needed to be able to provide support for "Apple Silicon").

IMHO it is not necessary to remove unavailable constants, on some platforms (in particular Linux) some of the constants might not work because the relevant kernel module is not loaded. That's something you can only find by actively probing, which is too expensive and error prone. 

I propose closing this issue: the installers are what they are for a reason, and a better solution than proposed in this issue is on its way.

Indeed, this issue can be closed, when the mac build update is confirmed to be included in a specific upcoming Python version.

The work in #441100 is merged into master and 3.9, including an (experimental) "universal2" build of the latter in 3.9.1rc1 that is build with the latest SDK.

I'm therefore closing this issue.