classification
Title: Lookup Error while importing idna from a worker thread
Type: Stage:
Components: Library (Lib) Versions: Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Ilya.Kulakov, jamesls
Priority: normal Keywords:

Created on 2017-01-16 22:30 by Ilya.Kulakov, last changed 2019-04-12 00:26 by jamesls.

Messages (2)
msg285586 - (view) Author: Ilya Kulakov (Ilya.Kulakov) * Date: 2017-01-16 22:30
See this post: https://github.com/kennethreitz/requests/issues/3578

The current workaround for requests is to have a no-op import somewhere in the code.

However, that doesn't really work for us: our python and stdlib are bundled by pyqtdeploy as in Qt resources system and we frequently see this error. On certain machines. Workaround that seems to work more reliably is a no-op look up via ''.encode('ascii').decode('idna').
msg340001 - (view) Author: James Saryerwinnie (jamesls) * Date: 2019-04-12 00:26
I ran into this as well also using the embedded distribution for windows (https://docs.python.org/3/using/windows.html#the-embeddable-package).

socket.getaddrinfo() will encode unicode hostnames using idna and trigger this error if you call this function in threads:

PS C:\Users\Administrator\Downloads\python-3.7.3-embed-amd64> cat .\repro.py
import threading
import socket


def task():
    try:
        socket.getaddrinfo('www.google.com', 443)
    except Exception as e:
        print("FAIL: %s" % e)
        raise


threads = []
for i in range(50):
    t = threading.Thread(target=task)
    threads.append(t)

for t in threads:
    t.start()

for t in threads:
    t.join()

print("DONE")


PS C:\Users\Administrator\Downloads\python-3.7.3-embed-amd64> .\python.exe .\repro.py
FAIL: unknown encoding: idna
FAIL: unknown encoding: idna
Exception in thread Thread-5:
Traceback (most recent call last):
  File "D:\obj\Windows-Release\37amd64_Release\msi_python\zip_amd64\threading.py", line 917, in _bootstrap_inner
  File "D:\obj\Windows-Release\37amd64_Release\msi_python\zip_amd64\threading.py", line 865, in run
  File ".\repro.py", line 7, in task
    socket.getaddrinfo('www.google.com', 443)
  File "D:\obj\Windows-Release\37amd64_Release\msi_python\zip_amd64\socket.py", line 748, in getaddrinfo
LookupError: unknown encoding: idna

FAIL: unknown encoding: idna
Exception in thread Thread-4:
Traceback (most recent call last):
  File "D:\obj\Windows-Release\37amd64_Release\msi_python\zip_amd64\threading.py", line 917, in _bootstrap_inner
  File "D:\obj\Windows-Release\37amd64_Release\msi_python\zip_amd64\threading.py", line 865, in run
  File ".\repro.py", line 7, in task
    socket.getaddrinfo('www.google.com', 443)
  File "D:\obj\Windows-Release\37amd64_Release\msi_python\zip_amd64\socket.py", line 748, in getaddrinfo
LookupError: unknown encoding: idna

Exception in thread Thread-6:
Traceback (most recent call last):
  File "D:\obj\Windows-Release\37amd64_Release\msi_python\zip_amd64\threading.py", line 917, in _bootstrap_inner
  File "D:\obj\Windows-Release\37amd64_Release\msi_python\zip_amd64\threading.py", line 865, in run
  File ".\repro.py", line 7, in task
    socket.getaddrinfo('www.google.com', 443)
  File "D:\obj\Windows-Release\37amd64_Release\msi_python\zip_amd64\socket.py", line 748, in getaddrinfo
LookupError: unknown encoding: idna

DONE


Confirmed that adding u''.encode('idna') fixes this issue.
History
Date User Action Args
2019-04-12 00:26:04jameslssetnosy: + jamesls
messages: + msg340001
2017-01-16 22:30:45Ilya.Kulakovcreate