New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bdist_wininst depends on MBCS codec, unavailable on non-Windows #55154
Comments
If distutils.commands.bdist_wininst.bdist_wininst.get_inidata returns a unicode string (which is always the case in 3.x and can happen in 2.x if if you pass a unicode object as one setup argument), bdist_wininst will try to use the MBCS codec, which is not available on non-Windows OSes (see http://docs.python.org/dev/library/codecs#module-encodings.mbcs). If someone wants to make a patch, here are some guidelines: http://wiki.python.org/moin/Distutils/FixingBugs |
I believe it is better to start and commit a testcase . |
We don’t commit failing tests :) If you mean you want to write a test case, please go ahead and attach a patch or changeset URI. |
Please don't ask me about patches. Core devs won't accept them. |
One of your patches for diff has been accepted; some distutils bugs have |
I've just been bitten by this when trying to do a new release of roundup, why not make the mbcs codec available on non-windows platforms as has been proposed (and rejected) in bpo-1251921 -- any non-technical reasons for not including this codec on other platforms? |
The mbcs codec depends on the Windows installation. On most Western Windows it is similar to cp1252, Japanese Windows will use cp932, and so on. |
Can't you only work with Unicode and avoid the MBCS encoding? |
On Thu, Jul 07, 2011 at 10:52:51AM +0000, STINNER Victor wrote:
I'm trying to build a windows binary package on Linux. This usually (and don't tell me I have to install windows for building a windows Thanks, RalfDr. Ralf Schlatterbeck Tel: +43/2243/26465-16 |
haypo:
|
bdist_command append configuration data to a wininst-xxx.exe binary. Where does this file come from? Can we modify wininst-xxx.exe binaries? If we can modify the binaries, we can change the format to store the configuration data as UTF-8 instead of the ANSI code page. It's surprising and "unsafe" (not portable) to use the ANSI code page for an installer: if you build your installer on a french setup (ANSI=cp1252), the configuration was be interpreted incorrectly on a japanese setup (ANSI=cp932). Example: >>> 'Hé ho'.encode('cp1252').decode('cp932')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'cp932' codec can't decode bytes in position 1-2: illegal multibyte sequence So if the configuration data (package metadata) contains non-ASCII characters, you will not be able to use your installer on a computer using a different ANSI code page than the code page of the computer used to build the installer... In the best case, you will just get mojibake. If we cannot modify wininst-xx.exe, an alternative to be able to generate installers on non-Windows platforms is to use the most common ANSI code page (cp1252?), or maybe ASCII. Use the ASCII encoding is the safest solution because you will be able to use your installer on all Windows setup (all ANSI code pages are compatible with ASCII), but you will not be able to generate an installer if the package metadata contains at least one non-ASCII character... |
|
The problem is that the config file is parsed using GetPrivateProfileString, and the result is then passed to TextOut, SetDlgItemText, CreateWindow, etc. all of which are defined to accept MBCS strings. I agree that this can't work correctly in the general case. Changing the GUI functions to operate on Unicode strings is certainly feasible and a good idea. The main challenge then is the format of the INI file. IIUC, GetPrivateProfileStringW is willing to process UTF-16 (with BOM) encoded INI files, but I never tested whether it actually does. If it does, I recommend to have the INI file encoded in UTF-16 (with BOM, using LE for safety). In porting wininst.exe, it seems tempting to use the TEXT family of APIs. I'd advise against that, and recommend to explicitly use the *W functions. |
These functions are available with an Unicode API: GetPrivateProfileStringW
bdist_wininst creates a long Unicode string for the INI data, and then encode if isinstance(cfgdata, str):
cfgdata = cfgdata.encode("mbcs") So I suppose that replacing "mbcs" by "UTF-16-LE" and add the BOM should be
Do we need to keep backward compatibility if we change the format of the config |
The wininst.exe belongs to the version of distutils it is released with. |
Would the proposed change mean that a bdist_wininst built with 3.2.0 won’t work with a patched 3.2.3? |
The proposition of using other C functions and changing the bdist_wininst code looks risky to me, especially as I don’t know how compatibility would be affected (see my previous message). We are free to improve the wininst code in distutils2, or discuss a replacement (Jeremy Kloth was working on something with all the features of MSI and wininst), but for distutils I would very much prefer the simplest fix that could possibly works. bdist_msi decodes data read from setup.py with MBCS on Windows; on other OSes, couldn’t the locale preferred encoding be used? |
The installer doesn't use distutils to read its configuration, so such binary runs with any installed Python version.
It would be worse: Linux doesn't use Windows code page. Most modern OSes are now using UTF-8 locale encoding, whereas Windows never use UTF-8 as the ANSI code page. |
I've opened a PR thet removes the support for bdist_wininst on non-Windows. Apparently, it was broken since the beginning of Py3k anyway. The support can be reintroduced once it is actually fixed (or never). |
It would be better to use the Unicode (wide character) flavor of the Windows API to avoid completely any explicitly encoding, and only pass Unicode strings. But This issue is open for 8 years and it seems like nobody cares enough to invest time to implement such change. So I'm fine with trivial PR 14506. It doesn't change the status quo: bdist_win was always broken on non-Windows platforms since Python 3.0, but the PR makes the status quo more obvious which is a good thing. |
I started a discussion on the Packaging list: "Deprecate bdist_wininst" |
Follow-up issue: bpo-37468 "Don't install wininst*.exe on non-Windows platforms". |
Lib/distutils/tests/test_bdist_wininst.py contains an interesting comment linking to bpo-5731:
Related commit: commit ad95826
But there is also bpo-8954 follow-up which is still open: "wininst regression: errors when building on linux". |
The distutils bdist_wininst command has been removed in Python 3.10: see bpo-42802. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: