We found that using -X ---- can be very confusing and annoying when you make typos, so I think having errors if you pass some unknown stuff will be outweighing any downside of making this more strict.

New changeset db2b6a2 by Pablo Galindo Salgado in branch 'main':
bpo-45445: Fail if an invalid X-option is provided in the command line (GH-28823)
db2b6a2

New changeset 79bc5e1 by Pablo Galindo Salgado in branch 'main':
bpo-45445: Remove incorrectly commited test file (GH-28972)
79bc5e1

I'm opening this again as, this is still not finished as we want to still extend the initialization API to allow a non static string to improve the error message.

In general, the protocol is to ask if there is something else to be done before closing (and giving it some time before closing if nobody answers) :)

Your documentation has stated for YEARS that arbitrary values can be passed via PyConfig.xoptions or -X. 9583cac

Quoting your own documentation:
"It also allows passing arbitrary values and retrieving them through the sys._xoptions dictionary."

There is client software out there relying on this. kovidgoyal/kitty#5223

Please do not randomnly break backwards compatibility, without even a deprecation period, because something "annoys" you.

@kovidgoyal Thanks for your message.

Please, be kind. There are much better ways to communicate your concerns than stating that we "randomly break backwards compatibility because something annoys us" or saying that "Apparently in Python-land its acceptable behavior to break backward
compatibility with documented interfaces on a whim. Bloody joke" as you did in your workaround commit.We all try to do our best here and although I can understand that you are concerned, there is no reason to not be polite.

This is a bit of a grey area because the _xoptions is a private attribute so we need to discuss what to do here.

Thanks in advance

CC: @vstinner

I was simply quoting you, you said you made this change because "We found that using -X ---- can be very confusing and annoying when you make typo". That is really not a good enough reason to break backwards compat.

I apologise for my tone, but you guys need to take backwards compat more seriously.

As for this issue, I dont care personally anymore, I have already worked around it. If you insist on breaking backwards compat then I suggest you add some loud warnings to the changelog. Just think of some poor dude running a script with python -X some_typo in it and that script now failing because python went from 3.10 to 3.11.

I apologise for my tone, but you guys need to take backwards compat more seriously.

We take backward compatibility very seriously. In this case, the incompatible change was done on purpose.

It was too easy to make typos and Python would not notice them. For example, python3.10 -X fautlhandler does not enable faulthandler and Python 3.10 doesn't say anything about the typo. There are many ways to introduce typos:

• pycacheprefix or pycache-prefix instead of pycache_prefix
• UTF8 or utf_8 or utf-8 instead of utf8
• no_debug_range instead of no_debug_ranges
• etc.

For other command line options, Python is strict. For example, python -o fails with a fatal error, whereas python -O works as expected. Same for long options like --check-hash-based-pyc always (instead of --check-hash-based-pycs always).

I don't see where the -X command line option is documented as it could be used by applications or 3rd party Python modules. It's documented as: Reserved for various implementation-specific options.

https://docs.python.org/dev/using/cmdline.html#cmdoption-X

IMO the most portable way to pass arbitrary options to Python is to use environment variables. It works on any Python versions and any operating system.

By the way, I added sys.orig_argv to Python 3.10: https://docs.python.org/dev/library/sys.html#sys.orig_argv

If you insist on breaking backwards compat then I suggest you add some loud warnings to the changelog.

I'm not comfortable with your tone of your messages.

If you are affected and you consider that the change is not properly advertized in https://docs.python.org/dev/whatsnew/3.11.html I suggest you proposing a pull request to better document the change. Python is maintained by volunteers, you're are not a customer of a paid service.

I agree - this was an unacceptable breaking change to the behaviour of the -X option. It's clearly (and still!) documented as accepting arbitrary values, so we can't change that without a deprecation period.

I am fine reverting the change in 3.11 and adding a deprecation period if we all agree that the API is considered public. I think personally I agree with @zooba and the OP, this change should be reverted (at least in 3.11) and a deprecation warning should be emitted.

What about deprecating un-namespaced options (namespacing to be bikeshedded)? So if it doesn't start with *waves hands* then it has to match one of our options, but otherwise it's passed through.

Clearly there are people with uses for this, and I've used it myself to inject arguments that weren't suitable for env variables for whatever reason (typically to avoid it being passed on to subprocesses but without having to control subprocess launches). Offering something to migrate to that isn't OS specific seems the least we can do here.

If there's a use case for which people have been using -x then we should definitely provide something to meet that need. However, the status quo isn't viable due to the typo problem. A namespace prefix on options could work.

For the time being, I am opening a revert

I have merged the revert as the change was technically backwards incompatible and we are super close to RC1:

#94745

We can discuss bringing it back and how for future versions.

Thanks for reverting this. I don't think making -X arbitrarything an error is a good idea. It means when we add a -X setting to a version of Python, be it a feature in a minor release or be it a security thing in a patch release, nobody can use it as a flag until they are guaranteed to never encounter specific interpreter versions that support the new flag value. That devalues the concept of -X. We've added -X things to control security fixes in the past and are likely to in the future.

I understand the typo problem. But people should be testing that their flags are doing what they intended in some form or another.

At most I suggest just issuing a warning about an unsupported -X flag name when in PYTHONVERBOSE mode.