In py3k, str.isalnum(), str.isdigit(), and str.isdecimal() are broken because they take into account various unicode numbers.

A common case is doing something like that:

num = -1
while num == -1:
    num_in = input('Enter a number> ')
    if num_in.isdigit():
        num = int(num_in)

# do stuff …

If you enter ¹, or any esoteric unicode representation of a number, all the methods referenced above will return True. I believe this is a bug.

It also affects the stdlib, e.g. in collection.namedtuple,
A = namedtuple('A¹', 'x y') will return an ugly Syntax Error, because the sanity check uses str.isalnum(), which says it’s ok. (n.b.: of course, no sane person should ever want to do the above, but I find it worth mentionning)

I think this is working as designed, and can't be changed (at least not easily) because of backward compatibility even if there are bits of the design that are deemed buggy.  The issue, I believe, is what is considered a number by the Unicode consortium.

See issue 10557 for some background.

Python should at least be consistent about what is treated as a number, though, so the ugly syntax error from namedtuple is probably a bug.

Actually, the character is SUPERSCRIPT ONE, in the category No (Number other).
http://www.fileformat.info/info/unicode/char/b9/index.htm

This is not a valid category for the identifiers, only "Nd" Number decimal is accepted.

The issue is probably in namedtuple, which should check the "unicodedata.category" or better, check with method str.isidentifier.

I understand the reasoning behind the feature, and the will to be unicode-compliant, but I think this might still break a lot of code (though it may never be detected).

I understand that isdecimal() is the safe way, because anything that is a decimal (Nd) can be translated to an integer by int() ; however, what is the recommended way to get something that isnumeric() into an int?

unicodedata.normalize('NFKD', num) or unicodedata.normalize('NFKC', num)?

Maybe str could have a method that does this, or methods performing exclusively on ascii values?

Sorry for the noise, I did not find issue 10557 when I searched.

This is consistent:

>>> '¹'.isnumeric(), '¹'.isdigit(), '¹'.isdecimal()
(True, True, False)

>>> unicodedata.numeric('¹')
1.0
>>> unicodedata.digit('¹')
1
>>> unicodedata.decimal('¹')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: not a decimal
>>> 

Changing the title to focus on the issue with collections.namedtuple.

namedtuple should simply use isidentifier(), rather than isalnum().

> namedtuple should simply use isidentifier(), rather than isalnum().
+2

New changeset 80bcc98bf939 by Raymond Hettinger in branch '3.3':
Issue #17331:  Use isidentifier() instead of isalnum() to check for valid identifiers.
http://hg.python.org/cpython/rev/80bcc98bf939

Thanks for the suggestion.  isidentifier() was exactly what was needed.