New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
str.capitalize contradicts oneself #56475
Comments
specification str.capitalize()¶
>>> '\u1ffc', '\u1ff3'
('ῼ', 'ῳ')
>>> '\u1ffc'.isupper()
False
>>> '\u1ff3'.islower()
True
>>> s = '\u1ff3\u1ff3\u1ffc\u1ffc'
>>> s
'ῳῳῼῼ'
>>> s.capitalize()
'ῼῳῼῼ'
>>> A: lower A -> B & !B -> A |
This looks like a duplicate of bpo-12204. |
in http://bugs.python.org/issue12204 this problem is another lowering works
>>> '\u1ffc'
'ῼ'
>>> '\u1ffc'.lower()
'ῳ'
>>> |
Indeed this seems a different issue, and might be worth fixing it.
Given this definition:
str.capitalize()¶
Return a copy of the string with its first character capitalized and the rest lowercased.
we might implement capitalize like:
>>> def mycapitalize(s):
... return s[0].upper() + s[1:].lower()
...
>>> 'fOoBaR'.capitalize()
'Foobar'
>>> mycapitalize('fOoBaR')
'Foobar'
And this would yield the correct result:
>>> s = u'\u1ff3\u1ff3\u1ffc\u1ffc'
>>> print s
ῳῳῼῼ
>>> print s.capitalize()
ῼῳῼῼ
>>> print mycapitalize(s)
ῼῳῳῳ
>>> s.capitalize().istitle()
False
>>> mycapitalize(s).istitle()
True This doesn't happen because the actual implementation of str.capitalize checks if a char is uppercase (and not if it's titlecase too) before converting it to lowercase. This can be fixed doing:
diff -r cb44fef5ea1d Objects/unicodeobject.c
--- a/Objects/unicodeobject.c Thu Jul 21 01:11:30 2011 +0200
+++ b/Objects/unicodeobject.c Thu Jul 21 07:57:21 2011 +0300
@@ -6739,7 +6739,7 @@
}
s++;
while (--len > 0) {
- if (Py_UNICODE_ISUPPER(*s)) {
+ if (Py_UNICODE_ISUPPER(*s) || Py_UNICODE_ISTITLE(*s)) {
*s = Py_UNICODE_TOLOWER(*s);
status = 1;
} |
I think it would be better to use this code: if (!Py_UNICODE_ISUPPER(*s)) {
*s = Py_UNICODE_TOUPPER(*s);
status = 1;
}
s++;
while (--len > 0) {
if (Py_UNICODE_ISLOWER(*s)) {
*s = Py_UNICODE_TOLOWER(*s);
status = 1;
}
s++;
} Since this actually implements what the doc-string says. Note that title case is not the same as upper case. Title case is |
Do you mean "if (!Py_UNICODE_ISLOWER(*s)) {" (with the '!')? This sounds fine to me, but with this approach all the uncased characters will go through a Py_UNICODE_TO* macro, whereas with the current code only the cased ones are converted. I'm not sure this matters too much though. OTOH if the non-lowercase cased chars are always either upper or titlecased, checking for both should be equivalent. |
Ezio Melotti wrote:
Sorry, here's the correct version: if (!Py_UNICODE_ISUPPER(*s)) {
*s = Py_UNICODE_TOUPPER(*s);
status = 1;
}
s++;
while (--len > 0) {
if (!Py_UNICODE_ISLOWER(*s)) {
*s = Py_UNICODE_TOLOWER(*s);
status = 1;
}
s++;
}
AFAIK, there are characters that don't have a case mapping at all. Someone would have to check this against the current Unicode database. |
>>> import sys; hex(sys.maxunicode)
'0x10ffff'
>>> import unicodedata; unicodedata.unidata_version
'6.0.0' import unicodedata
all_chars = list(map(chr, range(0x110000)))
Ll = [c for c in all_chars if unicodedata.category(c) == 'Ll']
Lu = [c for c in all_chars if unicodedata.category(c) == 'Lu']
Lt = [c for c in all_chars if unicodedata.category(c) == 'Lt']
Lo = [c for c in all_chars if unicodedata.category(c) == 'Lo']
Lm = [c for c in all_chars if unicodedata.category(c) == 'Lm'] >>> [len(x) for x in [Ll, Lu, Lt, Lo, Lm]]
[1759, 1436, 31, 97084, 210]
>>> sum(1 for c in Lu if c.lower() == c)
471 # uppercase chars with no lower
>>> sum(1 for c in Lt if c.lower() == c)
0 # titlecase chars with no lower
>>> sum(1 for c in Ll if c.upper() == c)
760 # lowercase chars with no upper
>>> sum(1 for c in Lo if c.upper() != c or c.title() != c or c.lower() != c)
0 # "Letter, other" chars with a different upper/title/lower case
>>> sum(1 for c in Lm if c.upper() != c or c.title() != c or c.lower() != c)
0 # "Letter, modifier" chars with a different upper/title/lower case
>>> sum(1 for c in all_chars if c not in L and (c.upper() != c or c.title() != c or c.lower() != c))
85 # non-letter chars with a different upper/title/lower case
>>> [c for c in all_chars if c not in L and (c.upper() != c or c.title() != c or c.lower() != c)]
['', 'Ⅰ', 'Ⅱ', 'Ⅲ', 'Ⅳ', 'Ⅴ', 'Ⅵ', 'Ⅶ', 'Ⅷ', 'Ⅸ', 'Ⅹ', 'Ⅺ', 'Ⅻ', 'Ⅼ', 'Ⅽ', 'Ⅾ', 'Ⅿ', 'ⅰ', 'ⅱ', 'ⅲ', 'ⅳ', 'ⅴ', 'ⅵ', 'ⅶ', 'ⅷ', 'ⅸ', 'ⅹ', 'ⅺ', 'ⅻ', 'ⅼ', 'ⅽ', 'ⅾ', 'ⅿ', 'Ⓐ', 'Ⓑ', 'Ⓒ', 'Ⓓ', 'Ⓔ', 'Ⓕ', 'Ⓖ', 'Ⓗ', 'Ⓘ', 'Ⓙ', 'Ⓚ', 'Ⓛ', 'Ⓜ', 'Ⓝ', 'Ⓞ', 'Ⓟ', 'Ⓠ', 'Ⓡ', 'Ⓢ', 'Ⓣ', 'Ⓤ', 'Ⓥ', 'Ⓦ', 'Ⓧ', 'Ⓨ', 'Ⓩ', 'ⓐ', 'ⓑ', 'ⓒ', 'ⓓ', 'ⓔ', 'ⓕ', 'ⓖ', 'ⓗ', 'ⓘ', 'ⓙ', 'ⓚ', 'ⓛ', 'ⓜ', 'ⓝ', 'ⓞ', 'ⓟ', 'ⓠ', 'ⓡ', 'ⓢ', 'ⓣ', 'ⓤ', 'ⓥ', 'ⓦ', 'ⓧ', 'ⓨ', 'ⓩ']
>>> list(c.lower() for c in _)
['', 'ⅰ', 'ⅱ', 'ⅲ', 'ⅳ', 'ⅴ', 'ⅵ', 'ⅶ', 'ⅷ', 'ⅸ', 'ⅹ', 'ⅺ', 'ⅻ', 'ⅼ', 'ⅽ', 'ⅾ', 'ⅿ', 'ⅰ', 'ⅱ', 'ⅲ', 'ⅳ', 'ⅴ', 'ⅵ', 'ⅶ', 'ⅷ', 'ⅸ', 'ⅹ', 'ⅺ', 'ⅻ', 'ⅼ', 'ⅽ', 'ⅾ', 'ⅿ', 'ⓐ', 'ⓑ', 'ⓒ', 'ⓓ', 'ⓔ', 'ⓕ', 'ⓖ', 'ⓗ', 'ⓘ', 'ⓙ', 'ⓚ', 'ⓛ', 'ⓜ', 'ⓝ', 'ⓞ', 'ⓟ', 'ⓠ', 'ⓡ', 'ⓢ', 'ⓣ', 'ⓤ', 'ⓥ', 'ⓦ', 'ⓧ', 'ⓨ', 'ⓩ', 'ⓐ', 'ⓑ', 'ⓒ', 'ⓓ', 'ⓔ', 'ⓕ', 'ⓖ', 'ⓗ', 'ⓘ', 'ⓙ', 'ⓚ', 'ⓛ', 'ⓜ', 'ⓝ', 'ⓞ', 'ⓟ', 'ⓠ', 'ⓡ', 'ⓢ', 'ⓣ', 'ⓤ', 'ⓥ', 'ⓦ', 'ⓧ', 'ⓨ', 'ⓩ']
>>> len(_)
85
>>> {unicodedata.category(c) for c in all_chars if c not in L and (c.upper() != c or c.title() != c or c.lower() != c)}
{'So', 'Mn', 'Nl'} So == Symbol, Other |
L ? |
L = set(sum([Ll, Lu, Lt, Lo, Lm], [])) |
Attached patch + tests. |
New changeset c34772013c53 by Ezio Melotti in branch '3.2': New changeset eab17979a586 by Ezio Melotti in branch '2.7': |
New changeset 1ea72da11724 by Ezio Melotti in branch 'default': |
Fixed, thanks for the report! |
New changeset d3816fa1bcdf by Ezio Melotti in branch '2.7': |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: