classification
Title: Abandoned Tools/unicode/mkstringprep.py
Type: behavior Stage: resolved
Components: Demos and Tools, Unicode Versions: Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: Arfrever, ezio.melotti, georg.brandl, loewis, python-dev, serhiy.storchaka
Priority: low Keywords: needs review, patch

Created on 2012-07-01 17:36 by serhiy.storchaka, last changed 2013-06-09 14:21 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
mkstringprep.patch serhiy.storchaka, 2012-07-01 17:37 review
mkstringprep_min.patch serhiy.storchaka, 2013-05-28 17:42 review
mkstringprep_cosmetic.patch serhiy.storchaka, 2013-05-28 17:51 review
mkstringprep_check.patch serhiy.storchaka, 2013-05-28 17:54 review
Messages (17)
msg164489 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-07-01 17:36
It seems that Tools/unicode/mkstringprep.py has not been used for many years. Now it is not valid Python3 code nor Python2 code. The proposed patch fixes all porting errors.

Apparently, Lib/stringprep.py would have regenerated. Tools/unicode/mkstringprep.py output is a bit different from Lib/stringprep.py (in b3_exceptions).
msg164490 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-07-01 17:40
Indeed, the code hasn't been run, and really shouldn't have to. If it produces different output today, we should investigate why.
msg164510 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-07-02 10:23
>  If it produces different output today, we should investigate why.

Lib/stringprep.py differs from updated Tools/unicode/mkstringprep.py output only by additional entity 0x130:'i\u0307' in b3_exceptions.

In 3.2 and lower '\u0130'.lower() == '\u0069', in 3.3 '\u0130'.lower() == '\u0069\u0307'.
msg167514 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-08-05 21:30
The difference is in the reverse direction, right? I.e. the special case for U+0130 is removed.

This is harmless: it just means that we don't need to special-case that anymore.

I'm puzzled though that b3_exceptions doesn't become empty in 3.3. Supposedly, B.3 is CaseFolding.txt, and supposedly, Python 3.3 supports CaseFolding.txt. For example, CaseFolding maps U+00B5 to U+03BC in lower case, yet Python 3.3 maps it to U+00B5.
msg167916 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-08-10 21:17
To protect themselves from the surprises we need working mkstringprep.py. Martin, what do you say about a patch?
msg167931 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-08-11 04:18
I don't consider it relevant for 3.3; I may have time to review only post-3.3.
msg178319 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-12-27 20:22
Martin, are you have time to review now?
msg179465 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-01-09 16:07
Georg, now Tools/unicode/mkstringprep.py is broken in all Python 3 branches (it not usable at all, under 3.2 it even raises SyntaxError). Do you object against fixing it in 3.2 and 3.3?
msg179470 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2013-01-09 16:19
No need to hurry; no one apparently needs it anyway, and if there is a change let Martin review it before commit.
msg190211 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-05-28 13:47
Is there a chance, Martin?
msg190223 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2013-05-28 16:16
I find the patch too large to review, and it appears to contain unrelated changes. Can you kindly split it up into two patches, namely

A) changes that are really absolutely necessary to make it work again
B) patches that are purely cosmetic, and do not affect the behavior at all (such as rewriting .startswith, or replacing %s with %r)

Possibly, there is also

C) changes that do change the behavior, but in a way unrelated to the issue at hand; such changes are best left out.

Sorry it took so long to react.
msg190229 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-05-28 17:42
Here is a minimal patch.
msg190230 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-05-28 17:51
Here is a pure cosmetic patch.
msg190231 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-05-28 17:54
And here is a patch which changes behavior. It adds check for validating end of table detection.
msg190405 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2013-05-31 13:49
Ok, these patches all look fine. Thanks for your effort.
msg190412 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-05-31 19:50
Thank you for review. Should we regenerate Lib/stringprep.py now?
msg190857 - (view) Author: Roundup Robot (python-dev) Date: 2013-06-09 14:13
New changeset 8f95d77443da by Serhiy Storchaka in branch '3.3':
Issue #15239: Make mkstringprep.py work again on Python 3.
http://hg.python.org/cpython/rev/8f95d77443da

New changeset 4abe61a412be by Serhiy Storchaka in branch 'default':
Issue #15239: Make mkstringprep.py work again on Python 3.
http://hg.python.org/cpython/rev/4abe61a412be
History
Date User Action Args
2013-06-09 14:21:03serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2013-06-09 14:13:10python-devsetnosy: + python-dev
messages: + msg190857
2013-05-31 19:50:59serhiy.storchakasetmessages: + msg190412
2013-05-31 13:49:48loewissetmessages: + msg190405
2013-05-28 17:54:09serhiy.storchakasetfiles: + mkstringprep_check.patch

messages: + msg190231
2013-05-28 17:51:24serhiy.storchakasetfiles: + mkstringprep_cosmetic.patch

messages: + msg190230
2013-05-28 17:42:00serhiy.storchakasetfiles: + mkstringprep_min.patch

messages: + msg190229
2013-05-28 16:16:30loewissetmessages: + msg190223
2013-05-28 13:47:55serhiy.storchakasetmessages: + msg190211
2013-01-09 16:19:04georg.brandlsetmessages: + msg179470
2013-01-09 16:07:11serhiy.storchakasetnosy: + georg.brandl
messages: + msg179465
2013-01-02 17:30:06serhiy.storchakasetassignee: serhiy.storchaka
2012-12-27 20:22:31serhiy.storchakasetmessages: + msg178319
2012-08-11 10:02:39Arfreversetnosy: + Arfrever
2012-08-11 09:09:36georg.brandlsetversions: + Python 3.4, - Python 3.2, Python 3.3
2012-08-11 04:18:05loewissetmessages: + msg167931
2012-08-10 21:17:50serhiy.storchakasetmessages: + msg167916
2012-08-05 21:30:12loewissetmessages: + msg167514
2012-08-05 10:37:03serhiy.storchakasetkeywords: + patch
2012-08-05 10:35:38serhiy.storchakasetkeywords: + needs review, - patch
priority: normal -> low
type: behavior
stage: patch review
2012-07-02 10:23:57serhiy.storchakasetmessages: + msg164510
2012-07-01 17:40:48loewissetnosy: + loewis
messages: + msg164490
2012-07-01 17:37:27serhiy.storchakasetfiles: + mkstringprep.patch
keywords: + patch
2012-07-01 17:36:08serhiy.storchakacreate