Author Greg Price
Recipients Greg Price, benjamin.peterson, ezio.melotti, mcepl, serhiy.storchaka, vstinner
Date 2019-08-13.20:20:56
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1565727657.32.0.799965039323.issue32771@roundup.psfhosted.org>
In-reply-to
Content
> Loading it dynamically reduces the memory footprint.

Ah, this is a good question to ask!

First, FWIW on my Debian buster desktop I get a smaller figure for `import unicodedata`: only 64 kiB.

$ python
Python 3.7.3 (default, Apr  3 2019, 05:39:12) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.system(f"grep ^VmRSS /proc/{os.getpid()}/status")
VmRSS:	    9888 kB

>>> import unicodedata
>>> os.system(f"grep ^VmRSS /proc/{os.getpid()}/status")
VmRSS:	    9952 kB

But whether 64 kiB or 160 kiB, it's much smaller than the 1.1 MiB of the whole module.  Which makes sense -- there's no need to bring the whole thing in memory when we only import it, or generally to bring into memory the parts we aren't using.  I wouldn't expect that to change materially if the tables and algorithms were built in.

Here's another experiment: suppose we load everything that ast.c needs in order to handle non-ASCII identifiers.

$ python
Python 3.7.3 (default, Apr  3 2019, 05:39:12) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.system(f"grep ^VmRSS /proc/{os.getpid()}/status")
VmRSS:	    9800 kB

>>> là = 3
>>> os.system(f"grep ^VmRSS /proc/{os.getpid()}/status")
VmRSS:	    9864 kB

So that also comes to 64 kiB.

We wouldn't want to add 64 kiB to our memory use for no reason; but I think 64 or 160 kiB is well within the range that's an acceptable cost if it gets us a significant simplification or improvement to core functionality, like Unicode.
History
Date User Action Args
2019-08-13 20:20:57Greg Pricesetrecipients: + Greg Price, vstinner, benjamin.peterson, mcepl, ezio.melotti, serhiy.storchaka
2019-08-13 20:20:57Greg Pricesetmessageid: <1565727657.32.0.799965039323.issue32771@roundup.psfhosted.org>
2019-08-13 20:20:57Greg Pricelinkissue32771 messages
2019-08-13 20:20:56Greg Pricecreate