Author jonathan.underwood
Recipients asottile, benjamin.peterson, brett.cannon, bukzor, geoffreyspear, georg.brandl, jonathan.underwood, ncoghlan, pitrou, serhiy.storchaka
Date 2018-06-02.16:37:12
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1527957432.21.0.592728768989.issue24085@psf.upfronthosting.co.za>
In-reply-to
Content
Seeing a very similar problem - very high memory useage during byte compilation.

Consider the very simple code in a file:

```
def test_huge():
    try:
        huge = b'\0' * 0x100000000  # this allocates 4GB of memory!
    except MemoryError:
        print('OOM')
```

Running this sequence of commands shows that during byte compilation, 4 GB memory is used. Presumably this is because of the `huge` object - note of course the function isn't actually executed.

```
valgrind --tool=massif python memdemo.py
ms_print massif.out.7591 | less
```

You'll need to replace 7591 with whatever process number valgrind reports.

Is there any hope of fixing this? It's currently a problem for me when running tests on Travis, where the memory limit is 3GB. I had hoped to use a conditional like the above to skip tests that would require more memory than is available. However, the testing is killed before that simply because the byte compilation is causing an OOM.
History
Date User Action Args
2018-06-02 16:37:12jonathan.underwoodsetrecipients: + jonathan.underwood, brett.cannon, georg.brandl, ncoghlan, pitrou, benjamin.peterson, bukzor, geoffreyspear, serhiy.storchaka, asottile
2018-06-02 16:37:12jonathan.underwoodsetmessageid: <1527957432.21.0.592728768989.issue24085@psf.upfronthosting.co.za>
2018-06-02 16:37:12jonathan.underwoodlinkissue24085 messages
2018-06-02 16:37:12jonathan.underwoodcreate