This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author dalke
Recipients dalke, docs@python
Date 2017-05-23.12:52:21
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1495543941.79.0.954903712475.issue30440@psf.upfronthosting.co.za>
In-reply-to
Content
The peephole optimizer is an overall benefit to Python but it has some side-effects that occasionally cause problems. These are well-known in the issue tracker, but there is no other documentation which will help a Python programmer figure out which constructs may be a problem.

1) it will compute large integer constants and save them in the .pyc file. The following results in a 19M .pyc file. 

def compute_largest_known_prime():
  return 2**74207281 - 1

As an example of someone affected by the compile-time evaluation, see https://stackoverflow.com/questions/34113609/why-does-python-preemptively-hang-when-trying-to-calculate-a-very-large-number/ . Note the many people who struggled to find definitive documentation.

2) it will create and discard large strings. Consider this variant of the code in test_zlib.py, where I have replaced the imported module variable "_1G" with its value:

    @bigmemtest(size=_4G + 4, memuse=1, dry_run=False)
    def test_big_buffer(self, size):
        data = b"nyan" * (2**30 + 1)  # the original uses "_1G"
        self.assertEqual(zlib.crc32(data), 1044521549)
        self.assertEqual(zlib.adler32(data), 2256789997)

The byte compiler will create the ~4GB string then discard it, even though the function will not be called on machines with insufficient RAM.

As an example of how I was affected by this, see #21074 .

3) The optimizer affects control flow such that the coverage.py gives false positives about unreached code.

As examples of how people are affected, see #2506 and https://bitbucket.org/ned/coveragepy/issues/198/continue-marked-as-not-covered . The last item on the coverage.py tracker asks "Is this limitation documented anywhere?"

I do not believe that the current peephole optimizer should be changed to support these use cases, only that there should be documentation about how the optimizer may negatively affect real-world code.

The domain expert on this topic is Raymond Hettinger. He does not feel safe in issues where I am involved. As I believe my continued presence on this issue will inhibit the documentation changes which I think are needed, I will remove my name from this issue and not be further involved.
History
Date User Action Args
2017-05-23 12:52:21dalkesetrecipients: + dalke, docs@python
2017-05-23 12:52:21dalkesetmessageid: <1495543941.79.0.954903712475.issue30440@psf.upfronthosting.co.za>
2017-05-23 12:52:21dalkelinkissue30440 messages
2017-05-23 12:52:21dalkecreate