Title: Imports at top of module is often not used
Type: performance Stage: resolved
Components: Library (Lib) Versions:
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: brett.cannon, christian.heimes, eric.smith, plokmijnuhby
Priority: normal Keywords:

Created on 2019-09-27 18:16 by plokmijnuhby, last changed 2019-10-01 09:16 by christian.heimes. This issue is now closed.

Messages (6)
msg353385 - (view) Author: Dominic Littlewood (plokmijnuhby) * Date: 2019-09-27 18:16
In PEP 8, it is stated that:
"Imports are always put at the top of the file, just after any module comments and docstrings, and before module globals and constants."

Note the word "always". This advice makes sense because it cuts down on performance costs.

I recently got into an argument about this, and so I created a script to find all the times in the standard library that an import statement was used inside a function or method. I was expecting to get one or two, but it's safe to say that the answer 1576 was a little surprising. (I was on 3.7.4, if anyone wants to check.)

It turns out that more than one in five modules (defined as any .py file) contain at least one violation of this rule. In the standard library, there are more violations of the rule than there are while loops. Do we need to either a) fix large parts of the standard library or b) rewrite PEP 8 to say that this might not actually be all that bad?
msg353387 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2019-09-27 18:33
I think PEP 8 should say "Module-level imports are always put at the top of the file, ...".
msg353442 - (view) Author: Dominic Littlewood (plokmijnuhby) * Date: 2019-09-28 09:19
I've slightly adjusted the script to check for module-level imports not at the top of the file. If we permit things like this:

if condition:
    import module

as long as they are at the top of the file, it seems that only one in ten modules have a problem. Which is better but not great.

A common pattern which breaks the rules is this:

# Large amounts of code here

if __name__ == '__main__':
    import unittest

although that by no means accounts for all of the problems.
msg353594 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2019-09-30 16:44
So it's a bit misleading to use the stdlib as a guideline because there are reasons we do this in certain spots. E.g. the os module purposefully does function-level imports to make startup cheaper. importlib does this to minimize how many modules need to be frozen into the interpreter.

But regardless of why, the PEP 8 guidance is still accurate as well as wholesale changes aren't allowed for stylistic reasons ("A Foolish Consistency is the Hobgoblin of Little Minds"). So while I appreciate the analysis and find it interesting, I'm closing this issue as there's nothing specific to do here.
msg353653 - (view) Author: Dominic Littlewood (plokmijnuhby) * Date: 2019-10-01 09:03
With all due respect, I disagree with what you're saying here.

"It's a bit misleading to use the stdlib as a guideline"
... when looking at the style guide for the stdlib? I appreciate that PEP 8 is used for other purposes, but its main purpose is clear from its very first line.

"There are reasons we do this in certain spots"
Yes, there are. And given that there are so many of these different spots where it would give better performance to ignore the rule, PEP 8 should reflect that.
msg353654 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2019-10-01 09:16
A considerable amount of code is a lot older than PEP 8, some even over a decade older. Even newer code in the stdlib does not conform to PEP 8 for multiple reasons, e.g. external modules was incorporated into the stdlib.

PEP 8 sections "Introduction" and "A Foolish Consistency is the Hobgoblin of Little Minds" explain why parts stdlib do not follow PEP 8.
Date User Action Args
2019-10-01 09:16:22christian.heimessetnosy: + christian.heimes
messages: + msg353654
2019-10-01 09:03:08plokmijnuhbysetmessages: + msg353653
2019-09-30 16:44:59brett.cannonsetstatus: open -> closed

nosy: + brett.cannon
messages: + msg353594

resolution: not a bug
stage: resolved
2019-09-28 09:19:26plokmijnuhbysetmessages: + msg353442
2019-09-27 18:33:44eric.smithsetnosy: + eric.smith
messages: + msg353387
2019-09-27 18:16:34plokmijnuhbycreate