The last time we applied the LRU cache to the re.py module, the overhead of the pure python version resulted in a net performance decrease. But now we have a highly performance C version and should consider reinstating the code.

Since that time the logic of re._compile() was changed. Now it can't just be wrapped with lru_cache().

Added comments on Rietveld.

lru_cache can be used for re._compile() if add the ability to bypass the cache and to validate cached value.

Yes, I saw that. If a function could raise a NoCache exception, re._compile() could take advantage of it. But I don't feel good about going down that path (adding coupling between the caching decorator and the cached function). It would be better to keep the lru_cache API simple. I already made the mistake of expanding the API for typed=True just to accommodate a single use case (re.compile).

Yes, raising an exception with a result as a payload is one option. Other option is to check a result. Something like:

def _compile_is_valid(value):
    p, loc = value
    return loc is None or loc == _locale.setlocale(_locale.LC_CTYPE)

def _compile_cache_if(value):
    p, loc = value
    return loc is not False

@lru_cache(_MAXCACHE, is_valid=_compile_is_valid, cache_if=_compile_cache_if)
def _compile1(pattern, flags):
    # internal: compile pattern
    if isinstance(pattern, _pattern_type):
        if flags:
            raise ValueError(
                "cannot process flags argument with a compiled pattern")
        return pattern, False
    if not sre_compile.isstring(pattern):
        raise TypeError("first argument must be string or compiled pattern")
    p = sre_compile.compile(pattern, flags)
    if flags & DEBUG:
        return p, False
    if not (p.flags & LOCALE):
        return p, None
    if not _locale:
        return p, False
    return p, _locale.setlocale(_locale.LC_CTYPE)

def _compile(pattern, flags):
    p, loc = _compile1(pattern, flags)
    return p

I think I'll just take the low hanging fruit in _compile_repl and call it a day.

New changeset 88110cfbf4dc by Raymond Hettinger in branch '3.6':
Issue bpo-28193: Use lru_cache in the re module.
https://hg.python.org/cpython/rev/88110cfbf4dc