First, a minor issue about class signatures:
doc: FileCookieJar(filename, delayload=None, policy=None)
code: def __init__(self, filename=None, delayload=False, policy=None)
Pretty clearly, doc should be changed to match code, as later code allow for possibility of filename = None (meaning that that is intentional).

Ditto for doc for FileCookieJar subclasses (which inherit __init__):
MozillaCookieJar(filename, delayload=None, policy=None) 
LWPCookieJar(filename, delayload=None, policy=None)

--- 
FileCookieJar has .load which in inherited by the subclasses. It checks for a filename and opens it and then calls ._really_load. The two subclasses have customized ._really_load methods that correspond to their customized .save methods.

FileCookieJar itself does not. If it did, it would have to somehow be 'generic'. This suggests to me that FileCookieJar was not intended to be directly used. This impression is reinforced by the definition of .save().

    def save(self, filename=None, ignore_discard=False, ignore_expires=False):
        """Save cookies to a file."""
        raise NotImplementedError()

In other words, there is no generic format to save to *or* load from.
There should be a corresponding ._really_load to raise the same exception.

Bottom line: as best I understand, your code is not intended to work, but both the doc and implementation are deficient in not saying so, and both should be improved.