I do not think your wish is sensibly possible. GzipFile wraps an object that is or simulates a file. This is necessary because GzipFile "simulates most of the methods of a file object, with the exception of the readinto() and truncate() methods." Note that seek, rewind, and tell are not excluded. For instance, I see no way that this:
    def rewind(self):
        '''Return the uncompressed stream file position indicator to the
        beginning of the file'''
        if self.mode != READ:
            raise IOError("Can't rewind in write mode")
        self.fileobj.seek(0)
        ...
could be implemented without seek, and without having a complete local copy of everything read.

urllib.request.urlopen returns a 'file-like' object that does not quite fully simulate a file. The downstream OP should save the gzip file locally using urlretrieve() and *then* open and iterate through it. Feel free to forward this suggestion to the OP.