This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author sdaoden
Recipients ixokai, nadeem.vawda, ned.deily, neologix, pitrou, sdaoden, skrah, vstinner
Date 2011-04-14.19:29:45
SpamBayes Score 0.0002950452
Marked as misclassified No
Message-id <20110414192937.GA58041@sherwood.local>
In-reply-to
Content
And about mmap(2):

import os,sys,time,mmap

MP0 = ((2**30)-1)
MP1 = ((2**31)-1)
MP2 = ((2**32)-1)
MPV = (2**20) * 100
SIZES = (MP0-MPV, MP0, MP0+MPV,
         MP1-MPV, MP1, MP1+MPV,
         MP2-MPV, MP2, MP2+MPV)

FILE = 'test.dat'

print('Start:', time.gmtime())
for i in SIZES:
    print('Testing file size ', i, ': ', sep='', end='')
    sys.stdout.flush()
    with open(FILE, "wb+") as f:
        f.seek(i)
        f.write(b'asdf')
        f.flush()
        sb = os.stat(FILE) 
        if sb.st_size != i+4:
            print('size failure:', sb.st_size, ' != ', i, sep='', end='')
            sys.stdout.flush()
        mem = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)
        if mem[0] != ord('\0'):
            print('offset 0 failed: ', ord(mem[0]), ' ', end='', sep='')
        else:
            print('offset 0 ok ', end='', sep='')
        sys.stdout.flush()
        if mem[i] != ord('a'):
            print('offset i failed: ', ord(mem[i]), ' ', end='', sep='')
        else:
            print('offset i ok ', end='', sep='')
        print()
        sys.stdout.flush()
    os.unlink(FILE)
print('End:', time.gmtime())

...

Start: time.struct_time(tm_year=2011, tm_mon=4, tm_mday=14, tm_hour=17, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=104, tm_isdst=0)
Testing file size 968884223: offset 0 ok offset i ok 
Testing file size 1073741823: offset 0 ok offset i ok 
Testing file size 1178599423: offset 0 ok offset i ok 
Testing file size 2042626047: offset 0 ok offset i ok 
Testing file size 2147483647: offset 0 ok offset i ok 
Testing file size 2252341247: offset 0 ok offset i ok 
Testing file size 4190109695: offset 0 ok offset i ok 
Testing file size 4294967295: offset 0 ok offset i ok 
Testing file size 4399824895: offset 0 ok offset i ok 
End: time.struct_time(tm_year=2011, tm_mon=4, tm_mday=14, tm_hour=17, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=104, tm_isdst=0)

Now i think that can't be any faster.
Changing to
    MP0 = ((2**30)-0)
    MP1 = ((2**31)-0)
    MP2 = ((2**32)-0)
results in

Start: time.struct_time(tm_year=2011, tm_mon=4, tm_mday=14, tm_hour=17, tm_min=27, tm_sec=55, tm_wday=3, tm_yday=104, tm_isdst=0)
Testing file size 968884224: offset 0 ok offset i ok 
Testing file size 1073741824: offset 0 ok offset i ok 
Testing file size 1178599424: offset 0 ok offset i ok 
Testing file size 2042626048: offset 0 ok offset i ok 
Testing file size 2147483648: offset 0 ok offset i ok 
Testing file size 2252341248: offset 0 ok offset i ok 
Testing file size 4190109696: offset 0 ok offset i ok 
Testing file size 4294967296: <- EOF here

Manually adjusted SIZES:

Testing file size 4294967295: offset 0 ok offset i ok 
Testing file size 4296015872: offset 0 ok offset i ok (MP2+1024*1024)
Testing file size 4295491584: offset 0 ok offset i ok (MP2+1024*512)
Testing file size 4295229440: offset 0 ok offset i ok (MP2+1024*256)
...
Testing file size 4294971392: offset 0 ok offset i ok (MP2+1024*4)
Testing file size 4294969344: <- EOF here (MP2+1024*2)

Pagesize = 4096.
I think the state machine can be easier than i thought.
History
Date User Action Args
2011-04-14 19:29:47sdaodensetrecipients: + sdaoden, ixokai, pitrou, vstinner, nadeem.vawda, ned.deily, skrah, neologix
2011-04-14 19:29:46sdaodenlinkissue11277 messages
2011-04-14 19:29:45sdaodencreate