Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mmap enhancement - resize with sequence notation #50138

Closed
bmearns mannequin opened this issue Apr 30, 2009 · 4 comments
Closed

mmap enhancement - resize with sequence notation #50138

bmearns mannequin opened this issue Apr 30, 2009 · 4 comments
Labels
stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@bmearns
Copy link
Mannequin

bmearns mannequin commented Apr 30, 2009

BPO 5888
Nosy @MojoVampire, @iritkatriel

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2022-03-19.21:43:39.786>
created_at = <Date 2009-04-30.18:34:09.945>
labels = ['type-feature', 'library']
title = 'mmap enhancement - resize with sequence notation'
updated_at = <Date 2022-03-19.21:43:39.785>
user = 'https://bugs.python.org/bmearns'

bugs.python.org fields:

activity = <Date 2022-03-19.21:43:39.785>
actor = 'iritkatriel'
assignee = 'none'
closed = True
closed_date = <Date 2022-03-19.21:43:39.786>
closer = 'iritkatriel'
components = ['Library (Lib)']
creation = <Date 2009-04-30.18:34:09.945>
creator = 'bmearns'
dependencies = []
files = []
hgrepos = []
issue_num = 5888
keywords = []
message_count = 4.0
messages = ['86849', '220767', '220771', '415575']
nosy_count = 4.0
nosy_names = ['bmearns', 'neologix', 'josh.r', 'iritkatriel']
pr_nums = []
priority = 'normal'
resolution = 'rejected'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue5888'
versions = ['Python 2.7', 'Python 3.5']

@bmearns
Copy link
Mannequin Author

bmearns mannequin commented Apr 30, 2009

I thought it would be nice if mmaps could generally look a little more
like sequences. Specifically, being able to resize+write using
square-bracket notation as with lists:

>>> x = [1,2,3,4,5]
>>> x
[1, 2, 3, 4, 5]
>>> x[2:2] = [6,7,8,9]
>>> x
[1, 2, 6, 7, 8, 9, 3, 4, 5]
>>>

If that could be done when x is an mmap.mmap, it'd be great.
alternatively, if mmap had an insert or an extend method that work like
with lists, the same behavior could be achieved without relying on mmap
specific method-names.

@bmearns bmearns mannequin added the type-feature A feature request or enhancement label Apr 30, 2009
@terryjreedy terryjreedy added the stdlib Python modules in the Lib dir label Jul 10, 2010
@BreamoreBoy
Copy link
Mannequin

BreamoreBoy mannequin commented Jun 16, 2014

@brian this will go nowhere without a patch covering code, tests and documentation, are you interested in providing one?

@MojoVampire
Copy link
Mannequin

MojoVampire mannequin commented Jun 16, 2014

I see a few issues with this:

  1. Changing the default behavior is a compatibility issue. I've written code that depends on exceptions being raised if slice assignment sizes don't match.
  2. The performance cost is high; changing from rewriting in place to shrinking or expanding slice assignment requires (in different orders for shrink/expand) truncating the file to the correct length, memcpy-ing data proportionate to the data after the end of the slice (not proportionate to the slice size) and probably remapping the file (which causes problems if someone has a buffer attached to the existing mapping). At least with non-file backed sequences, when we do work like this it's all in memory and typically smallish; with a file, most of it has to be read from and written to disk, and I'd assume the data being worked with is "largish" (if it's reliably small, the advantages of mmap-ing are small).
  3. Behavior in cases where the whole file isn't mapped is hard to intuit or define reasonably. If I map the first 1024 bytes of a 2 GB file, and I add 20 bytes in the middle of the block, what happens? Does data from the unmapped portions get moved? Overwritten? What about removing 20 bytes from the middle of the block? Do we write 0s, or copy down the data that appears after? And remember, for all but the "shrink and write 0s" option, we're moving or modifying data the user explicitly didn't mmap.

@MojoVampire MojoVampire mannequin changed the title mmap ehancement - resize with sequence notation mmap enhancement - resize with sequence notation Jun 24, 2014
@iritkatriel
Copy link
Member

I am closing this because in the 13 years since it was opened there was only one (negative) response and I don't think this will be picked up now.

If you still want to pursue this idea, I would suggest raising it on python-ideas to get more feedback, and then possibly reopening this or creating a new issue.

@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stdlib Python modules in the Lib dir type-feature A feature request or enhancement
Projects
None yet
Development

No branches or pull requests

2 participants