classification
Title: Comparison of memoryview
Type: enhancement Stage: needs patch
Components: Interpreter Core Versions: Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: fin.swimmer, james stone, josh.r, pitrou, remi.lapeyre
Priority: normal Keywords:

Created on 2014-01-26 20:37 by fin.swimmer, last changed 2019-04-02 19:16 by james stone.

Messages (9)
msg209351 - (view) Author: fin swimmer (fin.swimmer) Date: 2014-01-26 20:37
Comparison by using memoryview seems not to work completely.

This works:
>>> memoryview(bytearray(range(5))) != memoryview(bytearray(range(5)))
False
>>> memoryview(bytearray(range(5))) == memoryview(bytearray(range(5)))
True

But:
>>> memoryview(bytearray(range(5))) < memoryview(bytearray(range(5)))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unorderable types: memoryview() < memoryview()

So memoryview cannot be used as a key-value for sorting
msg209373 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-01-27 01:26
This sounds reasonable, at least when the two memoryviews have the same shape.
msg209421 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-01-27 09:27
Some discussion here https://groups.google.com/forum/#!topic/dev-python/1D_iExlsva8
msg209449 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2014-01-27 15:46
For integer sequences I think non-equality comparisons will be somewhat
confusing. Are the sequences little or big endian?

If we start to view bytes more like latin-1 again (PEP 461), it would
make sense for the 'B' and 'c' formats.
msg333985 - (view) Author: Josh Rosenberg (josh.r) * (Python triager) Date: 2019-01-18 15:59
The lack of support for the rich comparison operators on even the most basic memoryviews (e.g. 'B' format) means that memoryview is still a regression from some of the functionality buffer offered back in Python 2 ( https://stackoverflow.com/a/13574862/364696 ); you either need to convert back to bytes (losing the zero-copy behavior) or hand-write a comparator of your own to allow short-circuiting (which thanks to sort not taking cmp functions anymore, means you need to write it, then wrap it with functools.cmp_to_key if you're sorting, not just comparing individual items).

While I'll acknowledge it gets ugly to try to support every conceivable format, it seems like, at the very least, we could provide the same functionality as buffer for 1D contiguous memoryviews in the 'B' and 'c' formats (both of which should be handleable by a simple memcmp).
msg333987 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2019-01-18 16:20
Josh, could you say what your use case is?
msg334009 - (view) Author: Josh Rosenberg (josh.r) * (Python triager) Date: 2019-01-18 21:01
Not my use case specifically, but my link in the last post (repeated below) was to a StackOverflow answer to a problem where using buffer was simple and fast, but memoryview required annoying workarounds. Admittedly, in most cases it's people wanting to do this with strings, so in Python 3 it only actually works if you convert to bytes first (possibly wrapped in a memoryview cast to a larger width if you need to support ordinals outside the latin-1 range). But it seems a valid use case.

Examples where rich comparisons were needed include:

Effcient way to find longest duplicate string for Python (From Programming Pearls) - https://stackoverflow.com/a/13574862/364696 (which provides a side-by-side comparison of code using buffer and memoryview, and memoryview lost, badly)

strcmp for python or how to sort substrings efficiently (without copy) when building a suffix array - https://stackoverflow.com/q/2282579/364696 (a case where they needed to sort based on potentially huge suffixes of huge strings, and didn't want to end up copying all of them)
msg334013 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2019-01-18 21:29
If not wanting to support the whole gamut of PEP 3118 types, one could start by only providing ordered comparisons when format == 'B'.
msg339363 - (view) Author: james stone (james stone) Date: 2019-04-02 19:16
I encountered this issue as well when using python 3.6.7 and psycopg2. Postgres tries to sort rows by the primary key field and if the returned type is a memoryview an the error is thrown: "TypeError: '<' not supported between instances of 'memoryview' and 'memoryview'." It may be more on the postgres guys to use a proper type with full comparison support. But I wanted to mention it here as a datapoint for a valid use case in the wild. More details on the specific issue at https://github.com/modoboa/modoboa-amavis/issues/115
History
Date User Action Args
2019-04-02 19:16:07james stonesetnosy: + james stone
messages: + msg339363
2019-01-18 21:29:45pitrousetstage: needs patch
messages: + msg334013
versions: + Python 3.8, - Python 3.5
2019-01-18 21:01:17josh.rsetmessages: + msg334009
2019-01-18 16:20:43pitrousetmessages: + msg333987
2019-01-18 16:07:18remi.lapeyresetnosy: + remi.lapeyre
2019-01-18 15:59:04josh.rsetnosy: + josh.r
messages: + msg333985
2014-10-14 16:16:12skrahsetnosy: - skrah
2014-02-03 15:35:56BreamoreBoysetnosy: - BreamoreBoy
2014-01-27 15:46:56skrahsetmessages: + msg209449
2014-01-27 09:27:59BreamoreBoysetnosy: + BreamoreBoy
messages: + msg209421
2014-01-27 01:26:21pitrousetnosy: + skrah, pitrou
messages: + msg209373

components: + Interpreter Core, - Library (Lib)
type: behavior -> enhancement
2014-01-26 20:37:37fin.swimmercreate