classification
Title: add identity function
Type: enhancement Stage: test needed
Components: Extension Modules Versions: Python 3.1, Python 2.7
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: ajaksu2, belopolsky, christian.heimes, collinwinter, jackdied, josiahcarlson, phr, piotr.dobrogost, rhettinger, twobitsprite
Priority: low Keywords:

Created on 2007-03-04 00:21 by phr, last changed 2013-10-04 09:31 by piotr.dobrogost. This issue is now closed.

Files
File name Uploaded Description Edit
identity.patch jackdied, 2007-03-22 20:15
Messages (24)
msg55021 - (view) Author: paul rubin (phr) Date: 2007-03-04 00:21
Requested and assigned to Raymond at his suggestion:

http://groups.google.com/group/comp.lang.python/msg/603870361743c85c

There should be an identify function identity(x)=x.

I suggest it also take and ignore an optional second arg: identity(x1,x2)=x1.  The second arg can be useful in some generator expressions:

foo = (x for x in bar if condition(x) and identity(True, memoize(x))

That allows calling memoize (or some other function) on the selected elements in the genexp, and disposing of the returned value.  It's sort of like the const function (K combinator) to go along with the identity function's I combinator.  OK, the above is not really in the functional spirit, but it's been useful.

There could conceivably be also an actual const function const(k)=partial(identity,k) but I can't remember needing that in Python code.  The two-arg identity function (uncurried version of const) is probably enough.
msg55022 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2007-03-05 14:21
1. If this proposal is accepted, it will make sense to deprecate the use of None as an identity function in map:

>>> map(None, range(10))
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

2. Some other languages have an dyadic identity function that returns the *second* argument. 

For example, K has : primitive:

  identity:(:)
  identity[1;2]
2

The rationale in K is that it is useful in an ammed function that replaces entries of an an array with a result of a dyadic function applied to the old and the supplied value and it is natural to have old value first:

  @[1 2 3;1;-;20]
1 -18 3
  @[1 2 3;1;:;20]
1 20 3

This rationale does not apply to Python, but in the absence of other reasons to choose the order of arguments, Python may as well follow the precedent. Does anyone know a less exotic language that has a dyadic identity?


msg55023 - (view) Author: Josiah Carlson (josiahcarlson) * Date: 2007-03-12 20:06
Not all x line functions should be built into Python.  Further, Python's standard syntax offers an infix operator that does the same thing (though in slightly different order as described below, you can reorder with minimal effort).

identity(X, Y) -> (Y and False) or X

Also, the only use-case that you are provided and that I can imagine, are examples like you provide where one is changing state within a statement (if, elif, while, etc.) or expression (generator, list comprehension, conditional, etc.).
msg55024 - (view) Author: Collin Winter (collinwinter) * (Python committer) Date: 2007-03-19 19:05
I can see adding the 1-argument form to operator or functools (as it's useful in functional programming), but the 2-argument form you've suggested is right out. If you really feel the need to torture a "for" loop into a genexp/listcomp like that,

foo = (x for x in bar if condition(x) and [memoize(x)])

does the same thing using today's capabilities.
msg55025 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2007-03-19 19:57
I have just realized that the requested functionality is known in C as the comma operator.  I often find myself writing "return Py_INCREF(o),o;" in my C code, but I cannot really defend that idiom against "Py_INCREF(o); return o;" alternative.  My personal reason is entirely C-specific, if  followed an if(), the first form does not require curly braces.

In any case, comma operator can be emulated in python as

exp1,expr2,expr3   ->  (exp1,expr2,expr3)[-1]


Since multi-argument "identity" is likely to be rejected, my proposal to alter the order of arguments is moot.  My other suggestion that with identity, map(None, ..) should be deprecated in favor of map(identity, ..) is probably an arument against the identity proposal now.  
msg55026 - (view) Author: Memotype (twobitsprite) Date: 2007-03-22 13:06
I also would like to have a built-in identity function (in fact, I found this by googling "python identity function"). My use-case is a bit different. I ofter find myself wanting to simply specify a function for be used in a loop, something like:

def f(items):
    if something:
        wrapper = int
    else:
        wrapper = identity

    for item in items:
        yield wrapper(item)

of course, usually it's a bit more complex than that, but you get the idea... and I supposed its actually more like the previous use-case than I thought.

I realize I could just use "lambda x: x", but I feel that comes with an unnecessary performance impact for something so trivial. I don't know how much python does to compile built-in functions, but I imagine that the identity function can be mostly optimized out at compile time if it were built-in.

Just my two-cents.
msg55027 - (view) Author: Josiah Carlson (josiahcarlson) * Date: 2007-03-22 17:49
twobitsprite: your use-case is different from that of others.  While you could use an identity function for your purposes, a lambda would work just fine.  Regardless, there is call overhead, which can only be reduced by not performing a call at all.

In terms of an identity function versus tuple creation and indexing as per belopolsky's suggestion...

>>> timeit.Timer("x(1+2,3+4)", "x = lambda *args:args[-1]").timeit()
0.99381552025397468
>>> timeit.Timer("(1+2,3+4)[-1]", "").timeit()
0.49153579156927663

Tuple is faster.  Just use a tuple.
msg55028 - (view) Author: Memotype (twobitsprite) Date: 2007-03-22 19:06
It still feels like an ugly hack for something so simple. I can't believe there is such resistance for something as trivial as a function which returns it's first argument.
msg55029 - (view) Author: Memotype (twobitsprite) Date: 2007-03-22 19:25
Also, map like functions start to look a bit ugly (of course, this is overly simplistic, but serves as an example):

def my_map(func, items):
    acc = []
    for item in items:
        if func == None:
            acc.append(item)
        else:
            acc.append(func(item))
    return acc

which has the obvious overhead of a test for every iteration. Of course, you could just make two versions of the for loop, one which applies <func> and one which doesn't, but that immetiately violates once-and-only-once and also forces the function to be overly concerned with what it is passed.

this is _much_ cleaner:

def my_map2(func, items):
    return [func(i) for i in items]

and the user of the function can simply pass <identity> as <func> and this keeps the details of what to do with the items outside of the function.

Without identity, whenever I want to allow the user to inject code to modify data, I have to account for the fact that they might want to do nothing with it, and so I have to test for None constantly. This is simply bad practice and, IMHO, violates everything python stands for, which is elegance, simplicity and "batteries included", right?

If you want I patch, I can try to provide one; however I have very minimal knowledge of how the vm works, and I've only looked at the code a couple of times. In fact, I might even go ahead and do that, it can't be that difficult.
msg55030 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2007-03-22 19:32
> I can't believe there is such resistance for something
> as trivial as a function which returns it's first argument.

The resistance is precisely because it is so trivial. As  josiahcarlson suggested before, "Not all x line functions should be built into Python."  Compare the following two alternatives:

def identity(*args): return args[0]

and

from functools import identity

The first option is only slightly longer than the second, but it is actually much clearer.  With the  second option, it is not obvious that identity takes an arbitrary number of arguments and if it does, which argument it will return.

The only advantage is that if coded in C, functools.identity may be slightly faster. However, given that 

>>> dis(lambda x: x)
  1           0 LOAD_FAST                0 (x)
              3 RETURN_VALUE

I am not sure a C coded identity can significantly beat it.  The real savings would come if python compiler could optimize away calls to identity altogether, but nobody has explained how that could be done or why it would be easier to optimize away identity that lambda x:x.

msg55031 - (view) Author: Memotype (twobitsprite) Date: 2007-03-22 19:37
Well, I'm suggesting that it be in __builtins__, but whatever... If I had it my way I'd program in Haskell every chance I got anyways, I only use python because my boss/co-workers prefer it.
msg55032 - (view) Author: Memotype (twobitsprite) Date: 2007-03-22 20:06
--- bltinmodule-orig.c  2007-03-22 16:00:21.452245559 -0400
+++ bltinmodule.c       2007-03-22 15:56:19.353115310 -0400
@@ -69,6 +69,17 @@
        return PyNumber_Absolute(v);
 }

+static PyObject *
+builtin_identity(PyObject *self, PyObject *v)
+{
+       return v;
+}
+
+PyDoc_STRVAR(identity_doc,
+"identity(x) -> x\n\
+\n\
+The identity function. Simply returns is first argument.");
+
 PyDoc_STRVAR(abs_doc,
 "abs(number) -> number\n\
 \n\
@@ -2281,6 +2292,7 @@
 #endif
        {"vars",        builtin_vars,       METH_VARARGS, vars_doc},
        {"zip",         builtin_zip,        METH_VARARGS, zip_doc},
+       {"identity",    builtin_identity,   METH_O, identity_doc},
        {NULL,          NULL},
 };

msg55033 - (view) Author: Josiah Carlson (josiahcarlson) * Date: 2007-03-22 20:06
> Well, I'm suggesting that it be in __builtins__, but whatever...

You apparently didn't get the memo; map, filter, reduce, etc., are all going to be placed into functools and removed from __builtins__.  Adding identity to __builtins__ is not going to happen.

Further, your preferred programming language not being Python is not topical to the discussion.  If you want Haskell, and your boss isn't letting you use it, please don't complain here.
msg55034 - (view) Author: Josiah Carlson (josiahcarlson) * Date: 2007-03-22 20:13
Your patch isn't what was asked for by other users.  The Python equivalent to what was asked for was:

identity = lambda arg0, *args: arg0

Regardless, being that it is a functional language construct, it's not going to make it into __builtins__.  Never mind that adding to __builtins__ has a higher requirement than in other portions of the standard library.
msg55035 - (view) Author: Jack Diederich (jackdied) * (Python committer) Date: 2007-03-22 20:15
Raymond, please make a pronouncement.

+1 on adding it to the operator module.  Lots of one liners go there.
-1000 to having it take more than one argument.  If I saw this
  identity(ob, 42)
for the first time (or second, or ...) I would have to look up what
it does.  As a plain identity function it is obvious.

Patch to operator.c, liboperator.tex, test_operator.py, NEWS attached.
Raymond, there isn't a good section in the doc for this function so I added it to the logical operators.  The fact that there isn't a good section for it might be a mark against it being included.

It is a couple times faster than the python version which isn't exciting.

sprat:~/src/python-rw> ./python Lib/timeit.py -s "from operator import identity" "identity(None)"
10000000 loops, best of 3: 0.161 usec per loop
sprat:~/src/python-rw> ./python Lib/timeit.py -s "def identity(ob): return ob" "identity(None)"
1000000 loops, best of 3: 0.421 usec per loop

File Added: identity.patch
msg55036 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2007-03-22 20:18
Raising "Haskel is better than Python" argument is unlikely to win you friends here.  Neither will submitting a patch with a reference count bug :-).
msg55037 - (view) Author: Memotype (twobitsprite) Date: 2007-03-22 20:31
josiah,

My appologies for not being hip to the latest python dev news. I would have no problem with putting the function in a module like functools/itertools/operator/whatever, I just thought something like that might belong with map/filter/etc... so, if that's where they're going, I can just from functools import * and go on my merry way. I was just responding to your argument that defining it yourself would be just as easy as importing from a module.

+1 on Raymond's patch (not that I expect my vote to count much, being some random guy :P) execpt for it going into operator, being as map/etc are going somewhere else... either way, I think it's silly to mobe map/etc out of builtins, but hey... what am I gonna do about it? :P
msg55038 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2007-03-22 20:34
I need to clarify that my comment about a buggy patch was responding to twobitsprite, not to jackdied whose post I noticed only after I hit submit.  Jackdied's patch is correct, but my concern is that with identity in operators, functional programmers will start passing identity instead of None as func to map(func, ..) that will result in much slower code.
msg55039 - (view) Author: Jack Diederich (jackdied) * (Python committer) Date: 2007-03-22 20:57
Let's put this thread on hold until we get a pronouncement.  I don't think there is anything additional to be said.

Alexander, don't worry, I didn't think you were talking about my patch ;)
and don't worry about people writing sub-optimal code.  If you start doing that you'll never get any sleep.
msg55040 - (view) Author: Memotype (twobitsprite) Date: 2007-03-22 21:26
> Neither will submitting a patch with a reference count bug :-).

Bleh, generational garbage collectors are where its at anyways. ;P
msg55041 - (view) Author: Collin Winter (collinwinter) * (Python committer) Date: 2007-03-24 06:01
-1 on adding the identity function anywhere in the stdlib. This isn't just a 1-line function, it's only 10 characters: "lambda x:x".
msg59366 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-01-06 12:49
What's the conclusion of your discussion? Do you consent to close the
feature request?
msg63041 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-02-26 13:45
Raymond Hettinger wrote in msg63027 (issue2186):

"""
.. it looks like there [is an] agreement on dropping None for map() and 
going forward with the operator.identity() patch.  Will check these in 
in the next couple of days.
"""

This leaves open the issue of multi-argument identity.  I would argue 
that def identity(*args): return args[-1] # or args[0] will conflict 
with the use of identity instead of None in map: map(identity, x, y, ..) 
will silently produce results different from current map(None, x, y, 
..).  It is possible to make identity behave exactly like None in map by 
defining it  as

def identity(*args):
   if len(args) == 1:
      return args[0]
   else:
      return args
 
While there is certain cuteness in making identity(...) equivalent to 
(...), it is a bit too cute for my taste.  I am -1 on multi-argument 
identity.

Also placement of identity in operator while map is in builtin, may 
complicate implementation of passthrough map optimization.  While it is 
possible to avoid importing operator in builtin by implementing identity  
in builtin but exporting it only from operator, a more straightforward 
solution would be to keep map and identity in the same module.  This 
logic may be a slippery slope, however, because optimizing filterfalse, 
would suggest that operator.not_ should be moved in the same module as 
the ultimate location for filterfalse. I am +1 on implementing 
optimization for map(identity, ..), +0 on that for filterfalse(not_, 
..), 0 on the location of identity. (BTW, maybe we should implement 
optimization for filter(not_, ..) and drop filterfalse altogether.)
msg84661 - (view) Author: Daniel Diniz (ajaksu2) Date: 2009-03-30 21:42
I suggest closing per
http://mail.python.org/pipermail/python-ideas/2009-March/003647.html
History
Date User Action Args
2013-10-04 09:31:35piotr.dobrogostsetnosy: + piotr.dobrogost
2009-04-01 21:10:56rhettingersetstatus: open -> closed
resolution: rejected
2009-03-30 21:42:24ajaksu2setpriority: normal -> low

components: + Extension Modules, - None
versions: + Python 3.1, Python 2.7
nosy: + ajaksu2

messages: + msg84661
stage: test needed
2008-02-26 13:45:21belopolskysetmessages: + msg63041
2008-01-06 12:49:20christian.heimessetnosy: + christian.heimes
messages: + msg59366
2007-03-04 00:21:15phrcreate