classification
Title: sqlite3.Row doesn't support sequence protocol
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.5, Python 3.4, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: BreamoreBoy, Claudiu.Popa, eric.araujo, ghaering, jesstess, lemburg, pfalcon, pitrou, python-dev, serhiy.storchaka, vajrasky
Priority: normal Keywords: needs review, patch

Created on 2010-10-26 19:11 by pfalcon, last changed 2014-05-28 10:06 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
sqlite1.patch Claudiu.Popa, 2013-11-30 09:06 review
sqlite2.patch Claudiu.Popa, 2013-12-04 07:21 review
issue10203.patch Claudiu.Popa, 2014-05-25 19:41 review
issue10203_1.patch Claudiu.Popa, 2014-05-26 20:58 review
issue10203_2.patch Claudiu.Popa, 2014-05-27 07:49 review
Messages (23)
msg119639 - (view) Author: Paul Sokolovsky (pfalcon) Date: 2010-10-26 19:11
sqlite.Row class doesn't implement sequence protocol, which is rather unfortunate, because it is described and expected to work like a tuple, with extra mapping-like functionality.

Specific issue I hit:

Adding rows to PyGTK ListStore,

        model = gtk.ListStore(*db.getSchema())
        for r in listGen():
            model.append(r)

I get:

TypeError: expecting a sequence

Looking at PyGTK sources, append() method uses PySequence Check() on the argument. Looking at Python 2.6.5 abstract.c:

int
PySequence_Check(PyObject *s)
{
        if (s && PyInstance_Check(s))
                return PyObject_HasAttrString(s, "__getitem__");
        if (PyObject_IsInstance(s, (PyObject *)&PyDict_Type))
                return 0;
        return s != NULL && s->ob_type->tp_as_sequence &&
                s->ob_type->tp_as_sequence->sq_item != NULL;
}

And sqlite3.Row doesn't set ob_type->tp_as_sequence as of Py 2.6.5 or 2.7.
msg119698 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-10-27 11:58
Hello Paul, thanks for the report.  The doc only describe Row as a tuple-like object, but tuple does implement the Sequence ABC, so I’m inclined to agree with you this is a bug and not a feature request (my first reaction).

Adding Georg, the maintainer of the module, to the nosy list (found his username in Misc/maintainers.rst).
msg119699 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-10-27 11:58
(Gerhard, sorry, not well awake :)
msg119702 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2010-10-27 12:13
Just as data point: the DB-API 2.0 requires that the row objects returned by the various .fetch*() methods are sequences, i.e. they need to implement the sequence protocol.
msg204789 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2013-11-30 09:06
Hello! Here's a simple patch which makes sqlite.Row to act like a real sequence.
msg205172 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2013-12-03 23:11
Patch looks good!  Are documentation changes needed?
msg205195 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2013-12-04 06:35
I guess not, the documentation already states that Row tries to mimic a tuple in most of its features. Probably a MISC/News entry is required.
msg205197 - (view) Author: Vajrasky Kok (vajrasky) * Date: 2013-12-04 07:12
I got warning in compiling your patch:

gcc -pthread -fPIC -Wno-unused-result -Werror=declaration-after-statement -g -O0 -Wall -Wstrict-prototypes -DMODULE_NAME="sqlite3" -DSQLITE_OMIT_LOAD_EXTENSION=1 -IModules/_sqlite -I/usr/include -I./Include -I. -IInclude -I/usr/include/x86_64-linux-gnu -I/usr/local/include -I/home/ethan/Documents/code/python/cpython3.4/Include -I/home/ethan/Documents/code/python/cpython3.4 -c /home/ethan/Documents/code/python/cpython3.4/Modules/_sqlite/row.c -o build/temp.linux-x86_64-3.4-pydebug/home/ethan/Documents/code/python/cpython3.4/Modules/_sqlite/row.o
/home/ethan/Documents/code/python/cpython3.4/Modules/_sqlite/row.c:212:28: warning: initialization from incompatible pointer type [enabled by default]
/home/ethan/Documents/code/python/cpython3.4/Modules/_sqlite/row.c:212:28: warning: (near initialization for ‘pysqlite_row_as_sequence.sq_item’) [enabled by default]

$ gcc --version
gcc (Ubuntu/Linaro 4.7.2-2ubuntu1) 4.7.2
msg205201 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2013-12-04 07:21
Thanks, Vajrasky! Here's an updated patch.
msg213865 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2014-03-17 14:01
What can be done to move this forward?
msg213868 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-03-17 14:12
I've changed the versions, hope I've got them correct.
msg213876 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2014-03-17 17:02
Thanks Mark.  There will be a last 3.3 release with bugfixes before it switches to security fixes only.
msg213877 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2014-03-17 17:04
I would like another core developer more intimate with C to review the patch.
msg213880 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2014-03-17 18:24
Ah, 3.3 won’t follow that custom given that it had a bugfix release very recently.
msg217441 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-04-28 23:08
Issue #13583 ("sqlite3.Row doesn't support slice indexes") is partly related.
msg219099 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-05-25 18:50
LGTM. Perhaps it is worth to add a test for negative indices (valid (-1) and invalid (< -length)).
msg219103 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2014-05-25 19:41
Thanks. Here's the updated patch. It supports negative indeces (my previous patch didn't do that).
msg219176 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-05-26 20:23
Looks good, but there is one detail. Whith issue10203.patch when integer index overflows C long, sqlite3.Row.__getitem__() doesn't raise an exception. Instead overflow exception is raised later.

>>> import sqlite3
>>> con = sqlite3.connect(":memory:")
>>> con.row_factory = sqlite3.Row
>>> row = con.execute("select 1 as a, 2 as b").fetchone()
>>> row[2**1000]
2
>>> 
OverflowError: Python int too large to convert to C long
msg219179 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2014-05-26 20:58
Thanks. Here's a fix.
msg219206 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-05-27 07:30
Compare with tuple:

>>> (1, 2)[2**1000]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: cannot fit 'int' into an index-sized integer
msg219208 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2014-05-27 07:49
Thanks. Patch modified.
msg219260 - (view) Author: Roundup Robot (python-dev) Date: 2014-05-28 10:03
New changeset 6e2833ae1718 by Serhiy Storchaka in branch '2.7':
Issue #10203: sqlite3.Row now truly supports sequence protocol.  In particular
http://hg.python.org/cpython/rev/6e2833ae1718

New changeset 6af865f1a59d by Serhiy Storchaka in branch '3.4':
Issue #10203: sqlite3.Row now truly supports sequence protocol.  In particulr
http://hg.python.org/cpython/rev/6af865f1a59d

New changeset 474c97a5f0c8 by Serhiy Storchaka in branch 'default':
Issue #10203: sqlite3.Row now truly supports sequence protocol.  In particulr
http://hg.python.org/cpython/rev/474c97a5f0c8
msg219261 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-05-28 10:06
Thank you Claudiu for your contribution.

But please be more careful, your patches contained trailing whitespaces.

Thank you Paul for your report.
History
Date User Action Args
2014-05-28 10:06:50serhiy.storchakasetstatus: open -> closed
resolution: fixed
messages: + msg219261

stage: commit review -> resolved
2014-05-28 10:03:01python-devsetnosy: + python-dev
messages: + msg219260
2014-05-27 07:49:30Claudiu.Popasetfiles: + issue10203_2.patch

messages: + msg219208
2014-05-27 07:30:06serhiy.storchakasetmessages: + msg219206
2014-05-26 20:58:26Claudiu.Popasetfiles: + issue10203_1.patch

messages: + msg219179
2014-05-26 20:23:47serhiy.storchakasetassignee: serhiy.storchaka
messages: + msg219176
2014-05-25 19:41:28Claudiu.Popasetfiles: + issue10203.patch

messages: + msg219103
2014-05-25 18:50:33serhiy.storchakasetmessages: + msg219099
2014-05-25 18:32:19serhiy.storchakasetnosy: + serhiy.storchaka
2014-04-28 23:08:57pitrousetnosy: + jesstess, pitrou
messages: + msg217441
2014-03-17 18:24:49eric.araujosetmessages: + msg213880
versions: - Python 3.3
2014-03-17 17:04:20eric.araujosetmessages: + msg213877
stage: patch review -> commit review
2014-03-17 17:02:55eric.araujosetmessages: + msg213876
versions: + Python 3.3
2014-03-17 14:12:49BreamoreBoysetnosy: + BreamoreBoy

messages: + msg213868
versions: + Python 2.7, Python 3.4
2014-03-17 14:01:36Claudiu.Popasetmessages: + msg213865
2013-12-04 07:21:57Claudiu.Popasetfiles: + sqlite2.patch

messages: + msg205201
2013-12-04 07:12:33vajraskysetnosy: + vajrasky
messages: + msg205197
2013-12-04 06:35:05Claudiu.Popasetmessages: + msg205195
2013-12-03 23:11:07eric.araujosetkeywords: + needs review

messages: + msg205172
stage: patch review
2013-11-30 09:06:55Claudiu.Popasetfiles: + sqlite1.patch
versions: + Python 3.5, - Python 3.1, Python 2.7, Python 3.2
nosy: + Claudiu.Popa

messages: + msg204789

keywords: + patch
2010-10-27 12:13:46lemburgsetnosy: + lemburg
messages: + msg119702
2010-10-27 11:58:53eric.araujosetmessages: + msg119699
2010-10-27 11:58:18eric.araujosetnosy: + ghaering, eric.araujo

messages: + msg119698
versions: + Python 3.1, Python 3.2, - Python 2.6
2010-10-26 19:16:05pfalconsetversions: + Python 2.7
2010-10-26 19:11:31pfalconcreate