sqlite3 iterdump fails on column with reserved name #53959

MarkoKohtala · 2010-09-03T06:15:25Z

BPO	9750
Nosy	@merwok, @bitdancer, @akheron
Files	issue9750.patch: Patch with tests and fixes sqlite3dump.patch: Test cases and patch to fix problems

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2012-02-12.19:09:42.026>
created_at = <Date 2010-09-03.06:15:25.215>
labels = ['extension-modules', 'type-bug']
title = 'sqlite3 iterdump fails on column with reserved name'
updated_at = <Date 2013-01-10.16:31:33.794>
user = 'https://bugs.python.org/MarkoKohtala'

bugs.python.org fields:

activity = <Date 2013-01-10.16:31:33.794>
actor = 'python-dev'
assignee = 'ghaering'
closed = True
closed_date = <Date 2012-02-12.19:09:42.026>
closer = 'python-dev'
components = ['Extension Modules']
creation = <Date 2010-09-03.06:15:25.215>
creator = 'Marko.Kohtala'
dependencies = []
files = ['23629', '24429']
hgrepos = []
issue_num = 9750
keywords = ['patch']
message_count = 19.0
messages = ['115420', '115421', '115423', '115445', '116144', '116155', '131582', '146362', '146374', '146702', '146710', '147277', '152710', '152901', '153088', '153222', '163245', '179560', '179565']
nosy_count = 7.0
nosy_names = ['ghaering', 'eric.araujo', 'r.david.murray', 'Marko.Kohtala', 'python-dev', 'petri.lehtinen', 'xapple']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue9750'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3']

MarkoKohtala · 2010-09-03T06:15:23Z

Sqlite3 fails to dump a database with column names that are keywords.

MarkoKohtala · 2010-09-03T06:17:35Z

Here is a patch that may resolve the bug.

MarkoKohtala · 2010-09-03T08:34:59Z

The second patch contains also fixes for some places where the rules described in http://www.sqlite.org/lang_keywords.html are not followed.

MarkoKohtala · 2010-09-03T12:09:09Z

It also fails if table or column names contain double quote.

merwok · 2010-09-12T01:07:10Z

Thank you for the report and patch. Some bug tracker tips:

It’s possible to replace a file, so that there is always one bug.py and one diff, which is easier for reviewers to understand than a long list of files (possibly all with the same name). It’s also possible to remove obsolete files so that nobody loses time commenting on them.
Can you turn your bug reproducer into a patch for Lib/test/test_sqlite.py?
Please follow guidelines on http://www.python.org/dev/patches/ to make patches.

Regarding the code itself:

a) I don’t understand why you added quotes around names that seemed fine without them (for example in “SELECT name”).

b) I find the master line difficult to read: “q += ",".join(["'||quote(\"" + col.replace('"', '""') + "\")||'" for col in column_names])”. You’re using backslashes since there are both single and double quotes, but I would use triple-quoted strings here. I also find string formatting more readable than string concatenation: 'blah blah "{}" blah'.format(thing.replace('"', '""')).

c) Minor style thing: A generator expression works as fine as a list comprehension in str.join (that is, ",".join(i for i in source) == ";".join([i for i in source])).

Thanks again!

MarkoKohtala · 2010-09-12T08:57:19Z

Thank you for the review.

I have very limited time to use on this. So even when I'd like to make everything easy for you, have the time you give to python be as productive as possible, I can not.

But I'll respond to your comments on the patch.

a) I added the quotes to every identifier based on a comment in sqlite documentation "SQLite adds new keywords from time to time when it takes on new features. So to prevent your code from being broken by future enhancements, you should normally quote any identifier that is an English language word, even if you do not have to." While fixing one place, I fixed it to follow this recommendation in other places as well.

b) I added quotes using backslashes because it was consistent. The table name was already quoted like that. I agree it could be clearer.

c) I know. I only tried to make minimal changes targeted only to the issues at hand. I did not want to hide the fixes in middlde of changes to style.

merwok · 2011-03-21T00:33:22Z

I have very limited time to use on this.
This is a perfectly reasonable stance, and we’re grateful for the work you’re doing. When you say that you don’t have time, someone else will take up the patch and update it to address reviews, and you will get credited.

a) Thanks for the explanation. This would be a useful comment in the source.

b) It’s not uncommon to make cosmetic changes in the same commit as a bugfix. Readability is valued.

xapple · 2011-10-25T12:00:51Z

I just encountered this issue today.

So, it's been several months, will the patch be merged into the master branch ? Or will this never be fixed ?

merwok · 2011-10-25T15:24:14Z

Hi Lucas. Have you read my previous message? The patch needs to be updated. Would you like to do it?

xapple · 2011-10-31T15:16:56Z

Sure, I can have a try at it and address the issues you pointed out.

The URL to the guidelines you provided gives a 404. In what form exactly would you like the patch to be ?

I wouldn't mind either adding to the test suite, but I'm not sure how to do that.

merwok · 2011-10-31T16:41:23Z

Sure, I can have a try at it and address the issues you pointed out.
Nice!

The URL to the guidelines you provided gives a 404.
They’ve moved to http://docs.python.org/devguide meanwhile.

I wouldn't mind either adding to the test suite, but I'm not sure how to do that.
Edit Lib/sqlite3/test/dump.py

xapple · 2011-11-08T08:43:08Z

My patch contains two new dump tests which originally fail, and the fixes to the code that makes them pass.

MarkoKohtala · 2012-02-05T21:33:25Z

Here is finally an update to my patch modified according to comments received. It should apply on 2.7 and 3.3 branches.

akheron · 2012-02-08T19:37:44Z

Marko's last patch looks good to me.

merwok · 2012-02-11T03:59:52Z

Petri, I think you can go ahead.

Lucas, thanks for your help, even though your patch is not used.

python-dev · 2012-02-12T19:09:42Z

New changeset 60aa7dacb605 by Petri Lehtinen in branch '2.7':
Fix sqlite3.Connection.iterdump on tables/fields with reserved names or quotes
http://hg.python.org/cpython/rev/60aa7dacb605

New changeset 4b105d328fe7 by Petri Lehtinen in branch '3.2':
Fix sqlite3.Connection.iterdump on tables/fields with reserved names or quotes
http://hg.python.org/cpython/rev/4b105d328fe7

New changeset 5d0f7b275fe9 by Petri Lehtinen in branch 'default':
Merge branch '3.2'
http://hg.python.org/cpython/rev/5d0f7b275fe9

bitdancer · 2012-06-20T01:56:36Z

Note that the use of 'format' in the 2.7 patch caused a regression (% causes an implicit promotion to unicode, .format does not). See bpo-15109.

bitdancer · 2013-01-10T15:34:56Z

For the record, this patch also introduced another regression (bpo-15545).

python-dev · 2013-01-10T16:31:32Z

New changeset 2cdb599172ab by R David Murray in branch '3.2':
bpo-15545: fix sqlite3.iterdump regression on unsortable row_factory objects.
http://hg.python.org/cpython/rev/2cdb599172ab

New changeset 6a85894c428f by R David Murray in branch '3.3':
merge bpo-15545: fix sqlite3.iterdump regression on unsortable row_factory objects.
http://hg.python.org/cpython/rev/6a85894c428f

New changeset 7a62b5ee32ec by R David Murray in branch 'default':
merge bpo-15545: fix sqlite3.iterdump regression on unsortable row_factory objects.
http://hg.python.org/cpython/rev/7a62b5ee32ec

New changeset bb4e4f0cec2e by R David Murray in branch '2.7':
bpo-15545: sort iterdump via SQL instead of in python code
http://hg.python.org/cpython/rev/bb4e4f0cec2e

MarkoKohtala mannequin added extension-modules C modules in the Modules dir type-bug An unexpected behavior, bug, or error labels Sep 3, 2010

ghaering mannequin self-assigned this Sep 3, 2010

python-dev mannequin closed this as completed Feb 12, 2012

ezio-melotti transferred this issue from another repository Apr 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

sqlite3 iterdump fails on column with reserved name #53959

sqlite3 iterdump fails on column with reserved name #53959

MarkoKohtala mannequin commented Sep 3, 2010

MarkoKohtala mannequin commented Sep 3, 2010

MarkoKohtala mannequin commented Sep 3, 2010

MarkoKohtala mannequin commented Sep 3, 2010

MarkoKohtala mannequin commented Sep 3, 2010

merwok commented Sep 12, 2010

MarkoKohtala mannequin commented Sep 12, 2010

merwok commented Mar 21, 2011

xapple mannequin commented Oct 25, 2011

merwok commented Oct 25, 2011

xapple mannequin commented Oct 31, 2011

merwok commented Oct 31, 2011

xapple mannequin commented Nov 8, 2011

MarkoKohtala mannequin commented Feb 5, 2012

akheron commented Feb 8, 2012

merwok commented Feb 11, 2012

python-dev mannequin commented Feb 12, 2012

bitdancer commented Jun 20, 2012

bitdancer commented Jan 10, 2013

python-dev mannequin commented Jan 10, 2013

sqlite3 iterdump fails on column with reserved name #53959

sqlite3 iterdump fails on column with reserved name #53959

Comments

MarkoKohtala mannequin commented Sep 3, 2010

MarkoKohtala mannequin commented Sep 3, 2010

MarkoKohtala mannequin commented Sep 3, 2010

MarkoKohtala mannequin commented Sep 3, 2010

MarkoKohtala mannequin commented Sep 3, 2010

merwok commented Sep 12, 2010

MarkoKohtala mannequin commented Sep 12, 2010

merwok commented Mar 21, 2011

xapple mannequin commented Oct 25, 2011

merwok commented Oct 25, 2011

xapple mannequin commented Oct 31, 2011

merwok commented Oct 31, 2011

xapple mannequin commented Nov 8, 2011

MarkoKohtala mannequin commented Feb 5, 2012

akheron commented Feb 8, 2012

merwok commented Feb 11, 2012

python-dev mannequin commented Feb 12, 2012

bitdancer commented Jun 20, 2012

bitdancer commented Jan 10, 2013

python-dev mannequin commented Jan 10, 2013