classification
Title: JSON decoder reports wrong column number on first line
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.4, Python 3.3, Python 3.2, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: bob.ippolito, ezio.melotti, fbeyer, pitrou, python-dev, rhettinger, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2013-02-18 13:19 by fbeyer, last changed 2013-02-21 18:56 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
json_first_line_columns.patch serhiy.storchaka, 2013-02-18 17:05 review
Messages (9)
msg182320 - (view) Author: Ferdinand Beyer (fbeyer) Date: 2013-02-18 13:19
The linecol() function in json/decoder.py computes the line and column numbers for a byte offset in a string.  Both numbers are expected to start with 1 (as in text editors).

If the position is in the first line, the returned column is off by one (or starting with zero):

    >>> from json.decoder import linecol
    >>> linecol('spam', 0)  # Should be (1, 1)
    (1, 0)
    >>> linecol('\nspam', 1)
    (2, 1)

The problem is the line:

    if lineno == 1:
        colno = pos

that should read

    if lineno == 1:
        colno = pos + 1
msg182329 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-02-18 17:05
Here is a patch. This change breaks tests, but unlikely anything except tests depends on exact error message.
msg182578 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-02-21 02:54
Are these values accessible from somewhere (e.g. as attributes of the exception)?
msg182583 - (view) Author: Ferdinand Beyer (fbeyer) Date: 2013-02-21 10:19
Line and column number are included in the formatted error message ("raise ValueError(errormsg(...))").  They are currently not accessible separately as exception arguments.
msg182589 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-02-21 12:05
These values used only in the exception message. However current (3.0.8, stdlib json based on 2.0.9) simplejson exposes them as an exception attributes.

http://simplejson.readthedocs.org/en/latest/#exceptions
msg182603 - (view) Author: Bob Ippolito (bob.ippolito) * (Python committer) Date: 2013-02-21 17:50
I've applied a very similar patch to simplejson and released 3.0.9 

https://github.com/simplejson/simplejson/commit/44d7709a31f3a19f3d465411585ebb7be7fa2295
msg182604 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-02-21 17:52
https://github.com/simplejson/simplejson/issues/57

Simplejson has fixed this for about 6 hours.
msg182605 - (view) Author: Roundup Robot (python-dev) Date: 2013-02-21 18:31
New changeset ce583eb0bec2 by Serhiy Storchaka in branch '2.7':
Issue #17225: JSON decoder now counts columns in the first line starting
http://hg.python.org/cpython/rev/ce583eb0bec2

New changeset 36220cf535aa by Serhiy Storchaka in branch '3.2':
Issue #17225: JSON decoder now counts columns in the first line starting
http://hg.python.org/cpython/rev/36220cf535aa

New changeset 361ba6d4b7c9 by Serhiy Storchaka in branch '3.3':
Issue #17225: JSON decoder now counts columns in the first line starting
http://hg.python.org/cpython/rev/361ba6d4b7c9

New changeset 69f793cc34fc by Serhiy Storchaka in branch 'default':
Issue #17225: JSON decoder now counts columns in the first line starting
http://hg.python.org/cpython/rev/69f793cc34fc
msg182606 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-02-21 18:56
Thank you for the report, Ferdinand.
History
Date User Action Args
2013-02-21 18:56:10serhiy.storchakasetstatus: open -> closed
resolution: fixed
messages: + msg182606

stage: patch review -> resolved
2013-02-21 18:31:11python-devsetnosy: + python-dev
messages: + msg182605
2013-02-21 17:52:30serhiy.storchakasetmessages: + msg182604
2013-02-21 17:50:59bob.ippolitosetnosy: + bob.ippolito
messages: + msg182603
2013-02-21 12:05:35serhiy.storchakasetmessages: + msg182589
2013-02-21 10:19:55fbeyersetmessages: + msg182583
2013-02-21 02:54:55ezio.melottisetmessages: + msg182578
2013-02-18 17:05:25serhiy.storchakasetfiles: + json_first_line_columns.patch
keywords: + patch
messages: + msg182329

stage: needs patch -> patch review
2013-02-18 13:32:02serhiy.storchakasetassignee: serhiy.storchaka
stage: needs patch

nosy: + rhettinger, pitrou, ezio.melotti, serhiy.storchaka
versions: - Python 2.6, Python 3.1, Python 3.5
2013-02-18 13:19:26fbeyercreate