This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: csv.writer stopped to quote values with escapechar with csv.QUOTE_MINIMAL in Python 3.10
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.10
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: ebreck, serhiy.storchaka, taleinat, xflr6
Priority: normal Keywords:

Created on 2021-08-07 14:57 by xflr6, last changed 2022-04-11 14:59 by admin.

Messages (4)
msg399188 - (view) Author: Sebastian Bank (xflr6) Date: 2021-08-07 14:57
AFAICT there was an undocumented change in behaviour related to the fix of https://bugs.python.org/issue12178 (also reported in https://bugs.python.org/issue12178#msg397440):

Python 3.9 quotes values with escapechar:

```
import csv
import io

kwargs = {'escapechar': '\\'}

value = 'spam\\eggs'

print(value)

with io.StringIO() as buf:
    writer = csv.writer(buf, **kwargs)
    writer.writerow([value])
    line = buf.getvalue()

print(line.strip())

with io.StringIO(line) as buf:
    reader = csv.reader(buf, **kwargs)
    (new_value,), = reader

print(new_value)
spam\eggs
"spam\eggs"
spameggs
```

- quotes escapechar
- fails to double the escapechar (https://bugs.python.org/issue12178)

From https://docs.python.org/3/library/csv.html#csv.QUOTE_MINIMAL

> only quote those fields which contain special characters
> such as delimiter, quotechar or any of the characters in
> lineterminator.

The previous behaviour seems incorrect because escapechar is not explicitly mentioned, but at the same time the docs says 'such as'.

The new might better matching the name 'minimal', but at the same time one might regard 'quote when in doubt' as a safer behaviour for the default quoting rule.

Python 3.10:

https://github.com/python/cpython/blob/5c0eed7375fdd791cc5e19ceabfab4170ad44062/Lib/test/test_csv.py#L207-L208

See also https://github.com/xflr6/csv23/actions/runs/1027687524
msg399191 - (view) Author: Sebastian Bank (xflr6) Date: 2021-08-07 15:10
IIUC there is no way to work around this from client/downstream code (to get the olf 3.6 to 3.9 behaviour), so this might break assertions on the output of `csv.writer` for users of `escapechar` whenever the data to be written contains the escapcechar (e.g. calculating a hash/checksum).
msg399195 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-08-07 16:53
The 3.9 behavior is incorrect: write 'spam\\eggs', read 'spameggs'. The 3.10 behavior is correct: write 'spam\\eggs', read 'spam\\eggs'. What is your problem exactly?
msg399204 - (view) Author: Sebastian Bank (xflr6) Date: 2021-08-07 20:53
The 3.9 behaviour is write: "spam\eggs"

The 3.10 behaviour is write: spam\\eggs

I think at least the change in csv.QUOTE_MINIMAL behviour should be documented (maybe adding hint to avoid the `escapechar` option for consistent output).
History
Date User Action Args
2022-04-11 14:59:48adminsetgithub: 89024
2021-08-07 20:53:17xflr6setmessages: + msg399204
2021-08-07 16:53:42serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg399195
2021-08-07 15:10:44xflr6setmessages: + msg399191
2021-08-07 14:57:05xflr6create