classification
Title: csv.writer lineterminator affects csv escaping
Type: behavior Stage: resolved
Components: Versions: Python 3.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: flow2k, martin.panter
Priority: normal Keywords:

Created on 2019-03-08 21:59 by flow2k, last changed 2019-03-15 19:02 by flow2k. This issue is now closed.

Messages (3)
msg337537 - (view) Author: (flow2k) Date: 2019-03-08 21:59
output = io.StringIO()
csvData = [1, 2, 'a', 'He said "what do you mean?"', "Whoa!\rNewlines!"]
writer = csv.writer(output,lineterminator='\n')
writer.writerow(csvData)
print(repr(output.getvalue())) #does not escape \r as expected
msg337547 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2019-03-09 00:09
This is the result that I see:

>>> output = StringIO()
>>> csv.writer(output, lineterminator='\n').writerow(["Whoa!\rNewlines!"])
16
>>> output.getvalue()
'Whoa!\rNewlines!\n'

For comparison, this is the result with CRLF terminators (the default):

>>> output = StringIO()
>>> csv.writer(output, lineterminator='\r\n').writerow(["Whoa!\rNewlines!"])
19
>>> output.getvalue()
'"Whoa!\rNewlines!"\r\n'

Is it a problem that the line terminator determines whether the CR is quoted or not? I believe the default policy is “excel”, which happens to use QUOTE_MINIMAL. This behaviour is documented: <https://docs.python.org/3.7/library/csv.html#csv.QUOTE_MINIMAL>.
msg338016 - (view) Author: (flow2k) Date: 2019-03-15 19:02
Okay, thanks for pointing to the doc. I did not expect the line termination to affect escaping, but I can see why these may be related. 

Per your comment, I've added quoting=csv.QUOTE_NONNUMERIC. Instead of Excel, I am using Gdocs, and this escaping enables Gsheets to ingest it.

Closing the issue...
History
Date User Action Args
2019-03-15 19:02:13flow2ksetstatus: pending -> closed

messages: + msg338016
stage: resolved
2019-03-09 00:09:35martin.pantersetstatus: open -> pending

nosy: + martin.panter
messages: + msg337547

resolution: not a bug
2019-03-08 21:59:52flow2kcreate