classification
Title: Csv sniffer doesn't attempt to determine and set escape character.
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: evan.whitfield
Priority: normal Keywords:

Created on 2019-12-18 20:50 by evan.whitfield, last changed 2019-12-18 20:50 by evan.whitfield.

Messages (1)
msg358645 - (view) Author: Evan (evan.whitfield) Date: 2019-12-18 20:50
I observed a false positive for the csv sniffer has_header method. (It thought there was a header when there was not.) This is due to the fact that in has_header, it determines the csv dialect by sniffing it, and failed to determine that the file I was using had an escape character of '\'. Since it doesn't set the escape character, it then incorrectly broke the first line of the file into columns, since it encountered an escaped quote within a quoted column, and treated that as the end of that column. (It correctly determined that the dialect wasn't doublequote, but apparently still needs to have the escape character set to handle an escaped quotechar.) 

I think one (or both) of these things should be done here to avoid this false positive:
1.) Allow a dialect to be passed to has_header, so that someone could specify the escape character of the dialect if it were known.
2.) Allow the sniff method of the Sniffer class to detect and set the escapechar.
History
Date User Action Args
2019-12-18 20:50:42evan.whitfieldcreate