classification
Title: embedded null byte when connecting to sqlite database using a bytes object
Type: behavior Stage: resolved
Components: Extension Modules Versions: Python 3.8
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: SilentGhost, ferferga
Priority: normal Keywords:

Created on 2020-04-02 12:23 by ferferga, last changed 2020-04-17 14:47 by ferferga. This issue is now closed.

Files
File name Uploaded Description Edit
bytes_io.py ferferga, 2020-04-02 12:23 Proof of concept
Messages (4)
msg365582 - (view) Author: Fernando (ferferga) Date: 2020-04-02 12:23
Hello. I think that I found a bug in how sqlite3 module handle bytes.

The connect function of sqlite3 accepts strings, FilePath objects and bytes. However, it's impossible for me to connect to bytes objects that are read from BufferedReaders. I always get: 

"ValueError: embedded null byte"

This is my current code (byteDec is the BytesIO object):

==============================================
byteDec.seek(0)
conn = sqlite3.connect(byteDec.read())
==============================================

That returns the "embedded null byte" error. However, if I do:

==============================================
byteDec.seek(0)
with open("db.db", "wb" as f:
    f.write(byteDec.read())
conn = sqlite3.connect("db.db")
==============================================

Everything works flawlessly, so the BufferedReader that I have in-memory is not corrupted in any way, as it's readable from a file. I want to avoid writing to disk at all, so this is not a solution for me.


I attach to this issue a very basic proof of concept to understand the issue.

I'm running Pyhton 3.8.2 amd64 on Windows 10 1909
msg365985 - (view) Author: Fernando (ferferga) Date: 2020-04-08 13:44
bump?
msg365994 - (view) Author: SilentGhost (SilentGhost) * (Python triager) Date: 2020-04-08 17:43
Hi Fernando,

the first parameter of the connect function is described in documentation as follows: 

> database is a path-like object giving the pathname (absolute or relative to the current working directory) of the database file to be opened. You can use ":memory:" to open a database connection to a database that resides in RAM instead of on disk.

So, while it can be a bytes object, it's still would be a bytes object representing a file-path. It's not bytes object representing a file content of the database.

Hope that helps.
msg366656 - (view) Author: Fernando (ferferga) Date: 2020-04-17 14:47
Hello SilentGhost,

Okay, now I understand the difference and had my code working! Thank you very much for your answer and to all of you who help in making Python better.

(Wish I had more knowledge of it to help)

Have a nice day!
History
Date User Action Args
2020-04-17 14:47:39ferfergasetmessages: + msg366656
2020-04-08 17:43:28SilentGhostsetstatus: open -> closed

type: crash -> behavior

nosy: + SilentGhost
messages: + msg365994
resolution: not a bug
stage: resolved
2020-04-08 13:49:48ferfergasetcomponents: + Extension Modules, - IO
2020-04-08 13:44:43ferfergasetmessages: + msg365985
2020-04-02 12:23:03ferfergacreate