classification
Title: shutil.copyfile (or built-in open) releases sqlite3's transaction lock
Type: behavior Stage: resolved
Components: Versions: Python 2.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: h-miyajima
Priority: normal Keywords:

Created on 2015-03-13 08:13 by h-miyajima, last changed 2015-03-16 15:09 by r.david.murray. This issue is now closed.

Files
File name Uploaded Description Edit
Test_sqlite3_lock.py h-miyajima, 2015-03-13 08:13 A sample programs which demonstrates this case.
Messages (2)
msg238013 - (view) Author: Hiroaki Miyajima (h-miyajima) Date: 2015-03-13 08:13
When copying sqlite3's database file by shutil.copyfile while holding its transaction lock, its lock is released unexpectedly.

Attached is a sample program showing this. Inside,
process-1 and process-2 just do following steps respectively:
   (1) open a sqlite dbfile
   (2) execute('begin EXCLUSIVE transaction') to acquire lock
   (3) copy the dbfile
   (4) sleep some seconds for demonstration
   (4) rollback() to release lock

if shutil.copyfile() is used as (3), process-2 gets a lock
while process-1 must hold a lock.
It seems that this happens if database file is opened with built-in open(), which is used in shutil.copyfile.

In addition,
- With os.open(), there seems no problem.
- With linux cp command, it works correct.
- Using multi-thread instead of multi-process, it works expectedly.

I think this is a bug, but does someone explain this behaviour?
Thank you.

Below is the execute example of the attached on python 2.7.9.
----------
# python Test_sqlite3_lock.py
Use <function shutil_copyfile at 0x7fe810b22500>
process-1 : Transaction lock acquired
process-2 : Transaction lock acquired
process-1 : Transaction lock released
process-2 : Transaction lock released

Use <function low_copyfile at 0x7fe810b22578>
process-1 : Transaction lock acquired
process-1 : Transaction lock released
process-2 : Transaction lock acquired
process-2 : Transaction lock released

Use <function command_copyfile at 0x7fe810b225f0>
process-1 : Transaction lock acquired
process-1 : Transaction lock released
process-2 : Transaction lock acquired
process-2 : Transaction lock released

----------
msg238187 - (view) Author: Hiroaki Miyajima (h-miyajima) Date: 2015-03-16 09:27
I've realized that this is a bug of specification of fcntl system call.
I am sorry to bother this site.
History
Date User Action Args
2015-03-16 15:09:32r.david.murraysetresolution: not a bug
stage: resolved
2015-03-16 09:27:06h-miyajimasetstatus: open -> closed

messages: + msg238187
2015-03-13 08:13:02h-miyajimacreate