classification
Title: _winreg.QueryValueEx confused on double null terminated strings
Type: enhancement Stage: needs patch
Components: Windows Versions: Python 3.6, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 2.7, Python 2.5
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: AndrewZiem, Tony Grue, amaury.forgeotdarc
Priority: normal Keywords:

Created on 2009-07-09 03:21 by AndrewZiem, last changed 2016-06-02 01:55 by Tony Grue. This issue is now closed.

Files
File name Uploaded Description Edit
regedit_screenshot.PNG AndrewZiem, 2009-07-09 20:08 WinXP SP3 regedit screenshot
delete_locked_file.py AndrewZiem, 2009-07-10 03:09 shows win32file.MoveFileEx() creates double nulls in registry
Messages (9)
msg90300 - (view) Author: Andrew Ziem (AndrewZiem) Date: 2009-07-09 03:21
QueryValueEx only returns one string when the contents are double null
terminated, yet QueryValueEx works when the string is singly null
terminated.

In Python 2.5.4 on Windows XP Sp3, the first key-value only returns the
first part up to \0\0.
-------------------
C:\>reg query "HKLM\SYSTEM\CurrentControlSet\Control\Session Manager" /v
Pending
FileRenameOperations

! REG.EXE VERSION 3.0

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager
    PendingFileRenameOperations REG_MULTI_SZ    \??\C:\Documents and
Settings\All Users\Application Data\McAfee\Common
Framework\UpdateDir\0\0\??\C:\Program
Files\Google\GoogleToolbarNotifier\5.2.4204.1700\0\0\??\C:\Documents and
Settings\All Users\Application Data\McAfee\Common
Framework\UpdateDir\0\0\??\C:\Documents and Settings\All
Users\Application Data\McAfee\Common
Framework\UpdateDir\0\0\??\C:\Documents and Settings\All
Users\Application Data\McAfee\Common
Framework\UpdateDir\0\0\??\C:\Documents and Settings\All
Users\Application Data\McAfee\Common
Framework\UpdateDir\0\0\??\C:\Documents and Settings\All
Users\Application Data\McAfee\Common
Framework\UpdateDir\0\0\??\C:\Documents and Settings\All Users\A
pplication Data\McAfee\Common Framework\UpdateDir\0\0\??\C:\Documents
and Settings\All Users\Application Data\McAfee\Common
Framework\UpdateDir\0\0\0

C:\>reg query "HKLM\SYSTEM\CurrentControlSet\Services\Browser" /v 
DependOnService

! REG.EXE VERSION 3.0

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Browser
    DependOnService     REG_MULTI_SZ    LanmanWorkstation\0LanmanServer\0\0
msg90301 - (view) Author: Andrew Ziem (AndrewZiem) Date: 2009-07-09 03:28
The bug is noted in the Python source code in file PC/_winreg.c in in
function fixupMultiSZ() where it admits Python does not mimic Microsoft
regedit.exe
msg90317 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2009-07-09 09:24
The double null is supposed to mark the end of the whole sequence; the
content you show under "PendingFileRenameOperations" is a invalid
REG_MULTI_SZ value to me.
How does regedit display it?
msg90319 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2009-07-09 09:40
It seems that regedit itself cannot handle these values, see how users
have to edit data in binary mode to enter empty strings:
http://www.pcreview.co.uk/forums/thread-1715654.php

Here is a script that shows the problem: the value is returned truncated
before the empty string.

from _winreg import *
test_key_name = "SOFTWARE\\Python Registry Test Key - Delete Me"
hkey = CreateKey(HKEY_CURRENT_USER, test_key_name)
SetValueEx(hkey, "Multi", 0, REG_MULTI_SZ,
           ['test', 'with', '', 'an', 'empty', 'line'])
print QueryValueEx(hkey, "Multi")
msg90348 - (view) Author: Andrew Ziem (AndrewZiem) Date: 2009-07-09 20:08
Attached screenshot shows regedit handles \x00\x00 by showing all the
"invalid values" (each \x00 becomes a line break).  More importantly,
Windows apparently reads all the "invalid value" when processing
FileRenameOperations, so my Python program must be able to read "invalid
value" to append new instructions.
msg90357 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2009-07-09 22:07
Yes, but did you try to press "OK"? Regedit displays a warning that empty 
lines are not allowed, and *removes* them.
msg90369 - (view) Author: Andrew Ziem (AndrewZiem) Date: 2009-07-10 02:25
Yes, regedit does remove the blanks, but this bug report is about
QueryValueEx---not about SetValueEx :).  There must be a way to read all
the values compatible with Windows
msg90371 - (view) Author: Andrew Ziem (AndrewZiem) Date: 2009-07-10 03:09
Well, I no longer need QueryValueEx because I found out
win32file.MoveFileEx() does more easily what I need, but MoveFileEx (see
attachment for working example) shows that the Microsoft Window API
itself creates these "invalid values"
msg266861 - (view) Author: Tony Grue (Tony Grue) Date: 2016-06-02 01:55
I see this was closed,though it sounds like it's because a work around is found, not because the code now handles multiple null values in a row.

Looking at code from the github mirror (pasted below); it looks like the issue is that the countStrings function breaks out of iterating in the for loop, even if it hasn't reached the end of the buffer.

File PC/winreg.c:

static int
countStrings(wchar_t *data, int len)
{
    int strings;
    wchar_t *P;
    wchar_t *Q = data + len;

    for (P = data, strings = 0; P < Q && *P != '\0'; P++, strings++)
        for (; P < Q && *P != '\0'; P++)
            ;
    return strings;
}
History
Date User Action Args
2016-06-02 01:55:50Tony Gruesetnosy: + Tony Grue

messages: + msg266861
versions: + Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6
2010-08-03 21:23:44terry.reedysetstatus: open -> closed
type: behavior -> enhancement
resolution: wont fix
2009-07-10 03:09:04AndrewZiemsetfiles: + delete_locked_file.py

messages: + msg90371
2009-07-10 02:25:31AndrewZiemsetmessages: + msg90369
2009-07-09 22:07:34amaury.forgeotdarcsetmessages: + msg90357
2009-07-09 20:08:43AndrewZiemsetfiles: + regedit_screenshot.PNG

messages: + msg90348
2009-07-09 09:40:33amaury.forgeotdarcsetmessages: + msg90319
stage: needs patch
2009-07-09 09:24:56amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg90317
2009-07-09 03:28:53AndrewZiemsetmessages: + msg90301
2009-07-09 03:21:08AndrewZiemcreate