Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use WCHAR variant of OutputDebugString #48325

Closed
eckhardt mannequin opened this issue Oct 8, 2008 · 16 comments
Closed

Use WCHAR variant of OutputDebugString #48325

eckhardt mannequin opened this issue Oct 8, 2008 · 16 comments
Assignees
Labels
OS-windows type-feature A feature request or enhancement

Comments

@eckhardt
Copy link
Mannequin

eckhardt mannequin commented Oct 8, 2008

BPO 4075
Nosy @loewis, @theller, @amauryfa
Files
  • Python-OutputDebugStringW.0.patch: patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/loewis'
    closed_at = <Date 2009-01-02.20:33:18.060>
    created_at = <Date 2008-10-08.13:13:26.738>
    labels = ['type-feature', 'OS-windows']
    title = 'Use WCHAR variant of OutputDebugString'
    updated_at = <Date 2009-01-02.20:33:18.039>
    user = 'https://bugs.python.org/eckhardt'

    bugs.python.org fields:

    activity = <Date 2009-01-02.20:33:18.039>
    actor = 'loewis'
    assignee = 'loewis'
    closed = True
    closed_date = <Date 2009-01-02.20:33:18.060>
    closer = 'loewis'
    components = ['Windows']
    creation = <Date 2008-10-08.13:13:26.738>
    creator = 'eckhardt'
    dependencies = []
    files = ['11749']
    hgrepos = []
    issue_num = 4075
    keywords = ['patch']
    message_count = 16.0
    messages = ['74527', '74529', '74542', '74558', '74559', '74577', '74592', '74596', '74605', '74618', '74654', '74675', '74726', '74768', '74769', '78873']
    nosy_count = 5.0
    nosy_names = ['loewis', 'theller', 'amaury.forgeotdarc', 'rpetrov', 'eckhardt']
    pr_nums = []
    priority = 'high'
    resolution = 'accepted'
    stage = 'commit review'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue4075'
    versions = ['Python 2.7']

    @eckhardt
    Copy link
    Mannequin Author

    eckhardt mannequin commented Oct 8, 2008

    The attached patch converts the call to OutputDebugString() with a
    'TCHAR' parameter (which boils down to a 'char') to one using a 'WCHAR'
    parameter, allowing the code to be compiled under MS Windows CE, which
    doesn't have the 'char' version.

    @eckhardt eckhardt mannequin added OS-windows type-feature A feature request or enhancement labels Oct 8, 2008
    @amauryfa
    Copy link
    Member

    amauryfa commented Oct 8, 2008

    The alloca() function should be avoided here: the function may be called
    in extreme conditions, like stack overflow.
    I suggest to use a small static buffer (50 chars?), and call
    OutputDebugStringW in a loop.

    @loewis
    Copy link
    Mannequin

    loewis mannequin commented Oct 8, 2008

    I agree that a static buffer should be used. I think calling it in a
    loop is overkill. Instead, if an overrun occurs, adding "(truncated)"
    should be good enough. I could find only a single caller that doesn't
    pass a static string (_Py_NegativeRefcount), where I think 50 characters
    are still plenty.

    @rpetrov
    Copy link
    Mannequin

    rpetrov mannequin commented Oct 9, 2008

    Which CE version ? Is the patch required for previous/next CE version ?
    If the CE can't work with char why the compiler don't threat strings as
    wide characters always ?

    @eckhardt
    Copy link
    Mannequin Author

    eckhardt mannequin commented Oct 9, 2008

    Roumen, just and explanation on the TCHAR/WCHAR/CHAR issue under win32...

    In the old days, DOS/Windows was built with 8-bit characters using
    codepages. So functions like CreateFile() took a char string that used
    the current local codepage as encoding. Now, since NT 4 (maybe even 3)
    the internally used char type is a 16-bit type. In order to ease
    conversion, the function CreateFile() was removed (it still exists in
    oldnames.lib) and replaced with CreateFileW() and CreateFileA(), which
    explicitly take either a codepage-encoded 8-bit string or a UCS2/UTF-16
    16-bit string. Under win9x, CreateFileW() actually tried to convert to
    the internally used 8-bit character type, while under NT, CreateFileA()
    converted from the codepage to the UTF-16 character type.

    Now, under CE, which is an embedded OS, neither the
    (legacy/obsolete/deprecated) codepages nor the according CreateFileA()
    functions exist. They simply have been removed to save space and because
    they are of limited functionality anyway.

    Which CE version? All of them, since at least CE3 (CE6 is current). Why
    not treat all strings as wide string? Because that would actually change
    the existing meaning of them and make it harder to impossible to create
    code that is portable.

    @eckhardt
    Copy link
    Mannequin Author

    eckhardt mannequin commented Oct 9, 2008

    Actually, even _Py_NegativeRefcount() passes a statically sized buffer
    with 300 chars. Other than that, there is get_ref_type() which uses one
    with 350 chars, but AFAICT, that's the largest one. Attached accordingly
    modified patch.

    @loewis
    Copy link
    Mannequin

    loewis mannequin commented Oct 9, 2008

    If the CE can't work with char why the compiler don't threat strings as
    wide characters always ?

    I think this question is pointless - we don't have the power to change
    how CE works. You might question whether Ulrich's analysis of the issue
    is accurate (I think it is), or whether Python should support CE at all
    (I think it should). FWIW, the compiler *does* "work with char", and it
    needs to do so to support a byte type. It's just that the CE APIs don't
    support char (at least, some of them apparently don't).

    @rpetrov
    Copy link
    Mannequin

    rpetrov mannequin commented Oct 9, 2008

    My experience with windows CE ends with version about 3.1X. I couldn't
    remember wide character support on this version.
    PythonCE project use xxxA functions for CE .NET 4.20 platform.

    "Pointless" question is for compiler flags and is not related with the OS.

    @loewis
    Copy link
    Mannequin

    loewis mannequin commented Oct 9, 2008

    "Pointless" question is for compiler flags and is not related with the OS.

    I don't think the compiler has any such flag that you might consider
    useful. Do you have a specific flag in mind?

    @eckhardt
    Copy link
    Mannequin Author

    eckhardt mannequin commented Oct 10, 2008

    "PythonCE project use xxxA functions for CE .NET 4.20 platform."

    Look again. The PythonCE project adds a header and sourcefile
    (wince_compatibility.h/c) which adds these functions. In other words, it
    doesn't use the xxxA functions of the win32 API (which don't exist under
    CE) but its own replacements.

    I was thinking of going that way too, but in the end decided against it
    unless absolutely necessary. The point is that this approach allowed
    minimal changes to the Python code which still had to support the xxxA
    variants for win9x. However, since IIRC 2.6 support for win9x has been
    dropped, so now it makes much more sense to use the WCHAR APIs which is
    what all supported MS Windows versions use internally anyway. This
    allows code to work under CE unchanged, avoids unnecessary conversions
    and provides better Unicode support.

    BTW: in case somebody actually wants to resurrect the win9x support,
    there is a library from Microsoft that provides the xxxW functions for
    that platform. Of course that's not a cure but just a band-aid with
    reduced functionality, but at least it's possible.

    @rpetrov
    Copy link
    Mannequin

    rpetrov mannequin commented Oct 10, 2008

    I couldn't find in MSDN flags for Windows CE compilers similar to the
    GCC compiler that change representation of strings in C-code. The
    Microsoft recommend so called TCHAR technology that depend from UNICODE
    define I answer itself.

    From MSDN isn't so clear what is status of CE version 6.0 - R2?. The
    documentation of CE 5.0 looks complete, include details for compiler,
    migration guide.

    About wince_compatibility.h/c - it is just a part of the patch
    (PythonCE-2.5-20061219). If we look in the complete patch we see changes
    like GetLocaleInfo->GetLocaleInfoA(_localemodule.c) but
    windll.kernel32.GetProcAddress->cdll.coredll.GetProcAddressW
    (test_random_things.py). Another change (posixmodule.c)
    CRYPTACQUIRECONTEXTA->CRYPTACQUIRECONTEXT.

    So we see changes from xxx to xxxA, xxx to xxxW, xxxA to xxx (!).

    If this patch required for CE 5.0 ? If not why to change now for may
    upcoming 6.0 ?
    If the python switch to W functions then the issue will be resolved.
    Without general switch to W functions and to wide strings we need to
    convert every time from single chat to wide char. Is this acceptable in
    general ? (for the particular case yes - the method is called rarely).

    @loewis
    Copy link
    Mannequin

    loewis mannequin commented Oct 12, 2008

    If the python switch to W functions then the issue will be resolved.

    Python should use the *W functions wherever possible.

    Without general switch to W functions and to wide strings we need to
    convert every time from single chat to wide char. Is this acceptable in
    general ?

    Most certainly. If we call the *A function instead, it will convert to
    *W itself, so it's no loss if we do it right away (and we may save a
    copy in some cases).

    @eckhardt
    Copy link
    Mannequin Author

    eckhardt mannequin commented Oct 14, 2008

    "If this patch required for CE 5.0?"

    The patch I created is required for all CEs that I know of. I have
    personally worked with 4.20, 5 and now 6, and had some exchange with
    others who worked on 3.x variants to get STLport (C++ stdlibrary
    implementation) to run. AFAIK, none of them support the *A functions, so
    this patch or something similar is required for every CE flavor out there.

    Roumen, you mentioned the way that the PythonCE project did it, which
    also works, but I'd say that that code is obsolete, because the emulated
    functions actually create a win9x-like environment, while Python has
    officially dropped support for that. The problem is that the *W
    functions are badly supported on win9x while the *A functions are
    unsupported on CE. The NT variants (NT, win2000..) support both APIs,
    but the *A functions are wrappers around the *W functions and don't
    provide the whole functionality that the OS actually supports.

    So, what this path does is to help phase out the *A functions, gaining
    more thorough Unicode support while at the same time easing porting to CE.

    @rpetrov
    Copy link
    Mannequin

    rpetrov mannequin commented Oct 14, 2008

    May be OutputDebugStringA has to be part of "wince-port" library but if
    is fine all win platforms to call *W what about patch to use function
    MultiByteToWideChar() ?

    @loewis
    Copy link
    Mannequin

    loewis mannequin commented Oct 14, 2008

    May be OutputDebugStringA has to be part of "wince-port" library but if
    is fine all win platforms to call *W what about patch to use function
    MultiByteToWideChar() ?

    Is there a problem with the proposed patch?

    @loewis loewis mannequin self-assigned this Dec 30, 2008
    @loewis
    Copy link
    Mannequin

    loewis mannequin commented Jan 2, 2009

    Thanks for the patch. Committed as r68172.

    @loewis loewis mannequin closed this as completed Jan 2, 2009
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    OS-windows type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    1 participant