Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PyUnicode_EncodeFSDefault win32 inconsistancy. #60063

Closed
ideasman42 mannequin opened this issue Sep 4, 2012 · 11 comments
Closed

PyUnicode_EncodeFSDefault win32 inconsistancy. #60063

ideasman42 mannequin opened this issue Sep 4, 2012 · 11 comments
Labels
OS-windows topic-unicode type-bug An unexpected behavior, bug, or error

Comments

@ideasman42
Copy link
Mannequin

ideasman42 mannequin commented Sep 4, 2012

BPO 15859
Nosy @pitrou, @vstinner, @tiran, @ezio-melotti, @ideasman42, @berkerpeksag
Files
  • fix_unicode.diff: patch on python 3.3
  • fix_unicode_v2.diff
  • gns3-errorlog
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2015-01-26.15:48:24.759>
    created_at = <Date 2012-09-04.05:24:42.879>
    labels = ['type-bug', 'expert-unicode', 'OS-windows']
    title = 'PyUnicode_EncodeFSDefault win32 inconsistancy.'
    updated_at = <Date 2016-03-28.11:10:38.356>
    user = 'https://github.com/ideasman42'

    bugs.python.org fields:

    activity = <Date 2016-03-28.11:10:38.356>
    actor = 'berker.peksag'
    assignee = 'none'
    closed = True
    closed_date = <Date 2015-01-26.15:48:24.759>
    closer = 'vstinner'
    components = ['Unicode', 'Windows']
    creation = <Date 2012-09-04.05:24:42.879>
    creator = 'ideasman42'
    dependencies = []
    files = ['27114', '36913', '42313']
    hgrepos = []
    issue_num = 15859
    keywords = ['patch']
    message_count = 11.0
    messages = ['169816', '169834', '192793', '192795', '214543', '214559', '229288', '234751', '234752', '262548', '262567']
    nosy_count = 8.0
    nosy_names = ['pitrou', 'vstinner', 'christian.heimes', 'ezio.melotti', 'ideasman42', 'python-dev', 'berker.peksag', 'shanzhengcheng']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue15859'
    versions = ['Python 3.5']

    @ideasman42
    Copy link
    Mannequin Author

    ideasman42 mannequin commented Sep 4, 2012

    There is an inconsistency in PyUnicode_EncodeFSDefault(), on Linux its argument is checked to be unicode, and NULL is returned when its not. On windows however, this throws an assertion.

    The problem with this is, in some CAPI code you may pass an argument and check for NULL as an error case, and allow the python API's exception to be exposed to the script author.

    The problem here is a linux developer can use PyUnicode_EncodeFSDefault() this way, but not a windows developer.

    A simplified use case below of a case where PyUnicode_EncodeFSDefault would fail.

    Attached a fix so windows and posix systems behave the same.

    ---

    const char *PyC_UnicodeAsByte(PyObject *py_str, PyObject **coerce)
    {
    	if (PyBytes_Check(py_str)) {
    		return PyBytes_AS_STRING(py_str);
    	}
    	else if ((*coerce = PyUnicode_EncodeFSDefault(py_str))) {
    		return PyBytes_AS_STRING(*coerce);
    	}

    Also: heres a list to the bug report in blenders tracker, where the bug was found.

    https://projects.blender.org/tracker/index.php?func=detail&aid=31856&group_id=9&atid=498

    @ezio-melotti ezio-melotti added the type-bug An unexpected behavior, bug, or error label Sep 4, 2012
    @pitrou
    Copy link
    Member

    pitrou commented Sep 4, 2012

    Sounds reasonable to me.

    @tiran
    Copy link
    Member

    tiran commented Jul 10, 2013

    The code has changed considerable since you have created the patch. Is this still an issue for Python 3.3?

    @tiran tiran added the stale Stale PR or inactive for long period of time. label Jul 10, 2013
    @vstinner
    Copy link
    Member

    Is this still an issue for Python 3.3?

    Yes.

    @BreamoreBoy
    Copy link
    Mannequin

    BreamoreBoy mannequin commented Mar 23, 2014

    msg192795 says this is still an issue so I'm not sure why it's been set to languishing as it's only 18 months old.

    @vstinner
    Copy link
    Member

    It's not really a bug, Python behaves badly when you pass invalid parameters.

    Campbell was asked to update his patch but he didn't 2 years later. So I just close the issue. Reopen a new issue with an updated patch if you want a nice exception message instead of a patch.

    Or just fix your application to use Python C API correctly.

    @vstinner vstinner removed the stale Stale PR or inactive for long period of time. label Mar 23, 2014
    @ideasman42
    Copy link
    Mannequin Author

    ideasman42 mannequin commented Oct 14, 2014

    Updated the patch for '93049:d9a3d23cf8f0'

    Note, the link for the original bug report has changed: See https://developer.blender.org/T31856

    @ideasman42 ideasman42 mannequin reopened this Oct 14, 2014
    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Jan 26, 2015

    New changeset e124aab5d9a0 by Victor Stinner in branch 'default':
    Issue bpo-15859: PyUnicode_EncodeFSDefault(), PyUnicode_EncodeMBCS() and
    https://hg.python.org/cpython/rev/e124aab5d9a0

    @vstinner
    Copy link
    Member

    I applied fix_unicode_v2.diff to Python 3.5.

    For older Python versions, it's easy to workaround this issue: ensure explicitly that your object is a Unicode object using PyUnicode_Check().

    You may write your function to call PyUnicode_EncodeFSDefault() with a PyUnicode_Check() check.

    I prefer to not touch the stable 3.4 branch.

    @shanzhengcheng
    Copy link
    Mannequin

    shanzhengcheng mannequin commented Mar 28, 2016

    this issue occureed when i run gns3-1.4.5 in python3.5.1
    help me !
    my system is linux 3.6.11 release version is red flag inwise 8.0

    @shanzhengcheng shanzhengcheng mannequin removed the OS-windows label Mar 28, 2016
    @shanzhengcheng shanzhengcheng mannequin changed the title PyUnicode_EncodeFSDefault win32 inconsistancy. Fail update installation: 'utf-8' codec can't decode byte 0x90 in position 0: invalid start byte Mar 28, 2016
    @berkerpeksag
    Copy link
    Member

    shanzhengcheng: Please don't update closed issues.

    gns3-1.4.5 is not part of the Python standard library. You'll get better support by using the gns3 support channels.

    If you still think that this is a bug in Python, please file a new issue by using the form at http://bugs.python.org/issue?@template=item (it would be great if you provide a reproducer without using gns3)

    Thanks!

    @berkerpeksag berkerpeksag changed the title Fail update installation: 'utf-8' codec can't decode byte 0x90 in position 0: invalid start byte PyUnicode_EncodeFSDefault win32 inconsistancy. Mar 28, 2016
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    OS-windows topic-unicode type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    5 participants