New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Python exception on strftime with %f on Python 3 and Python 2 on windows #68432
Comments
http://bugs.python.org/issue11735 fixed in 2.7, but in 3.4 and 3.5 the same problem exists. Just crashes python. |
Please follow Ned's suggestion that prompted you to open this issue and report the exact version of python, your OS, and how to reproduce it. I cannot reproduce it based on the existing issues using python 3.4.1 on Windows 7. |
Majeed could be referring to the ValueError that gets raised for this only on Windows. In Linux %f passes through silently. Maybe for 3.5 an alternate approach would be to disable the Windows CRT's invalid parameter handler using _Py_BEGIN_SUPPRESS_IPH and _Py_END_SUPPRESS_IPH. Looking at this brought bpo-10653 to my attention. It seems VC++ 14 has exacerbated that problem, but that the current solution no longer works even in 3.4.3 with VC++ 10. Should the issue be reopened or a new issue opened? |
The ValueError is not a bug. Not all format codes are supported on all platforms (we start from what the platform supports, and then we have a few we have implemented cross platform, but that isn't one of them). We also pass through the behavior of the underlying system as far as handling unknown codes. Whether that is the best policy or not is an interesting question, but one that should probably be addressed on python-ideas if someone wants to tackle it, since it has backward compatibility implications (and the wider discussion would affect the the behavior on FreeBSD as well, where unknown codes are handled differently than they are on linux, though they don't produce an exception.) For the other I think you should open a new issue referencing the old one. |
When we have this for format: When I change the format to: Unhandled exception in thread started by |
Actually it is happening in Python 2.7 too.. Here is the format we are trying to use: date_format = '%Y %m %d %H:%M:%S:%f %z' Also, is there something millisecond if not microsecond? |
So, this is expected behavior on Windows. I'm inclined to close the issue as not a bug unless the Windows folks think eryksun suggestion is worth considering. (If the error is suppressed, does windows fill in the rest of the values and just leave the %f in the string?) |
No, it does not. I understand now why ValueError was an intentional choice here. The CRT actually breaks the loop with a goto if any call to expand_time fails. So raising a Python exception is the only reasonable course of action. (I still think maybe it's a good place to use the new _Py_BEGIN_SUPPRESS_IPH macro. That way instead of presuming what's a valid format code, it could just handle the EINVAL.) |
_Py_*_SUPPRESS_IPH is the right thing to use here. It still displays an assertion dialog in debug builds, but ignoring it has the correct effect. Patch attached for 3.5 |
What's the behavior after this patch? Does it just return the string unmodified? Or return a null string? Or is the traceback generated at a higher level (and if so what is the error message?) |
It raises ValueError just like now (no visible change), but it won't crash and doesn't require being able to validate the complete format string. If we want any different behaviour, we need to reimplement strftime for Python, which I'd be okay with, but I'm not the one to do it :) |
OK. What made me wonder is that I saw that the 'invalid format string' exception was removed by the patch...I guess that is also raised at a higher level in the code. |
It's raised by the existing handling for EINVAL at the end of the function. Previously we'd crash before getting that far because of the invalid parameter handler. |
New changeset 254b8e68959e by Steve Dower in branch 'default': |
That handles Python 3.5 and future versions, even if the supported formats change. Is there something that needs fixing in 3.4? I don't get a crash, just the ValueError... |
timemodule.c no longer compiles on MacOSX: gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -I./Include -I. -IInclude -I/usr/local/include -I/Users/yury/dev/py/cpython/Include -I/Users/yury/dev/py/cpython -c /Users/yury/dev/py/cpython/Modules/timemodule.c -o build/temp.macosx-10.10-x86_64-3.5/Users/yury/dev/py/cpython/Modules/timemodule.o |
That change was in for beta 1, so we would have noticed if we didn't get Mac builds. Something else has changed, probably some headers. |
Yury, I'm not seeing that compile error with current head of default on OS X. Try a clean build, perhaps? |
This exact sequence of commands $ make clean
$ ./configure
$ make -j8 does not build. |
$ hg status
shows nothing, branch is default (but 3.5 doesn't get built either) etc. |
I think you have a Python installed in /usr/local that is interfering. |
But why was it building just fine before this commit? I haven't updated my system packages in a while. |
Those macros are only included if Py_BUILD_CORE is defined, regardless of platform (see Include/pyport.h). Is it possible that's being undefined somehow? |
Wild guess: perhaps you did a ./configure or the Makefile did an implicit call to configure recently and/or you did a make install (to /usr/local) before? |
I don't have 'python' in /usr/local and /usr/local/bin |
But do you have any Python header files in /usr/local/include? The gcc command you pasted shows -I/usr/local/include? Mine don't show that. |
yury@ysmac ~/dev/py/cpython (HG: default?) $ ls /usr/local/include/ |
FWIW, I think that in order to use _Py_BEGIN_SUPPRESS_IPH timemodule.c should be compiled with PY_CORE_CFLAGS, and that should be reflected in the Makefile. |
Looking at a I assumed that all core files were already being compiled with Py_BUILD_CORE (they certainly are for Windows), so this seems to be an oversight for timemodule.c. |
Steve, maybe you can surround "_Py_BEGIN_SUPPRESS_IPH" with "#ifdef Py_BUILD_CORE"? |
Yury, another (less) wild guess: do you have an out-of-date Modules/Setup or Setup.local? timemodule is defined in Setup.dist but that will be overridden by a locally modified copy in the Modules directory. Towards the end of the configured top-level Makefile, you should see: # Rules appended by makedepend [...] Modules/timemodule.o: |
When do we build timemodule.c outside of core? |
Yes, I don't see that line. What should I do to regenerate it? And another question: what did go wrong with my checkout? |
Check the times and contents on all of your Modules/Setup* files. Try deleting Setup.local for one. |
Though %f is a valid format from Python's doc https://docs.python.org/2/library/datetime.html, the fix just ignores it on Windows? can we atleast get milliseconds on Windows and Micro on Linux? %f Microsecond as a decimal number, zero-padded on the left. 000000, 000001, ..., 999999 (4) %f is an extension to the set of format characters in the C standard (but implemented separately in datetime objects, and therefore always available). When used with the strptime() method, the %f directive accepts from one to six digits and zero pads on the right. New in version 2.6. |
I just made a clean checkout and that helped. I have no idea what caused this. Thank you, Ned, for troubleshooting this with me! |
Note that when I run into build problems after an update, I generally run 'make distclean' and then redo the configure/make. This generally cleans up any problems like this (and I don't find that I need to do it very often.) |
Yes, but I believe that won't help with changed Modules/Setup* files and, because it is needed in the core interpreter executable, timemodule.c is supposed to be built using Setup and not setup.py. Setup and Setup.local are designed to be locally modified and thus not revision controlled nor cleaned by make clean: $ cat Modules/Setup.dist
# The file Setup is used by the makesetup script to construct the files
# Makefile and config.c, from Makefile.pre and config.c.in,
# respectively. The file Setup itself is initially copied from
# Setup.dist; once it exists it will not be overwritten, so you can edit
# Setup to your heart's content. Note that Makefile.pre is created
# from Makefile.pre.in by the toplevel configure script.
[...]
$ hg stat --all Modules
I Modules/Setup
I Modules/Setup.config
I Modules/Setup.local |
Right, that's why I said make *dist*clean. That does delete any existing Setup file(s), which is what you want when working with a checkout for Python development purposes. |
So you did, sorry! Another, perhaps evan more reliable option is (requires enabling the purge extension in hg): hg purge --all |
Finally got back to looking at this, and since %f works against MSVC 14.0 I'm just going to remove the part of the test that is currently failing and close this issue. |
New changeset 2c10e9f62613 by Steve Dower in branch '3.5': New changeset f0ca1fabb41f by Steve Dower in branch 'default': |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: