classification
Title: calendar.prcal(9999) output has a problem
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.7, Python 3.6, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder: Make Calendar.itermonthdates() behave consistently in edge cases
View: 28292
Assigned To: belopolsky Nosy List: belopolsky, jiangping.li, python-dev, rhettinger, serhiy.storchaka, xiang.zhang
Priority: normal Keywords: patch

Created on 2016-09-23 05:59 by jiangping.li, last changed 2017-10-23 21:21 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
20160923154147.png jiangping.li, 2016-09-23 06:48 output result
calendar_pryear_9999.patch xiang.zhang, 2016-09-23 08:02 review
calendar_prcal_9999_demo.patch xiang.zhang, 2016-09-23 13:13 review
issue28253.diff belopolsky, 2016-09-26 20:26 review
issue28253-2.diff belopolsky, 2016-09-26 22:28 review
issue28253-3.diff belopolsky, 2016-09-26 22:37 review
issue28253-4.diff belopolsky, 2016-09-27 17:56 review
Messages (29)
msg277244 - (view) Author: Jiangping Li (jiangping.li) Date: 2016-09-23 05:59
hi
I`m a python newer.
when I use python3.4.3 to learn about package calendar,I found a problem, Could you help me to confirm it which is is a bug.

Below is my env:
------------------------------
os: win7 x86
python version:3.4.3
------------------------------
problem:
when I use calendar.prcal() method to print detail calendar information. the display December of year 9999 `s localtion is wrong.
my python source:
--------------------------------------
import calendar
calendar.prcal(9999)
--------------------------------------

partial output:

but,when I use [print (calendar.month(9999,12))] to confirm, it`s OK .

please help me to confirm.
thank you.
msg277246 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-09-23 08:00
Thank you for your report lijp. This is interesting issue. It is not specific for Windows.

itermonthdates() yields shortened sequence of dates at the end of maximal year. The formatted row of day numbers is centered. Since the number of days is less than expected, additional spaces are added at the left.
msg277247 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2016-09-23 08:02
calendar internally relies on datetime and uses datetime.date. datetime.date can't represent date more than 9999.12.31 so some logic is broken for it(normally the week list should be of length 7, date outside the month will be (0, x) pair but this is not the case for 9999.12). This won't affect prmonth since it doesn't do center formatting. We can change to that behaviour to fix this. calendar_pryear_9999 tires to fix this.
msg277250 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-09-23 08:15
Html calendar formats the last week differently for years 9982 and 9999.

Year 9982:
<tr><td class="mon">27</td><td class="tue">28</td><td class="wed">29</td><td class="thu">30</td><td class="fri">31</td><td class="noday">&nbsp;</td><td class="noday">&nbsp;</td></tr>

Year 9999:
<tr><td class="mon">27</td><td class="tue">28</td><td class="wed">29</td><td class="thu">30</td><td class="fri">31</td></tr>

I think this issue should be fixed on lower level than text calendar formatter.
msg277253 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2016-09-23 08:28
Something like this?

diff -r 15f82b64eee0 Lib/calendar.py
--- a/Lib/calendar.py	Thu Sep 22 17:11:53 2016 -0700
+++ b/Lib/calendar.py	Fri Sep 23 16:27:03 2016 +0800
@@ -181,6 +181,9 @@
                 yield (0, date.weekday())
             else:
                 yield (date.day, date.weekday())
+                if year == 9999 and month == 12 and date.day == 31:
+                    yield (0, 6)
+                    yield (0, 7)
 
     def itermonthdays(self, year, month):
         """

Is there a more elegant way? I considered this but didn't quite like it so abandoned it.
msg277257 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-09-23 08:51
This is not just not elegant, but it doesn't work if firstweekday is not 0.

The lowest level in the calendar module is itermonthdates(). But the problem is that dates outside supported range can't be represented.

While the output for December of the year 9999 looks badly formatted, calendar doesn't work at all with January of the year 1 if firstweekday is not 0:

>>> import calendar
>>> calendar.setfirstweekday(6)
>>> calendar.prmonth(1, 1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/serhiy/py/cpython-3.6/Lib/calendar.py", line 318, in prmonth
    print(self.formatmonth(theyear, themonth, w, l), end=' ')
  File "/home/serhiy/py/cpython-3.6/Lib/calendar.py", line 331, in formatmonth
    for week in self.monthdays2calendar(theyear, themonth):
  File "/home/serhiy/py/cpython-3.6/Lib/calendar.py", line 211, in monthdays2calendar
    days = list(self.itermonthdays2(year, month))
  File "/home/serhiy/py/cpython-3.6/Lib/calendar.py", line 179, in itermonthdays2
    for date in self.itermonthdates(year, month):
  File "/home/serhiy/py/cpython-3.6/Lib/calendar.py", line 162, in itermonthdates
    date -= datetime.timedelta(days=days)
OverflowError: date value out of range
msg277275 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2016-09-23 13:13
How about the approach in calendar_prcal_9999_demo.patch? If it's not bad I can add tests then.
msg277315 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-09-24 12:26
Having additional tests is always nice. After writing tests we can search whether there is other solution.

AFAIK the dummy data needs also the day attribute.
msg277316 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-09-24 12:31
The problem with year 1 was reported in issue26650.
msg277433 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2016-09-26 16:54
I would leave itermonthdates() alone and just fix itermonthdays2() (and itermonthdays() for consistency) as Xiang suggested.  The fix can be implemented by breaking on date.month != month and adding something like

for wd in range(date.weekday(), 7):
    yield (0, wd)

after the existing for loop.
msg277435 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2016-09-26 17:16
On the second thought, I don't see why itermonthdays2() and itermonthdays() need to use itermonthdates() at all.  It looks like it is easy to implement those using monthrange() and some simple integer arithmetics.
msg277436 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-09-26 17:31
itermonthdates() is documented public method. We should do something with it. Maybe emitting dummy data instances is the simplest way to solve this issue.
msg277446 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2016-09-26 18:33
Note that the stop on date.max behavior was introduced in #15421.
msg277447 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2016-09-26 18:34
> itermonthdates() is documented public method

The current documentation is an impossibility:

"The iterator will yield datetime.date values and will always iterate through complete weeks, so it will yield dates outside the specified
month."

The current implementation deals with this impossibility differently for months (9999, 12) and (1, 1).  In the first case, the iterators stops on an out of bounds date:

>>> list(calendar.Calendar().itermonthdates(9999, 12))[-1]
datetime.date(9999, 12, 31)
>>> list(calendar.Calendar().itermonthdates(9999, 12))[-1].weekday()
4

but in the second, it raises the OverflowError:

>>> next(calendar.Calendar(1).itermonthdates(1, 1))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "calendar.py", line 160, in itermonthdates
    date -= datetime.timedelta(days=days)
OverflowError: date value out of range

Returning dummy instances instead of datetime.date in these cases will only make debugging harder for the users of .itermonthdates().  Sooner or later they would want to do something the returned value that the dummy won't support.  If you are willing to sacrifice the "will yield datetime.date values" for "will always iterate through complete weeks", I would make it yield None for out of bounds values and require the caller to deal with this possibility right away.

A better solution would be to simply raise OverflowError whenever the range of itermonthdates() does not fit within [date.min, date.max].
msg277452 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-09-26 19:43
Yes, the current documentation is an impossibility (unless we remove all date limits that is much harder issue). Raisin OverflowError will make the implementation of itermonthdays2() and itermonthdays() more complex. Yielding dummy instances or None requires rewriting user code (less with a dummy instance if we are lucky). I agree that in long perspective yielding None looks better.
msg277464 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2016-09-26 20:26
I am attaching the proposed reimplementation of itermonthdays2() and itermonthdays().  It does not use itermonthdates() and is not that complicated.  It passes test_calendar, but I did not test it further than that.

I would rather not mess up with itermonthdates(), particularly in a bugfix release.  We can postpone the discussion of a better way to handle date over/underflow in itermonthdates() until 3.7.
msg277468 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2016-09-26 20:46
> unless we remove all date limits that is much harder issue

I don't think this is too hard.  I think the original implementation did not have date limits.  I've opened a separate issue for this.  See #28281.
msg277474 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2016-09-26 22:28
issue28253-2.diff is a small performance improvement over issue28253.diff
msg277475 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2016-09-26 22:37
issue28253-3.diff uses itertools.repeat().
msg277476 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2016-09-26 23:20
issue28253-4.diff is issue28253-3.diff with tests.
msg277484 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2016-09-27 04:07
Patch LGTM. 

> I would rather not mess up with itermonthdates(), particularly in a bugfix release.  We can postpone the discussion of a better way to handle date over/underflow in itermonthdates() until 3.7.

Before finally find a better way, can we at least make the two extreme cases behaviours consistent? Both emitting an exception or a shorter list.
msg277489 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-09-27 05:09
The patch LGTM except tests.

But we should at least document the behavior of itermonthdates(), monthdatescalendar() and yeardatescalendar() at corner cases.
msg277523 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2016-09-27 16:48
> The patch LGTM except tests.

What are your issues with the tests?  Did you see the -4 patch?

> But we should at least document the behavior of itermonthdates(), monthdatescalendar() and yeardatescalendar() at corner cases.

I would rather not. At least not before we make under and overflow behavior consistent.  Frankly, I don't see why anyone would want to use these iterators.
msg277530 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2016-09-27 17:42
Something went wrong with issue28253-4.diff.  I'll investigate and replace.
msg277532 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2016-09-27 17:58
issue28253-4.diff should be good now.
msg277539 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-09-27 18:44
The patch LGTM.

You can see how itermonthdates() is used in third-party code:

https://github.com/sunlightlabs/django-locksmith/blob/master/locksmith/hub/dataviews.py
https://github.com/quandyfactory/Quandy/blob/master/quandy.py
https://github.com/takanory/plone.app.event/blob/master/plone/app/event/portlets/portlet_calendar.py
https://github.com/gerow/gnome-shell-google-calendar/blob/master/gnome-shell-google-calendar.py
https://bitbucket.org/benallard/msgboard/src/1c08fa3ba040f8151d0e28130b01b30e0595e448/msgboard/controller.py?at=default
msg277567 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-09-28 00:29
New changeset c439bce36bf2 by Alexander Belopolsky in branch '3.5':
Issue #28253: Fixed calendar functions for extreme months: 0001-01 and 9999-12.
https://hg.python.org/cpython/rev/c439bce36bf2

New changeset cd384c4b441a by Alexander Belopolsky in branch '3.6':
Issue #28253: Fixed calendar functions for extreme months: 0001-01 and 9999-12.
https://hg.python.org/cpython/rev/cd384c4b441a

New changeset bc285a9ecc58 by Alexander Belopolsky in branch 'default':
Issue #28253: Fixed calendar functions for extreme months: 0001-01 and 9999-12.
https://hg.python.org/cpython/rev/bc285a9ecc58
msg277568 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-09-28 00:34
New changeset 7efba48299e9 by Alexander Belopolsky in branch 'default':
Issue #28253: Added a NEWS entry.
https://hg.python.org/cpython/rev/7efba48299e9

New changeset 55f11196c949 by Alexander Belopolsky in branch '3.5':
Issue #28253: Added a NEWS entry.
https://hg.python.org/cpython/rev/55f11196c949

New changeset 1f1a085f533f by Alexander Belopolsky in branch '3.6':
Issue #28253: Added a NEWS entry.
https://hg.python.org/cpython/rev/1f1a085f533f
msg277575 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-09-28 02:45
New changeset f2aff898f7c8 by Alexander Belopolsky in branch '2.7':
Issue #28253: Fixed calendar functions for extreme months: 0001-01 and 9999-12.
https://hg.python.org/cpython/rev/f2aff898f7c8
History
Date User Action Args
2017-10-23 21:21:44serhiy.storchakasetpull_requests: - pull_request982
2017-03-31 16:36:24dstufftsetpull_requests: + pull_request982
2016-09-28 03:05:06belopolskylinkissue28292 dependencies
2016-09-28 03:02:52belopolskysetstatus: open -> closed
superseder: Make Calendar.itermonthdates() behave consistently in edge cases
resolution: fixed
stage: commit review -> resolved
2016-09-28 02:45:13python-devsetmessages: + msg277575
2016-09-28 00:34:25python-devsetmessages: + msg277568
2016-09-28 00:29:29python-devsetnosy: + python-dev
messages: + msg277567
2016-09-27 19:06:30serhiy.storchakasetassignee: serhiy.storchaka -> belopolsky
2016-09-27 18:44:53serhiy.storchakasetmessages: + msg277539
2016-09-27 17:58:30belopolskysetmessages: + msg277532
2016-09-27 17:56:49belopolskysetfiles: - issue28253-4.diff
2016-09-27 17:56:39belopolskysetfiles: + issue28253-4.diff
2016-09-27 17:42:51belopolskysetmessages: + msg277530
2016-09-27 17:17:35belopolskylinkissue28281 dependencies
2016-09-27 16:48:48belopolskysetmessages: + msg277523
stage: commit review
2016-09-27 05:09:19serhiy.storchakasetmessages: + msg277489
2016-09-27 04:07:16xiang.zhangsetmessages: + msg277484
2016-09-26 23:20:06belopolskysetfiles: + issue28253-4.diff

messages: + msg277476
2016-09-26 22:37:58belopolskysetfiles: + issue28253-3.diff

messages: + msg277475
2016-09-26 22:28:19belopolskysetfiles: + issue28253-2.diff

messages: + msg277474
2016-09-26 20:46:14belopolskysetmessages: + msg277468
2016-09-26 20:26:28belopolskysetfiles: + issue28253.diff

messages: + msg277464
2016-09-26 19:43:13serhiy.storchakasetmessages: + msg277452
2016-09-26 18:34:26belopolskysetmessages: + msg277447
2016-09-26 18:33:11belopolskysetmessages: + msg277446
2016-09-26 17:31:34serhiy.storchakasetmessages: + msg277436
2016-09-26 17:16:57belopolskysetmessages: + msg277435
2016-09-26 16:54:46belopolskysetnosy: + belopolsky
messages: + msg277433
2016-09-24 12:31:05serhiy.storchakasetmessages: + msg277316
2016-09-24 12:26:24serhiy.storchakasetmessages: + msg277315
2016-09-23 13:13:29xiang.zhangsetfiles: + calendar_prcal_9999_demo.patch

messages: + msg277275
2016-09-23 08:51:31serhiy.storchakasetmessages: + msg277257
2016-09-23 08:28:55xiang.zhangsetmessages: + msg277253
2016-09-23 08:15:20serhiy.storchakasetmessages: + msg277250
2016-09-23 08:02:44xiang.zhangsetfiles: + calendar_pryear_9999.patch
keywords: + patch
2016-09-23 08:02:14xiang.zhangsetnosy: + xiang.zhang
messages: + msg277247
2016-09-23 08:00:33serhiy.storchakasetversions: + Python 2.7, Python 3.5, Python 3.6, Python 3.7, - Python 3.4
nosy: + rhettinger, serhiy.storchaka, - paul.moore, tim.golden, zach.ware, steve.dower

messages: + msg277246

assignee: serhiy.storchaka
components: + Library (Lib), - Windows
2016-09-23 06:48:47jiangping.lisetfiles: + 20160923154147.png
2016-09-23 06:38:38jiangping.lisettitle: the reply's additional "Re:" -> calendar.prcal(9999) output has a problem
nosy: + tim.golden, steve.dower, zach.ware, paul.moore

versions: + Python 3.4
components: + Windows
type: behavior
2016-09-23 05:59:40jiangping.licreate