classification
Title: doctest should support fixtures
Type: enhancement Stage: resolved
Components: Tests Versions: Python 3.2
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: tim.peters Nosy List: BreamoreBoy, LambertDW, belopolsky, dalloliogm, rhettinger, terry.reedy, tim.peters
Priority: normal Keywords:

Created on 2009-01-09 16:18 by dalloliogm, last changed 2014-06-27 20:11 by terry.reedy. This issue is now closed.

Messages (11)
msg79477 - (view) Author: Giovanni (dalloliogm) Date: 2009-01-09 16:18
The doctest module should have support for fixtures (e.g. setUp and
tearDown methods).

It is very silly to be forced to always re-import all the modules needed
in every function tested with doctest.
For example, when you have to document functions that open files, you
have to import StringIO or TempFile, and then create a file, while this
could be done easily with a fixture.
msg79495 - (view) Author: David W. Lambert (LambertDW) Date: 2009-01-09 19:52
I disagree.  Purpose of __doc__ is to explain functionality all at once.
This command idiom is useful:

$ python -c 'from a_module import thing; help(thing)'

The doctest module is a lightweight nicety that helps verify that which
is suitable.  The sufficiently simple algorithms of my code have doc
strings that are the complete test and explanation.  For others I
provide both docstring and unit tests.  But with many I explain the
arguments and output, possibly the algorithm in a doc string.  Tests and
use case examples reside in the module's unit test.

I'm among the "Choose correct tool for the job.  python comes with full
tool bag." group.
msg79513 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2009-01-09 21:40
I concur with David.  This is not in the spirit of the doc test module.
 We already have the heavy-weight unittest module as an alternative when
more firepower is needed.  Also, adding more infra-structure to the this
already lengthy module will make it harder to learn, use, and remember.

Tim, I recommend rejecting this request.
msg79743 - (view) Author: Giovanni (dalloliogm) Date: 2009-01-13 14:03
I was proposing to adopt doctest in the biopython project (modules for
bioinformatics in python, http://biopython.org/).

Doctest is very useful to document modules that will be used by many
other people: for example, there are many different file formats in
bioinformatics, and it is very useful to add an example of the file to
be parsed in the documentation of a file parser.
Look at my code here: 
-
http://github.com/dalloliogm/biopython---popgen/blob/980419dbc0666e2578c2486dab1fef23ccfbb72c/src/PopGen/Gio/TpedIO.py


However, it is very uncomfortable to have to create a file-like object
in every docstring, especially when you want to document the methods of
a class.
It would be useful if at least the doctests of the methods of a class
share the objects created in the main doctest of the class.

Let's say I have a class called FastaIO (fasta is a file format for
sequence).
This module would have many methods: format, to_dict (returns a
dictionary of the sequences included in the file), and many others.
The main docstring of the class will have an example of a fasta file,
and shows how to create an instance of FastaIO.
It is silly to have to repeat this example (creating an instance of
FastaIO) in every submethod. Moreover, it is more difficult to maintain
and more error prone (imagine you have a class with one hundred methods).
msg79754 - (view) Author: David W. Lambert (LambertDW) Date: 2009-01-13 17:44
My goodness, that's the starting base sequence to gene 38c, chromosome 4
of the Columbian caldera cricket!  But seriously...

1) The relevant part of the doc string is this, and this is how it
should read (your argument being "if doctests provided setUp framework
my doc string would look like this!"):

def TpedIterator(handle):

    '''
        Iterates on an TPed file handler.
        Returns Marker objects.

            Tped_stream = open('cricket.sequence','r')
            ti = TpedIterator(Tped_stream)
            for marker in ti:
                use(marker)
    '''


2) (With the caveat that I am unfamilar with your project.)  You should
choose terminology appropriate for your project.  A computer scientist
would expect "file handle" to be an integer.  What you call "handle" is
clearly a "stream" object and therefore of uncommon name.  Since the
file objects are more likely to be from the computer sciences rather
than the biological realm you should stick with "stream".

3) We agree, "Don't Repeat Yourself".  The last two chunks of your file
enable doctest.  I'll guess that similar lines may be found repeated
throughout your sources.  Instead of internal support, write a single
test script that provides external support.  It would process named
files with unittest, doctest.[, and customtests.]
    $ python -c 'import test' glob
There may be a python library for this.  I can't guide you easily
because I built and use my own framework.  Nor have I bothered to figure
out how python runs its own installation tests.

4) Yes, unittest are quite appropriate for your project.  When you move
your docstring tests to unittests try to isolate the tests.  For
instance, the test you've shown couples the TpedIterator with the string
representation of a Marker object.

5) Is your system really so simple that it's useful to run
interactively?  I may be out of touch but I script most of my codes and
tests because I make so many errors and module changes.  In other words,
is your interactive docstring example a reasonable use case?
msg79757 - (view) Author: David W. Lambert (LambertDW) Date: 2009-01-13 18:03
For unittests I recommend two things instead of need for doctest change.
A decoupled strict test to prove that the iterator works, and this class
to publish,

class Tped_use_cases(...):

    def test_Marker_iteration(self):

        '''
            Illustrative code adapted from what is now your doctest
        '''

    ...
msg79788 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2009-01-13 23:15
The OP's problem, i.e. the need to reimport modules in every docstring 
can  easily be addressed by injecting the necessary names using 
extraglobs argument to doctest.testmod().

I like the following trick:

def getextraglobs():
   import StringIO, tempfile, ...
   return locals()
doctest.testmod(extraglobs=getextraglobs())

This however does not solve a problem of cleanup after examples and some 
way to specify a tearDown fixture would be helpful, particularly because  
cleanup code in examples will rarely improve documentation.
msg113362 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2010-08-09 03:14
Tim, any opinion on whether this should be kept open or not?
msg113381 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2010-08-09 04:40
I stopped understanding doctest the last time it was rewritten - it got far more generalized than I ever intended already.  It's up to the younger generation to decide how much more inscrutable to make it now ;-)
msg221693 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-06-27 17:07
I suggest this is closed as "won't fix" as David and Raymond are against it and Tim says he no longer understands doctest.
msg221716 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-06-27 20:11
I am closing this for both the general reasons already given and the lack of a proposed api that could programmed, Hence there is no example code that would run and hence no specifics to approve. If someone wanted to pursue this, python-ideas would be a better place.

One of the general reasons given is that the example does not demonstrate a real need for the fixtures proposed. My take on the example code is this.

The input to TpedIterator is an iterable of lines. A list of lines will work as well an an open file. I would create a file of example lists and import the one needed for each docstring. For the example given.

def TpedIterator(lines):
    '''Yield Marker for each line of lines.

    >>> from biopython.sample_data import Tped
    >>> for marker in TpedIterator(Tped):
    ...    print(marker)
    Marker rs10000543, 2 individuals
    Marker rs10000929, 2 individuals
    Marker rs10002472, 2 individuals
    '''
History
Date User Action Args
2014-06-27 20:11:16terry.reedysetstatus: open -> closed
resolution: rejected
messages: + msg221716

stage: test needed -> resolved
2014-06-27 17:07:42BreamoreBoysetnosy: + BreamoreBoy
messages: + msg221693
2010-08-09 04:40:04tim.peterssetmessages: + msg113381
2010-08-09 03:14:33terry.reedysetversions: + Python 3.2, - Python 3.1, Python 2.7
nosy: + terry.reedy

messages: + msg113362

stage: test needed
2009-01-13 23:15:54belopolskysetmessages: + msg79788
2009-01-13 22:57:48belopolskysetnosy: + belopolsky
2009-01-13 18:03:59LambertDWsetmessages: + msg79757
2009-01-13 17:44:47LambertDWsetmessages: + msg79754
2009-01-13 14:03:42dalloliogmsetmessages: + msg79743
2009-01-09 21:40:27rhettingersetassignee: tim.peters
messages: + msg79513
nosy: + rhettinger, tim.peters
versions: + Python 3.1, Python 2.7
2009-01-09 19:52:59LambertDWsetnosy: + LambertDW
messages: + msg79495
2009-01-09 16:18:59dalloliogmcreate