Index: Misc/NEWS =================================================================== --- Misc/NEWS (revision 86534) +++ Misc/NEWS (working copy) @@ -163,6 +163,19 @@ encoding, instead of the locale encoding. Patch written by Alexander Belopolsky. +- Issue #2001: New html server with enhanced web page features. + * A ``-b`` option to start a enhanced browsing session. + * Allow ``-b`` and ``-p`` options to be used together. + * Specifying port 0, will pick an arbitrary unused socket port. + * A new ``browse()`` function to to start the new server and browser. + * Show python version information in the header. + * A *Get* field which takes the same input as the ``help()`` function. + * A *Search* field which replaces the tkinter search box. + * HTML links to *Module Index*, *Topics*, and *Keywords*. + * Improved source file viewing. + * A ``HTMLDoc.filelink()`` method. + * The ``-g`` option, ``gui()``, and ``serve()`` functions are deprecated. + - Issue #10126: Fix distutils' test_build when Python was built with --enable-shared. Index: Misc/ACKS =================================================================== --- Misc/ACKS (revision 86534) +++ Misc/ACKS (working copy) @@ -12,6 +12,7 @@ and the list is in rough alphabetical order by last names. David Abrahams +Ron Adam Jim Ahlstrom Farhan Ahmad Matthew Ahrens Index: Doc/whatsnew/3.2.rst =================================================================== --- Doc/whatsnew/3.2.rst (revision 86534) +++ Doc/whatsnew/3.2.rst (working copy) @@ -553,7 +553,13 @@ (Contributed by R. David Murray, :issue:`10321`.) +* The :mod:`pydoc` module now provides a much improved web server interface, + as well as a new command-line option to automatically open a browser + window to display that server. + + (Contributed by Ron Adam; :issue:`2001`.) + Multi-threading =============== Index: Doc/library/pydoc.rst =================================================================== --- Doc/library/pydoc.rst (revision 86534) +++ Doc/library/pydoc.rst (working copy) @@ -50,13 +50,21 @@ module is the first line of its documentation string. You can also use :program:`pydoc` to start an HTTP server on the local machine -that will serve documentation to visiting Web browsers. :program:`pydoc -p 1234` -will start a HTTP server on port 1234, allowing you to browse -the documentation at ``http://localhost:1234/`` in your preferred Web browser. +that will serve documentation to visiting Web browsers. :program:`pydoc -p 1234` +will start a HTTP server on port 1234, allowing you to browse the +documentation at ``http://localhost:1234/`` in your preferred Web browser. +Specifying ``0`` as the port number will select an arbitrary unused port. + :program:`pydoc -g` will start the server and additionally bring up a small :mod:`tkinter`\ -based graphical interface to help you search for -documentation pages. +documentation pages. (The ``-g`` option is deprecated.) +:program:`pydoc -b` will start the server and additionally open a web +browser to a module index page. Each served page has a navigation bar at the +top where you can *Get* help on an individual item, *Search* all modules with a +keyword in their synopsis line, and goto to the *Module index*, *Topics*, and +*Keywords* pages. + When :program:`pydoc` generates documentation, it uses the current environment and path to locate modules. Thus, invoking :program:`pydoc spam` documents precisely the version of the module you would get if you started the @@ -69,3 +77,4 @@ to a different URL or to a local directory containing the Library Reference Manual pages. +.. versionchanged:: 3.2 Index: Lib/pydoc.py =================================================================== --- Lib/pydoc.py (revision 86534) +++ Lib/pydoc.py (working copy) @@ -15,11 +15,16 @@ Run "pydoc -k " to search for a keyword in the synopsis lines of all available modules. -Run "pydoc -p " to start an HTTP server on a given port on the -local machine to generate documentation web pages. +Run "pydoc -p " to start an HTTP server on the given port on the +local machine. Port number 0 can be used to get an arbitrary unused port. +Run "pydoc -b" to start an HTTP server on an arbitrary unused port and +open a web browser to interactively browse documentation. The -p option +can be used with the -b option to explicitly specify the server port. + For platforms without a command line, "pydoc -g" starts the HTTP server and also pops up a little window for controlling it. +(The "-g" option is deprecated.) Run "pydoc -w " to write out the HTML documentation for a module to a file named ".html". @@ -51,10 +56,21 @@ # the current directory is changed with os.chdir(), an incorrect # path will be displayed. -import sys, imp, os, re, inspect, builtins, pkgutil +import sys +import imp +import os +import re +import inspect +import builtins +import pkgutil +import time +import warnings +import platform from reprlib import Repr from traceback import extract_tb as _extract_tb from collections import deque + + # --------------------------------------------------------- common routines def pathdirs(): @@ -417,6 +433,7 @@ Python: %s + %s ''' % (title, contents) @@ -512,6 +529,10 @@ else: text = name return '%s' % (url, text) + + def filelink(self, url, path): + """Make a link to source file.""" + return '%s' % (url, path) def markup(self, text, escape=None, funcs={}, classes={}, methods={}): """Mark up some plain text, given a context of symbols to look for. @@ -591,7 +612,7 @@ if sys.platform == 'win32': import nturl2path url = nturl2path.pathname2url(path) - filelink = '%s' % (url, path) + filelink = self.filelink(url, path) except TypeError: filelink = '(built-in)' info = [] @@ -1843,6 +1864,34 @@ 'Related help topics: ' + ', '.join(xrefs.split()) + '\n') self.output.write('\n%s\n' % buffer.getvalue()) + def _gettopic(self, topic, more_xrefs=''): + """ Returns unbuffered tuple of (topic, xrefs). + + If an error occurs, topic is the error message, and xrefs is ''. + This function duplicates the showtopic method but returns it's + result directly so it can be formatted for display in an html page. + """ + try: + import pydoc_data.topics + except ImportError: + return(''' +Sorry, topic and keyword documentation is not available because the +module "pydoc_data.topics" could not be found. +''' , '') + target = self.topics.get(topic, self.keywords.get(topic)) + if not target: + return 'no documentation found for %s' % repr(topic), '' + if type(target) is type(''): + return self._gettopic(target, more_xrefs) + label, xrefs = target + try: + doc = pydoc_data.topics.topics[label] + except KeyError: + return 'no documentation found for %s' % repr(topic), '' + if more_xrefs: + xrefs = (xrefs or '') + ' ' + more_xrefs + return doc, xrefs + def showsymbol(self, symbol): target = self.symbols[symbol] topic, _, xrefs = target.partition(' ') @@ -1924,6 +1973,15 @@ for importer, modname, ispkg in pkgutil.walk_packages(onerror=onerror): if self.quit: break + + # XXX Skipping this file is a get-around for a bug + # that causes python to crash with a segfault. + # http://bugs.python.org/issue9319 + # + # TODO: Remove the this once the bug is fixed. + if modname == "test.badsyntax_pep3120": + continue + if key is None: callback(None, modname, '') else: @@ -1979,6 +2037,9 @@ def serve(port, callback=None, completer=None): import http.server, email.message, select + msg = 'the pydoc.serve() function is deprecated' + warnings.warn(msg, DeprecationWarning, stacklevel=2) + class DocHandler(http.server.BaseHTTPRequestHandler): def send_document(self, title, contents): try: @@ -2058,6 +2119,11 @@ def gui(): """Graphical interface (starts web server and pops up a control window).""" + + msg = ('the pydoc.gui() function and "pydoc -g" option are deprecated, ' + 'use "pydoc.browse() function and "pydoc -b" option instead.') + warnings.warn(msg, DeprecationWarning, stacklevel=2) + class GUI: def __init__(self, window, port=7464): self.window = window @@ -2236,7 +2302,396 @@ root.destroy() except KeyboardInterrupt: pass + + +# --------------------------------------- enhanced web browser interface +def _startserver(urlhandler, port): + """ Start an HTTP server thread on a specific port. + + Start an HTML/text server thread, so HTML or text documents can be + browsed dynamically and interactively with a web browser. + + Example use + =========== + + >>> import time + >>> import pydoc + + Define a URL handler. To determine what the client is asking + for, check the URL and content_type. + + Then get or generate some text or HTML code and return it. + + >>> def my_url_handler(url, content_type): + ... text = 'the URL sent was: (%s, %s)' % (url, content_type) + ... return text + + Start server thread on port 0. + If you use port 0, the server will pick a random port number. + You can then use serverthread.port to get the port number. + + >>> port = 0 + >>> serverthread = pydoc._startserver(my_url_handler, port) + + Check that the server is really started. If it is, open browser + and get first page. Use serverthread.url as the starting page. + + >>> if serverthread.serving: + ... import webbrowser + + The next two lines are commented out so a browser doesn't open if + doctest is run on this module. + + #... webbrowser.open(serverthread.url) + #True + + Let the server do it's thing. We just need to monitor its status. + Use time.sleep so the loop doesn't hog the CPU. + + >>> starttime = time.time() + >>> timeout = 1 #seconds + + This is a short timeout for testing purposes. + + >>> while serverthread.serving: + ... time.sleep(.01) + ... if serverthread.serving and time.time() - starttime > timeout: + ... serverthread.stop() + ... break + + Print any errors that may have occurred. + + >>> print(serverthread.error) + None + + """ + import http.server + import email.message + import select + import threading + + class DocHandler(http.server.BaseHTTPRequestHandler): + """ Handle server requests from browser. """ + def do_GET(self): + """ Process a request from a HTML browser. + The URL received is in self.path. + Get an HTML page from self.urlhandler and send it. + """ + if self.path.endswith('.css'): + content_type = 'text/css' + else: + content_type = 'text/html' + self.send_response(200) + self.send_header('Content-Type', content_type + ';chrset=UTF-8') + self.end_headers() + self.wfile.write(bytes(self.urlhandler(self.path, content_type), + 'UTF-8')) + + def log_message(self, *args): + # Don't log messages. + pass + + class DocServer(http.server.HTTPServer): + def __init__(self, port, callback): + self.host = (sys.platform == 'mac') and '127.0.0.1' or 'localhost' + self.address = ('', port) + self.callback = callback + self.base.__init__(self, self.address, self.handler) + self.quit = False + + def serve_until_quit(self): + while not self.quit: + rd, wr, ex = select.select([self.socket.fileno()], [], [], 1) + if rd: + self.handle_request() + + def server_activate(self): + self.base.server_activate(self) + if self.callback: + self.callback(self) + + class ServerThread(threading.Thread): + """ Use to start the server as a thread in an application. """ + def __init__(self, urlhandler, port): + self.urlhandler = urlhandler + self.port = int(port) + threading.Thread.__init__(self) + self.serving = False + self.error = None + + def run(self): + """ Start the server. """ + try: + DocServer.base = http.server.HTTPServer + DocServer.handler = DocHandler + DocHandler.MessageClass = email.message.Message + DocHandler.urlhandler = staticmethod(self.urlhandler) + docsvr = DocServer(self.port, self.ready) + self.docserver = docsvr + docsvr.serve_until_quit() + except Exception as e: + self.error = e + + def ready(self, server): + self.serving = True + self.host = server.host + self.port = server.server_port + self.url = 'http://%s:%d/' % (self.host, self.port) + + def stop(self): + """ Stop the server and this thread nicely """ + self.docserver.quit = True + self.serving = False + self.url = None + + thread = ServerThread(urlhandler, port) + thread.start() + # Wait until thread.serving is True to make sure we are + # really up before returning. + while not thread.error and not thread.serving: + time.sleep(.01) + return thread + +def browse(port=0, *, open_browser=True): + """ Start the enhanced pydoc web server and open a web browser. + """ + import webbrowser + + class _HTMLDoc(HTMLDoc): + def filelink(self, url, path): + return '%s' % (url, path) + + html = _HTMLDoc() + + def html_navbar(): + version = "%s [%s, %s]" % ( + platform.python_version(), + platform.python_build()[0], + platform.python_compiler()) + return """ +
+ Python %s
%s

+
+
+ +
+
+ + +     +
+
+ + +
+
+
+
 
+ """ % (version, platform.platform(terse=True)) + + def html_index(): + """ Module Index web page. """ + def bltinlink(name): + return '%s' % (name, name) + heading = html.heading( + 'Index of Modules', + '#ffffff', '#7799ee') + names = list(filter(lambda x: x != '__main__', + sys.builtin_module_names)) + contents = html.multicolumn(names, bltinlink) + indices = ['

' + html.bigsection( + 'Built-in Modules', '#ffffff', '#ee77aa', contents)] + seen = {} + for dir in sys.path: + indices.append(html.index(dir, seen)) + contents = heading + ''.join(indices) + \ + '''

+ pydoc by Ka-Ping Yee <ping@lfw.org>''' + return html.page('Index of Modules' ,contents) + + def html_search(key): + """ Search results page. """ + # scan for modules + search_result = [] + def callback(path, modname, desc): + if modname[-9:] == '.__init__': + modname = modname[:-9] + ' (package)' + search_result.append((modname, desc and '- ' + desc)) + try: + import warnings + except ImportError: + pass + else: + warnings.filterwarnings('ignore') # ignore problems during import + ModuleScanner().run(callback, key) + # format page + def bltinlink(name): + return '%s' % (name, name) + results = [] + heading = html.heading( + 'Search Results', + '#ffffff', '#7799ee') + for name, desc in search_result: + results.append(bltinlink(name) + desc) + contents = heading + html.bigsection( + 'key = %s' % key, '#ffffff', '#ee77aa', '
'.join(results)) + return html.page('Search Results', contents) + + def html_getfile(path): + """ Get and display a source file listing safely. """ + path = os.sep + path.replace('%20', ' ') + with open(path, 'r') as fp: + lines = html.escape(fp.read()) + body = '

%s
' % lines + heading = html.heading( + 'File Listing', + '#ffffff', '#7799ee') + contents = heading + html.bigsection( + 'File: %s' % path, '#ffffff', '#ee77aa', body) + return html.page('getfile: %s' % path, contents) + + def html_topics(): + """ Index of topic texts available. """ + def bltinlink(name): + return '%s' % (name, name) + heading = html.heading( + 'INDEX', + '#ffffff', '#7799ee') + names = sorted(Helper.topics.keys()) + def bltinlink(name): + return '%s' % (name, name) + contents = html.multicolumn(names, bltinlink) + contents = heading + html.bigsection( + 'Topics', '#ffffff', '#ee77aa', contents) + return html.page('Topics', contents) + + def html_keywords(): + """ Index of keywords. """ + heading = html.heading( + 'INDEX', + '#ffffff', '#7799ee') + names = sorted(Helper.keywords.keys()) + def bltinlink(name): + return '%s' % (name, name) + contents = html.multicolumn(names, bltinlink) + contents = heading + html.bigsection( + 'Keywords', '#ffffff', '#ee77aa', contents) + return html.page('Keywords', contents) + + def html_topicpage(topic): + """ Topic or keyword help page. """ + import io + buf = io.StringIO() + htmlhelp = Helper(buf, buf) + contents, xrefs = htmlhelp._gettopic(topic) + if topic in htmlhelp.keywords: + title = 'KEYWORD' + else: + title = 'TOPIC' + heading = html.heading( + '%s' % title, + '#ffffff', '#7799ee') + contents = '
%s
' % contents + contents = html.bigsection(topic , '#ffffff','#ee77aa', contents) + xrefs = sorted(xrefs.split()) + def bltinlink(name): + return '%s' % (name, name) + xrefs = html.multicolumn(xrefs, bltinlink) + xrefs = html.section('Related help topics: ', '#ffffff', '#ee77aa', xrefs) + return html.page('%s: %s' % (title, topic), heading + contents + xrefs) + + def html_error(url): + heading = html.heading( + 'Error', + '#ffffff', '#ee0000') + return heading + url + + def get_html_page(url): + """ Function url handler uses to get the html page to get + depending on the url. + """ + if url[-5:] == '.html': + url = url[:-5] + if url[:1] == '/': + url = url[1:] + if url.startswith("get?key="): + url = url[8:] + title = url + contents = '' + if url in ("", ".", "index"): + contents = html_index() + elif url == "topics": + contents = html_topics() + elif url == "keywords": + contents = html_keywords() + elif url.startswith("search?key="): + contents = html_search(url[11:]) + elif url.startswith("getfile?key="): + url = url[12:] + try: + contents = html_getfile(url) + except IOError: + contents = html_error('could not read file %s' % repr(url)) + else: + obj = None + try: + obj = locate(url, forceload=1) + except ErrorDuringImport as value: + contents = html.escape(str(value)) + if obj: + title = describe(obj) + contents = html.document(obj, url) + elif url in Helper.keywords or url in Helper.topics: + contents = html_topicpage(url) + else: + contents = html_error('no Python documentation found for %s' + % repr(url)) + return html.page(title, html_navbar() + contents) + + def url_handler(url, content_type): + """ html server html and style sheet requests. """ + if url.startswith('/'): + url = url[1:] + if content_type == 'text/css': + path_here = os.path.dirname(os.path.realpath(__file__)) + with open(os.path.join(path_here, url)) as fp: + return ''.join(fp.readlines()) + elif content_type == 'text/html': + return get_html_page(url) + return 'Error: unknown content type ' + content_type + + serverthread = _startserver(url_handler, port) + if serverthread.error: + print(serverthread.error) + return + if serverthread.serving: + server_help_msg = """Python version: %s +Server ready at: %s +Server commands: [b]rowser, [q]uit'""" % (sys.version, serverthread.url) + if open_browser: + webbrowser.open(serverthread.url) + try: + print(server_help_msg) + while serverthread.serving: + cmd = input('server>') + cmd = cmd.lower() + if cmd == 'q': + break + elif cmd == 'b': + webbrowser.open(serverthread.url) + else: + print(server_help_msg) + finally: + if serverthread.serving: + serverthread.stop() + print('Server stopped') + + # -------------------------------------------------- command-line interface def ispath(x): @@ -2256,29 +2711,32 @@ sys.path.insert(0, '.') try: - opts, args = getopt.getopt(sys.argv[1:], 'gk:p:w') - writing = 0 - + opts, args = getopt.getopt(sys.argv[1:], 'bgk:p:w') + writing = False + start_server = False + open_browser = False + port = None for opt, val in opts: if opt == '-g': gui() return + if opt == '-b': + start_server = True + open_browser = True if opt == '-k': apropos(val) return if opt == '-p': - try: - port = int(val) - except ValueError: - raise BadUsage - def ready(server): - print('pydoc server ready at %s' % server.url) - def stopped(): - print('pydoc server stopped') - serve(port, ready, stopped) - return + start_server = True + port = val if opt == '-w': - writing = 1 + writing = True + + if start_server == True: + if port == None: + port = 0 + browse(port, open_browser=open_browser) + return if not args: raise BadUsage for arg in args: @@ -2302,27 +2760,35 @@ cmd = os.path.basename(sys.argv[0]) print("""pydoc - the Python documentation tool -%s ... +{cmd} ... Show text documentation on something. may be the name of a Python keyword, topic, function, module, or package, or a dotted reference to a class or function within a module or module in a - package. If contains a '%s', it is used as the path to a + package. If contains a '{sep}', it is used as the path to a Python source file to document. If name is 'keywords', 'topics', or 'modules', a listing of these things is displayed. -%s -k +{cmd} -k Search for a keyword in the synopsis lines of all available modules. -%s -p - Start an HTTP server on the given port on the local machine. +{cmd} -p + Start an HTTP server on the given port on the local machine. Port + number 0 can be used to get an arbitrary unused port. -%s -g - Pop up a graphical interface for finding and serving documentation. +{cmd} -b + Start an HTTP server on an arbitrary unused port and open a web browser + to interactively browse documentation. The -p option can be used with + the -b option to explicitly specify the server port. + +{cmd} -g + The "-g" option is deprecated. -%s -w ... +{cmd} -w ... Write out the HTML documentation for a module to a file in the current - directory. If contains a '%s', it is treated as a filename; if + directory. If contains a '{sep}', it is treated as a filename; if it names a directory, documentation is written for all the contents. -""" % (cmd, os.sep, cmd, cmd, cmd, cmd, os.sep)) +""".format(cmd=cmd, sep=os.sep)) -if __name__ == '__main__': cli() +if __name__ == '__main__': + cli() + Index: Lib/_pydoc.css ===================================================================