=====================================
 An Introduction to reStructuredText
=====================================
:Author: David Goodger
:Contact: goodger@users.sourceforge.net
:Revision: $Revision: 684 $
:Date: $Date: 2005-10-06 15:45:31 -0500 (Thu, 06 Oct 2005) $
:Copyright: This document has been placed in the public domain.

reStructuredText_ is an easy-to-read, what-you-see-is-what-you-get
plaintext markup syntax and parser system.  It is useful for inline
program documentation (such as Python docstrings), for quickly
creating simple web pages, and for standalone documents.
reStructuredText_ is a proposed revision and reinterpretation of the
StructuredText_ and Setext_ lightweight markup systems.

reStructuredText is designed for extensibility for specific
application domains.  Its parser is a component of Docutils_.

This document defines the goals_ of reStructuredText and provides a
history_ of the project.  It is written using the reStructuredText
markup, and therefore serves as an example of its use.  For a gentle
introduction to using reStructuredText, please read `A
ReStructuredText Primer`_.  The `Quick reStructuredText`_ user
reference is also useful.  The `reStructuredText Markup
Specification`_ is the definitive reference.  There is also an
analysis of the `Problems With StructuredText`_.

ReStructuredText's web page is
http://docutils.sourceforge.net/rst.html.

.. _reStructuredText: http://docutils.sourceforge.net/rst.html
.. _StructuredText:
    http://dev.zope.org/Members/jim/StructuredTextWiki/FrontPage
.. _Setext: http://docutils.sourceforge.net/mirror/setext.html
.. _Docutils: http://docutils.sourceforge.net/
.. _A ReStructuredText Primer: ../../docs/rst/quickstart.html
.. _Quick reStructuredText: ../../docs/rst/quickref.html
.. _reStructuredText Markup Specification: reStructuredText.html
.. _Problems with StructuredText: problems.html


Goals
=====

The primary goal of reStructuredText_ is to define a markup syntax for
use in Python docstrings and other documentation domains, that is
readable and simple, yet powerful enough for non-trivial use.  The
intended purpose of the reStructuredText markup is twofold:

- the establishment of a set of standard conventions allowing the
  expression of structure within plaintext, and

- the conversion of such documents into useful structured data
  formats.

The secondary goal of reStructuredText is to be accepted by the Python
community (by way of being blessed by PythonLabs and the BDFL [#]_) as
a standard for Python inline documentation (possibly one of several
standards, to account for taste).

.. [#] Python's creator and "Benevolent Dictator For Life",
   Guido van Rossum.

To clarify the primary goal, here are specific design goals, in order,
beginning with the most important:

1. Readable.  The marked-up text must be easy to read without any
   prior knowledge of the markup language.  It should be as easily
   read in raw form as in processed form.

2. Unobtrusive.  The markup that is used should be as simple and
   unobtrusive as possible.  The simplicity of markup constructs
   should be roughly proportional to their frequency of use.  The most
   common constructs, with natural and obvious markup, should be the
   simplest and most unobtrusive.  Less common constructs, for which
   there is no natural or obvious markup, should be distinctive.

3. Unambiguous.  The rules for markup must not be open for
   interpretation.  For any given input, there should be one and only
   one possible output (including error output).

4. Unsurprising.  Markup constructs should not cause unexpected output
   upon processing.  As a fallback, there must be a way to prevent
   unwanted markup processing when a markup construct is used in a
   non-markup context (for example, when documenting the markup syntax
   itself).

5. Intuitive.  Markup should be as obvious and easily remembered as
   possible, for the author as well as for the reader.  Constructs
   should take their cues from such naturally occurring sources as
   plaintext email messages, newsgroup postings, and text
   documentation such as README.txt files.

6. Easy.  It should be easy to mark up text using any ordinary text
   editor.

7. Scalable.  The markup should be applicable regardless of the length
   of the text.

8. Powerful.  The markup should provide enough constructs to produce a
   reasonably rich structured document.

9. Language-neutral.  The markup should apply to multiple natural (as
   well as artificial) languages, not only English.

10. Extensible.  The markup should provide a simple syntax and
    interface for adding more complex general markup, and custom
    markup.

11. Output-format-neutral.  The markup will be appropriate for
    processing to multiple output formats, and will not be biased
    toward any particular format.

The design goals above were used as criteria for accepting or
rejecting syntax, or selecting between alternatives.

It is emphatically *not* the goal of reStructuredText to define
docstring semantics, such as docstring contents or docstring length.
These issues are orthogonal to the markup syntax and beyond the scope
of this specification.

Also, it is not the goal of reStructuredText to maintain compatibility
with StructuredText_ or Setext_.  reStructuredText shamelessly steals
their great ideas and ignores the not-so-great.

Author's note:

    Due to the nature of the problem we're trying to solve (or,
    perhaps, due to the nature of the proposed solution), the above
    goals unavoidably conflict.  I have tried to extract and distill
    the wisdom accumulated over the years in the Python Doc-SIG_
    mailing list and elsewhere, to come up with a coherent and
    consistent set of syntax rules, and the above goals by which to
    measure them.

    There will inevitably be people who disagree with my particular
    choices.  Some desire finer control over their markup, others
    prefer less.  Some are concerned with very short docstrings,
    others with full-length documents.  This specification is an
    effort to provide a reasonably rich set of markup constructs in a
    reasonably simple form, that should satisfy a reasonably large
    group of reasonable people.

    David Goodger (goodger@users.sourceforge.net), 2001-04-20

.. _Doc-SIG: http://www.python.org/sigs/doc-sig/


History
=======

reStructuredText_, the specification, is based on StructuredText_ and
Setext_.  StructuredText was developed by Jim Fulton of `Zope
Corporation`_ (formerly Digital Creations) and first released in 1996.
It is now released as a part of the open-source "Z Object Publishing
Environment" (ZOPE_).  Ian Feldman's and Tony Sanders' earlier Setext_
specification was either an influence on StructuredText or, by their
similarities, at least evidence of the correctness of this approach.

I discovered StructuredText_ in late 1999 while searching for a way to
document the Python modules in one of my projects.  Version 1.1 of
StructuredText was included in Daniel Larsson's pythondoc_.  Although
I was not able to get pythondoc to work for me, I found StructuredText
to be almost ideal for my needs.  I joined the Python Doc-SIG_
(Documentation Special Interest Group) mailing list and found an
ongoing discussion of the shortcomings of the StructuredText
"standard".  This discussion has been going on since the inception of
the mailing list in 1996, and possibly predates it.

I decided to modify the original module with my own extensions and
some suggested by the Doc-SIG members.  I soon realized that the
module was not written with extension in mind, so I embarked upon a
general reworking, including adapting it to the "re" regular
expression module (the original inspiration for the name of this
project).  Soon after I completed the modifications, I discovered that
StructuredText.py was up to version 1.23 in the ZOPE distribution.
Implementing the new syntax extensions from version 1.23 proved to be
an exercise in frustration, as the complexity of the module had become
overwhelming.

In 2000, development on StructuredTextNG_ ("Next Generation") began at
`Zope Corporation`_ (then Digital Creations).  It seems to have many
improvements, but still suffers from many of the problems of classic
StructuredText.

I decided that a complete rewrite was in order, and even started a
`reStructuredText SourceForge project`_ (now inactive).  My
motivations (the "itches" I aim to "scratch") are as follows:

- I need a standard format for inline documentation of the programs I
  write.  This inline documentation has to be convertible to other
  useful formats, such as HTML.  I believe many others have the same
  need.

- I believe in the Setext/StructuredText idea and want to help
  formalize the standard.  However, I feel the current specifications
  and implementations have flaws that desperately need fixing.

- reStructuredText could form part of the foundation for a
  documentation extraction and processing system, greatly benefitting
  Python.  But it is only a part, not the whole.  reStructuredText is
  a markup language specification and a reference parser
  implementation, but it does not aspire to be the entire system.  I
  don't want reStructuredText or a hypothetical Python documentation
  processor to die stillborn because of over-ambition.

- Most of all, I want to help ease the documentation chore, the bane
  of many a programmer.

Unfortunately I was sidetracked and stopped working on this project.
In November 2000 I made the time to enumerate the problems of
StructuredText and possible solutions, and complete the first draft of
a specification.  This first draft was posted to the Doc-SIG in three
parts:

- `A Plan for Structured Text`__
- `Problems With StructuredText`__
- `reStructuredText: Revised Structured Text Specification`__

__ http://mail.python.org/pipermail/doc-sig/2000-November/001239.html
__ http://mail.python.org/pipermail/doc-sig/2000-November/001240.html
__ http://mail.python.org/pipermail/doc-sig/2000-November/001241.html

In March 2001 a flurry of activity on the Doc-SIG spurred me to
further revise and refine my specification, the result of which you
are now reading.  An offshoot of the reStructuredText project has been
the realization that a single markup scheme, no matter how well
thought out, may not be enough.  In order to tame the endless debates
on Doc-SIG, a flexible `Docstring Processing System framework`_ needed
to be constructed.  This framework has become the more important of
the two projects; reStructuredText_ has found its place as one
possible choice for a single component of the larger framework.

The project web site and the first project release were rolled out in
June 2001, including posting the second draft of the spec [#spec-2]_
and the first draft of PEPs 256, 257, and 258 [#peps-1]_ to the
Doc-SIG.  These documents and the project implementation proceeded to
evolve at a rapid pace.  Implementation history details can be found
in the `project history file`_.

In November 2001, the reStructuredText parser was nearing completion.
Development of the parser continued with the addition of small
convenience features, improvements to the syntax, the filling in of
gaps, and bug fixes.  After a long holiday break, in early 2002 most
development moved over to the other Docutils components, the
"Readers", "Writers", and "Transforms".  A "standalone" reader
(processes standalone text file documents) was completed in February,
and a basic HTML writer (producing HTML 4.01, using CSS-1) was
completed in early March.

`PEP 287`_, "reStructuredText Standard Docstring Format", was created
to formally propose reStructuredText as a standard format for Python
docstrings, PEPs, and other files.  It was first posted to
comp.lang.python_ and the Python-dev_ mailing list on 2002-04-02.

Version 0.4 of the reStructuredText__ and `Docstring Processing
System`_ projects were released in April 2002.  The two projects were
immediately merged, renamed to "Docutils_", and a 0.1 release soon
followed.

.. __: `reStructuredText SourceForge project`_

.. [#spec-2] The second draft of the spec:

   - `An Introduction to reStructuredText`__
   - `Problems With StructuredText`__
   - `reStructuredText Markup Specification`__
   - `Python Extensions to the reStructuredText Markup
     Specification`__

   __ http://mail.python.org/pipermail/doc-sig/2001-June/001858.html
   __ http://mail.python.org/pipermail/doc-sig/2001-June/001859.html
   __ http://mail.python.org/pipermail/doc-sig/2001-June/001860.html
   __ http://mail.python.org/pipermail/doc-sig/2001-June/001861.html

.. [#peps-1] First drafts of the PEPs:

   - `PEP 256: Docstring Processing System Framework`__
   - `PEP 258: DPS Generic Implementation Details`__
   - `PEP 257: Docstring Conventions`__

   Current working versions of the PEPs can be found in
   http://docutils.sourceforge.net/docs/peps/, and official versions
   can be found in the `master PEP repository`_.

   __ http://mail.python.org/pipermail/doc-sig/2001-June/001855.html
   __ http://mail.python.org/pipermail/doc-sig/2001-June/001856.html
   __ http://mail.python.org/pipermail/doc-sig/2001-June/001857.html


.. _Zope Corporation: http://www.zope.com
.. _ZOPE: http://www.zope.org
.. _reStructuredText SourceForge project:
   http://structuredtext.sourceforge.net/
.. _pythondoc: http://starship.python.net/crew/danilo/pythondoc/
.. _StructuredTextNG:
   http://dev.zope.org/Members/jim/StructuredTextWiki/StructuredTextNG
.. _project history file: http://docutils.sourceforge.net/HISTORY.html
.. _PEP 287: http://docutils.sourceforge.net/spec/pep-0287.html
.. _Docstring Processing System framework:
   http://docutils.sourceforge.net/spec/pep-0256.html
.. _comp.lang.python: news:comp.lang.python
.. _Python-dev: http://mail.python.org/pipermail/python-dev/
.. _Docstring Processing System: http://docstring.sourceforge.net/
.. _master PEP repository: http://www.python.org/peps/


..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End: