1 .. This work is licensed under a Creative Commons Attribution 4.0
2 .. International License. http://creativecommons.org/licenses/by/4.0
3 .. Copyright 2017 AT&T Intellectual Property. All rights reserved.
9 ONAP documentation is stored in git repositories, changes are managed
10 with gerrit reviews, and published documents generated when there is a
11 change in any source used to build the documentation.
13 Authors create source for documents in reStructured Text (RST) that is
14 rendered to HTML and published on Readthedocs.io.
15 The developer Wiki or other web sites can reference these rendered
16 documents directly allowing projects to easily maintain current release
19 Some initial set up is required to connect a project with
20 the master document structure and enable automated publishing of
21 changes as summarized in the following diagram and description below
29 DA [label = "Doc Project\nAuthor/Committer", color=lightblue];
30 DR [label = "Doc Gerrit Repo" , color=pink];
31 PR [label = "Other Project\nGerrit Repo", color=pink ];
32 PA [label = "Other Project\nAuthor/Committer", color=lightblue];
34 PA -> DR [label = "Add project repo as\ngit submodule" ];
35 DR -> DA [label = "Review & Plan to\nIntegrate Content with\nTocTree Structure" ];
36 DR <-- DA [label = "Vote +2/Merge" ];
37 PA <-- DR [label = "Merge Notification" ];
38 PA -> PR [label = "Create in project repo\ntop level directory and index.rst" ];
39 PR -> DA [label = "Add as Reviewer" ];
40 PR <-- DA [label = "Approve and Integrate" ];
41 PA <-- PR [label = "Merge" ];
44 Setup project repositories
45 --------------------------
46 These steps are performed for each project repository that
47 provides documentation.
49 1. Set two variables that will be used in the subsequent steps.
50 Set *reponame* to the project repository you are setting up
51 just as it appears in the **Project Name** column of
52 the Gerrit projects page.
53 Set *lfid* to your Linux Foundation identity that you use to
54 login to gerrit or for git clone requests over ssh.
61 2. Add a directory in the doc project where your
62 project will be included as a submodule and at least one reference
63 from the doc project to the documentation index in your repository.
64 The following sequence will do this over ssh. Please note that the
65 reference to your project in *repolist.rst* should be considered
66 temporary and removed when you reference it from more appropriate
71 If your access network restricts ssh, you will need to use equivalent
72 git commands and HTTP Passwords as described `here <http://wiki.onap.org/x/X4AP>`_.
76 Don't replace ../ in *git submodule add* with any relative path on
77 your local file system. It refers to the location of your repository
82 git clone ssh://$lfid@gerrit.onap.org:29418/doc
84 mkdir -p `dirname docs/submodules/$reponame`
85 git submodule add ../$reponame docs/submodules/$reponame.git
86 git submodule init docs/submodules/$reponame.git
87 git submodule update docs/submodules/$reponame.git
89 echo " $reponame <../submodules/$reponame.git/docs/index>" >> docs/release/repolist.rst
96 Wait for the above change to be merged before any merge to the
97 project repository that you have just added as a submodule.
98 If the project repository added as submodule changes before the
99 doc project merge, git may not automatically update the submodule
100 reference on changes and/or the verify job will fail in the step below.
103 3. Create a docs directory in your repository with
104 an index.rst file. The following sequence will complete the minimum
105 required over ssh. As you have time to convert or add new content you
106 can update the index and add files under the docs folder.
109 If you have additional content, you can include it by editing the
110 index.rst file and/or adding other files before the git commit.
111 See `Templates and Examples`_ below and :ref:`converting-to-rst`
112 for more information.
117 git clone ssh://$lfid@gerrit.onap.org:29418/$reponame
120 echo ".. This work is licensed under a Creative Commons Attribution 4.0 International License.
122 TODO Add files to toctree and delete this header
123 ------------------------------------------------
134 The diagram below illustrates what is accomplished in the setup steps
135 above from the perspective of a file structure created for a local test,
136 a jenkins verify job, and/or published release documentation including:
138 - ONAP gerrit project repositories,
140 - doc project repository master document index.rst, templates,
141 configuration, and other documents
143 - submodules directory where other project repositories and
144 directories/files are referenced
146 - file structure: directories (ellipses), files(boxes)
148 - references: directory/files (solid edges), git submodule
149 (dotted edges), sphinx toctree (dashed edges)
154 digraph docstructure {
156 node [fontname = "helvetica"];
157 // Align gerrit repos and docs directories
158 {rank=same doc aaf aai reponame repoelipse vnfsdk vvp}
159 {rank=same confpy release templates masterindex submodules otherdocdocumentelipse}
160 {rank=same releasedocumentindex releaserepolist}
162 //Illustrate Gerrit Repos and provide URL/Link for complete repo list
163 gerrit [label="gerrit.onap.org/r", href="https://gerrit.onap.org/r/#/admin/projects/" ];
164 doc [href="https://gerrit.onap.org/r/gitweb?p=doc.git;a=tree"];
169 gerrit -> repoelipse;
170 repoelipse [label=". . . ."];
174 //Show example of local reponame instance of component info
175 reponame -> reponamedocsdir;
176 reponamesm -> reponamedocsdir;
177 reponamedocsdir [label="docs"];
178 reponamedocsdir -> repnamedocsdirindex;
179 repnamedocsdirindex [label="index.rst", shape=box];
181 //Show detail structure of a portion of doc/docs
184 confpy [label="conf.py",shape=box];
186 masterindex [label="Master\nindex.rst", shape=box];
189 docs -> otherdocdocumentelipse;
190 otherdocdocumentelipse [label="...other\ndocuments"];
193 masterindex -> releasedocumentindex [style=dashed, label="sphinx\ntoctree\nreference"];
195 //Show submodule linkage to docs directory
196 submodules -> reponamesm [style=dotted,label="git\nsubmodule\nreference"];
197 reponamesm [label="reponame.git"];
199 //Example Release document index that references component info provided in other project repo
200 release -> releasedocumentindex;
201 releasedocumentindex [label="index.rst", shape=box];
202 releasedocumentindex -> releaserepolist [style=dashed, label="sphinx\ntoctree\nreference"];
203 releaserepolist [label="repolist.rst", shape=box];
204 release -> releaserepolist;
205 releaserepolist -> repnamedocsdirindex [style=dashed, label="sphinx\ntoctree\nreference"];
209 Branches in the DOC Project
210 ---------------------------
212 The DOC project 'master' branch aggregates the 'latest' content
213 from all ONAP project repositories contributing documentation into a
214 single tree file structure as described in the previous section. This
215 branch is continuously integrated and deployed at Read The
216 Docs as the 'latest' ONAP Documentation by:
218 * Jenkins doc-verify-rtd and doc-merge-rtd jobs triggered whenever patches on
219 contributing repositories contain rst files at or below a top level
222 * Subscription in the DOC project to changes in submodule repositories.
223 These changes appear in the DOC project as commits with title
224 'Updated git submodules' when a change to a contributing project
225 repository is merged. No DOC project code review occurs, only a
226 submodule repository commit hash is updated to track the head of each
227 contributing master branch.
229 For each ONAP named release the DOC project creates a branch with the
230 release name. The timing of the release branch is determined by
231 work needed in the DOC project to prepare the release branch and the
232 amount of change unrelated to the release in the master branch.
233 For example contributing projects that create named release branches
234 early to begin work on the next release and/or contributing projects
235 to the master that are not yet part of the named release would result
236 in an earlier named release branch to cleanly separate work to stabilize
237 a release from other changes in the master branch.
239 A named release branch is integrated and deployed at Read The Docs
240 as the 'named release' by aggregating content from contributing
241 project repositories. A contributing project repository can
242 choose one of the following for the 'named release' branch:
244 * Remove the contributing project repository submodule and RST
245 references when not part of the named release.
247 * Provide a commit hash or tag for the contributing project master
248 branch to be used for the life of the release branch or until a
249 request is submitted to change the commit hash or tag.
251 * Provide the commit hash for the head of a named release branch
252 created in the contributing project repository. This option
253 may be appropriate if frequent changes are expected over the
254 life of the named release and work the same way as the continuous
255 integration and deployment described for the master branch.
257 The decision on option for each contributing project repository
258 can be made or changed before the final release is approved. The
259 amount of change and expected differences between master and a
260 named release branch for each repository should drive the choice of
266 GIT is a powerful tool allowing many actions, but without respecting some rules
267 the GIT structure can be quickly hard to maintain.
269 Here are some conventions about GIT branches:
271 - ALWAYS create a local branch to edit or create any file. This local branch
272 will be considered as a topic in Gerrit and allow contributors to
273 work at the same time on the same project.
275 - 1 feature = 1 branch. In the case of documentation, a new chapter
276 or page about a new code feature can be considered as a 'doc feature'
278 - 1 bug = 1 branch. In the case of documentation, a correction on an
279 existing sentence can be considered as a 'doc bug'
281 - the master branch is considered as "unstable", containing new features that
282 will converge to a stable situation for the release date.
284 The day of the release, the repository owner will create a new branch to
285 fix the code and documentation. This will represent the 'stable' code of the
286 release. In this context:
288 - NEVER push a new feature on a stable branch
290 - Only bug correction are authorized on a stable branch using
293 .. image:: git_branches.png
295 Creating Restructured Text
296 ==========================
298 ReStructuredText markup conventions
299 -----------------------------------
300 For detailed information on ReStructuredText and how to best use the format,
303 - `ReStructured Text Primer <http://docutils.sourceforge.net/docs/user/rst/quickstart.html>`_
304 - `ReStructured Text Quick Reference <http://docutils.sourceforge.net/docs/user/rst/quickref.html>`_
307 Templates and Examples
308 ----------------------
309 Templates are available that capture the kinds of information
310 useful for different types of projects and provide some examples of
311 restructured text. We organize templates in the following way to:
313 - help authors understand relationships between documents
315 - keep the user audience context in mind when writing and
317 - tailor sections for different kinds of projects.
320 **Sections** Represent a certain type of content. A section
321 is **provided** in an project repository, to describe something about
322 the characteristics, use, capability, etc. of things in that repository.
323 A section may also be **referenced** from other sections and in
324 other repositories. For example, an API specification provided in a project
325 repository might be referenced to in a Platform API Reference Guide.
326 The notes in the beginning of each section template provide
327 additional detail about what is typically covered and where
328 there may be references to the section.
330 **Collections** Are a set of sections that are typically provided
331 for a particular type of project, repository, guide, reference manual, etc.
332 For example, a collection for a platform component, an SDK, etc.
334 You can: browse the template *collections* and *sections* below;
335 show source to look at the Restructured Text and Sphinx directives used.
340 Section examples are available here: :ref:`Templates<templates>`
345 In addition to these simple templates and examples
346 there are many open source projects (e.g. Open Daylight, Open Stack)
347 that are using Sphinx and Readthedocs where you may find examples
348 to start with. Working with project teams we will continue to enhance
349 templates here and capture frequently asked questions on the developer
350 wiki question topic `documentation <https://wiki.onap.org/questions/topics/16384055/documentation>`_.
354 - decide what is relevant content
356 - determine the best way to create/maintain it in the CI/CD process and
358 - work with the documentation team to reference content from the
359 master index and guides.
361 Consider options including filling in a template, identifying existing
362 content that can be used as is or easily converted, and use of Sphinx
363 directives/extensions to automatically generate restructured text
364 from other source you already have.
366 Collection examples are available here: :ref:`Templates<templates>`
370 It's pretty common to want to reference another location in the
371 ONAP documentation and it's pretty easy to do with
372 reStructuredText. This is a quick primer, more information is in the
373 `Sphinx section on Cross-referencing arbitrary locations
374 <http://www.sphinx-doc.org/en/stable/markup/inline.html>`_.
376 Within a single document, you can reference another section simply by::
378 This is a reference to `The title of a section`_
380 Assuming that somewhere else in the same file there a is a section
381 title something like::
383 The title of a section
384 ^^^^^^^^^^^^^^^^^^^^^^
386 It's typically better to use ``:ref:`` syntax and labels to provide
387 links as they work across files and are resilient to sections being
388 renamed. First, you need to create a label something like::
392 The title of a section
393 ^^^^^^^^^^^^^^^^^^^^^^
395 .. note:: The underscore (_) before the label is required.
397 Then you can reference the section anywhere by simply doing::
399 This is a reference to :ref:`a-label`
403 This is a reference to :ref:`a section I really liked <a-label>`
405 .. note:: When using ``:ref:``-style links, you don't need a trailing
408 Because the labels have to be unique, it usually makes sense to prefix
409 the labels with the project name to help share the label space, e.g.,
410 ``sfc-user-guide`` instead of just ``user-guide``.
415 The index file must relatively reference your other rst files in that directory.
417 Here is an example index.rst :
429 documentation-example
434 Document source files have to be written in reStructuredText format (rst).
435 Each file would be built as an html page.
437 Here is an example source rst file :
456 See http://sphinx-doc.org/rest.html .
459 You can add html content that only appears in html output by using the
460 'only' directive with build type
461 ('html' and 'singlehtml') for an ONAP document. But, this is not encouraged.
466 This line will be shown only in html version.
472 Building an index for your Sphinx project is relatively simple. First, tell Sphinx that
473 you want it to build an index by adding something like this after your TOC tree:
484 Note that search was included here. It works out of the box with any Sphinx project, so you
485 don't need to do anything except include a reference to it in your :code:`index.rst` file.
487 Now, to generate a index entry in your RST, do one of the following:
491 Some content that requires an :index:`index`.
500 Some header containing myterm
501 =============================
503 In the second case, Sphinx will create a link in the index to the paragraph that follows
504 the index entry declaration.
506 When your project is built, Sphinx will generate an index page populated with the entries
507 you created in the source RST.
509 These are simple cases with simple options. For more information about indexing with Sphinx,
510 please see the `official Sphinx documentation <http://www.sphinx-doc.org/en/stable/markup/misc.html>`_.
519 The verify job name is **doc-{stream}-verify-rtd**
521 Proposed changes in files in any repository with top level docs folder
522 in the repository and RST files in below this folder
523 will be verified by this job as part of a gerrit code review.
526 The contributing author and every reviewer on a gerrit code review
527 should always review the Jenkins log before approving and merging a
528 change. The log review should include:
530 * Using a browser or other editor to search for a pattern in the
531 *console log* that matches files in the patch set. This will quickly
532 identify errors and warnings that are related to the patch set and
533 repository being changed.
535 * Using a browser to click on the *html* folder included in the log
536 and preview how the proposed changes will look when published at
537 Read The Docs. Small changes can be easily made in the patch set.
542 The merge job name is **doc-{stream}-merge-rtd**.
544 When a committer merges a patch that includes files matching the
545 path described above, the doc project merge job will trigger an
546 update at readthedocs. There may be some delay after the merge job
547 completes until new version appears at Read The Docs.
554 It is recommended that all rst content is validated by `doc8 <https://pypi.python.org/pypi/doc8>`_ standards.
555 To validate your rst files using doc8, install doc8.
559 sudo pip install doc8
561 doc8 can now be used to check the rst files. Execute as,
565 doc8 --ignore D000,D001 <file>
571 To test how the documentation renders in HTML, follow these steps:
573 Install `virtual environment <https://pypi.org/project/virtualenv>`_ & create one.
577 sudo pip install virtualenv
580 Activate `onap_docs` virtual environment.
584 source onap_docs/bin/activate
586 .. note:: Virtual environment activation has to be performed before attempting to build documentation.
587 Otherwise, tools necessary for the process might not be available.
589 Download a project repository.
593 git clone http://gerrit.onap.org/r/<project>
595 Download the doc repository.
599 git clone http://gerrit.onap.org/r/doc
601 Change directory to doc & install requirements.
606 pip install -r etc/requirements.txt
610 Just follow the next step (copying conf.py from Doc project to your project)
611 if that is your intention, otherwise skip it. Currently all projects should already have a conf.py file.
612 Through the next step, this file and potential extensions in your project get overriden.
614 Copy the conf.py file to your project folder where RST files have been kept:
618 cp docs/conf.py <path-to-project-folder>/<folder where are rst files>
620 Copy the static files to the project folder where RST files have been kept:
624 cp -r docs/_static/ <path-to-project-folder>/<folder where are rst files>
626 Build the documentation from within your project folder:
630 sphinx-build -b html <path-to-project-folder>/<folder where are rst files> <path-to-output-folder>
632 Your documentation shall be built as HTML inside the
633 specified output folder directory.
635 You can use your Web Browser to open
636 and check resulting html pages in the output folder.
638 .. note:: Be sure to remove the `conf.py`, the static/ files and the output folder from the `<project>/docs/`. This is for testing only. Only commit the rst files and related content.
640 .. _building-all-documentation:
644 To build the all documentation under doc/, follow these steps:
646 Install `tox <https://pypi.org/project/tox>`_.
652 Download the DOC repository.
656 git clone http://gerrit.onap.org/r/doc
658 Build documentation using tox local environment & then open using any browser.
664 firefox docs/_build/html/index.html
666 .. note:: Make sure to run `tox -elocal` and not just `tox`.
667 This updates all submodule repositories that are integrated
670 There are additional tox environment options for checking External
671 URLs and Spelling. Use the tox environment options below and then
672 look at the output with the Linux `more` or similar command
673 scan for output that applies to the files you are validating.
678 more < docs/_build/linkcheck/output.txt
681 more < docs/_build/spellcheck/output.txt