From 89701dc6483800daef21eac3feafd4f74553f238 Mon Sep 17 00:00:00 2001 From: efiacor Date: Mon, 7 Nov 2022 09:16:55 +0000 Subject: [PATCH] [DMAAP-MR] Update RTD setup files Signed-off-by: efiacor Change-Id: I108c7940fe36a889287bf6b72a87942fee930411 Issue-ID: DMAAP-1824 --- .gitreview | 1 + .readthedocs.yaml | 8 +-- docs/Installation/Installation.rst | 113 ++++++++++++++++++++++++++++--------- docs/_static/css/ribbon.css | 4 +- docs/conf.py | 64 +++++++++++++++------ docs/conf.yaml | 7 --- docs/index.rst | 8 +-- docs/offeredapis/api.rst | 84 +++++++++++++-------------- docs/requirements-docs.txt | 6 +- docs/tox.ini | 31 +++++----- 10 files changed, 206 insertions(+), 120 deletions(-) delete mode 100644 docs/conf.yaml diff --git a/.gitreview b/.gitreview index c1492a9..23a16ba 100644 --- a/.gitreview +++ b/.gitreview @@ -2,3 +2,4 @@ host=gerrit.onap.org port=29418 project=dmaap/messagerouter/messageservice +defaultbranch=master diff --git a/.readthedocs.yaml b/.readthedocs.yaml index 3797dc8..3758842 100644 --- a/.readthedocs.yaml +++ b/.readthedocs.yaml @@ -5,9 +5,6 @@ # Required version: 2 -formats: - - htmlzip - build: image: latest @@ -16,5 +13,8 @@ python: install: - requirements: docs/requirements-docs.txt +submodules: + include: all + sphinx: - configuration: docs/conf.py + configuration: docs/conf.py \ No newline at end of file diff --git a/docs/Installation/Installation.rst b/docs/Installation/Installation.rst index 53d9fd2..97fe229 100644 --- a/docs/Installation/Installation.rst +++ b/docs/Installation/Installation.rst @@ -6,117 +6,174 @@ Installation Environment ----------- -Message Router is developed using Kafka, Zookeeper and Java. AJSC framework is used to create the REST service and Docker was used to package the service. + +Message Router is developed using Kafka, Zookeeper and Java. AJSC framework is +used to create the REST service and Docker was used to package the service. Steps ----- + Message Router has 3 docker containers. Dmaap\_container, kafka\_container and zookeeper\_container. Zookeeper runs on 172.18.0.2, kafka runs on 172.18.0.3 and dmaap on 172.18.0.4. 1) Clone message service repo - git clone http://gerrit.onap.org/r/dmaap/messagerouter/messageservice + .. code-block:: + + git clone http://gerrit.onap.org/r/dmaap/messagerouter/messageservice -2) copy - messageservice/bundleconfig-local/etc/appprops/MsgRtrApi.properties +2) copy messageservice/bundleconfig-local/etc/appprops/MsgRtrApi.properties to /var/tmp directory 3) In /var/tmp/MsgRtrApi.properties, change value of below variables as shown below: - config.zk.servers=172.18.0.2(Change as per where ZooKeepeer is deployed) + .. code-block:: - kafka.metadata.broker.list=172.18.0.3:9092(Change as per where Kafka is deployed) + config.zk.servers=172.18.0.2 (Change as per where ZooKeepeer is deployed) + kafka.metadata.broker.list=172.18.0.3:9092 (Change as per where Kafka is deployed) 4) Install docker and docker-compose 5) Go to messageservice/src/main/resources/docker-compose and run: - - .. code:: bash - - docker-compose up # add -d argument to start process as a daemon (background process) + + .. code-block:: + + docker-compose up # add -d argument to start process as a daemon (background process) + This should start 3 containers. 6) Run docker ps. It should show 3 containers. |image0| - + .. |image0| image:: docker.png - + Testing ------- - For publishing, create a sample.txt file with some content in the directory where you will run below rest api. Run below rest api: - curl -H "Content-Type:text/plain" -X POST -d @sample.txt - http://172.18.0.4:3904/events/TestTopic1 + .. code-block:: + + curl -H "Content-Type:text/plain" -X POST -d @sample.txt http://172.18.0.4:3904/events/TestTopic1 - For subscribing, run below rest api: - curl -H "Content-Type:text/plain" -XGET - http://172.18.0.4:3904/events/TestTopic1/CG1/C1?timeout=1000 + .. code-block:: + + curl -H "Content-Type:text/plain" -XGET http://172.18.0.4:3904/events/TestTopic1/CG1/C1?timeout=1000 + Note: You will only receive messages which have been published after you have subscribed to a topic. Steps for local development and test ------------------------- +------------------------------------ + On Intel dev machine, in terminal (> indicates prompt) : + 1) Build kafka11aaf - > git clone https://gerrit.onap.org/r/dmaap/kafka - > cd kafka11aaf - > mvn clean install -Pdocker + + .. code-block:: + + > git clone https://gerrit.onap.org/r/dmaap/kafka + > cd kafka11aaf + > mvn clean install -Pdocker + + 2) Build messageservice - > git clone https://gerrit.onap.org/r/dmaap/messagerouter/messageservice - - anonymous http, can't push changes - > cd messageservice - > mvn clean install -Pdocker + + .. code-block:: + + > git clone https://gerrit.onap.org/r/dmaap/messagerouter/messageservice + (Note: anonymous http, can't push changes) + > cd messageservice + > mvn clean install -Pdocker + + 3) Run tests - > cp bundleconfig-local/etc/appprops/MsgRtrApi.properties /var/tmp/ + + .. code-block:: + + > cp bundleconfig-local/etc/appprops/MsgRtrApi.properties /var/tmp/ + + - edit /var/tmp/MsgRtrApi.properties config.zk.servers=zookeeper kafka.metadata.broker.list=kafka:9092 + - docker-compose network maps service name(zookeeper, kafka) to IP + - set docker preferences/file sharing to access /var/tmp + > cd src/main/resources/docker-compose + - edit docker-compose.yml - remove "nexus3.onap.org:10001/" from kafka and dmaap image names to use local images + > docker-compose up -d - create sample.txt file (as above)(content of file not important) + > curl -H "Content-Type:text/plain" -X POST -d @sample.txt http://localhost:3904/events/TestTopic1 On Arm: + 1) Build kafka11aaf - > git clone https://gerrit.onap.org/r/dmaap/kafka + + .. code-block:: + + > git clone https://gerrit.onap.org/r/dmaap/kafka + > cd kafka11aaf + > mvn clean install -Pdocker -Ddocker.pull.registry=docker.io - ensure we pull Arm version of base image + + 2) Build messageservice + + .. code-block:: + > git clone https://gerrit.onap.org/r/dmaap/messagerouter/messageservice - anonymous http, can't push changes + > cd messageservice + > mvn clean install -Pdocker -Ddocker.pull.registry=docker.io - ensure we pull Arm version of base image + + 3) Run tests + + .. code-block:: + > cp bundleconfig-local/etc/appprops/MsgRtrApi.properties /var/tmp/ + - edit /var/tmp/MsgRtrApi.properties config.zk.servers=zookeeper kafka.metadata.broker.list=kafka:9092 + - docker-compose network maps service name(zookeeper, kafka) to IP + - set docker preferences/file sharing to access /var/tmp + > cd src/main/resources/docker-compose - edit docker-compose.yml + - remove "nexus3.onap.org:10001/" from from kafka and dmaap image names to use local images + - replace 'nexus3.onap.org:10001/onap/dmaap/zookeeper:1.0.0' with multi-platform 'zookeeper' + > docker-compose up -d + - create sample.txt file (as above)(content of file not important) + > curl -H "Content-Type:text/plain" -X POST -d @sample.txt http://localhost:3904/events/TestTopic1 - - diff --git a/docs/_static/css/ribbon.css b/docs/_static/css/ribbon.css index 6008cb1..7342cdc 100644 --- a/docs/_static/css/ribbon.css +++ b/docs/_static/css/ribbon.css @@ -59,5 +59,5 @@ /* fix width of the screen */ .wy-nav-content { - max-width: none; -} + max-width: 800px; +} \ No newline at end of file diff --git a/docs/conf.py b/docs/conf.py index bd96005..88443af 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -1,27 +1,57 @@ -from docs_conf.conf import * +project = "onap" +release = "master" +version = "master" -master_doc = 'index' +author = "Open Network Automation Platform" +# yamllint disable-line rule:line-length +copyright = "ONAP. Licensed under Creative Commons Attribution 4.0 International License" -intersphinx_mapping = {} +pygments_style = "sphinx" +html_theme = "sphinx_rtd_theme" +html_theme_options = { + "style_nav_header_background": "white", + "sticky_navigation": "False" } +html_logo = "_static/logo_onap_2017.png" +html_favicon = "_static/favicon.ico" +html_static_path = ["_static"] +html_show_sphinx = False -linkcheck_ignore = [ - 'http://localhost', - 'https://example.com', - 'about:config', - # this URL is not directly reachable and must be configured in the system hosts file. - 'https://portal.api.simpledemo.onap.org:30225/ONAPPORTAL/login.htm', - # anchor issues - 'https://docs.onap.org/projects/onap-integration/en/latest/docs_usecases_release.html#.*', - 'https://docs.linuxfoundation.org/docs/communitybridge/easycla/contributors/contribute-to-a-gerrit-project#.*', - 'https://docs.onap.org/projects/onap-integration/en/latest/docs_robot.html#docs-robot', - 'https://docs.onap.org/projects/onap-integration/en/latest/docs_usecases_release.html#docs-usecases-release', - 'https://docs.onap.org/projects/onap-integration/en/latest/docs_usecases.html#docs-usecases', - 'https://docs.onap.org/projects/onap-integration/en/latest/usecases/release_non_functional_requirements.html#release-non-functional-requirements', +extensions = [ + 'sphinx.ext.intersphinx', + 'sphinx.ext.graphviz', + 'sphinxcontrib.blockdiag', + 'sphinxcontrib.seqdiag', + 'sphinxcontrib.swaggerdoc', + 'sphinxcontrib.plantuml' ] +# +# Map to 'latest' if this file is used in 'latest' (master) 'doc' branch. +# Change to {releasename} after you have created the new 'doc' branch. +# -html_last_updated_fmt = '%d-%b-%y %H:%M' +branch = 'latest' + +intersphinx_mapping = {} +doc_url = 'https://docs.onap.org/projects' +master_doc = 'index' +exclude_patterns = ['.tox'] + +spelling_word_list_filename='spelling_wordlist.txt' +spelling_lang = "en_GB" + +# +# Example: +# intersphinx_mapping['onap-aai-aai-common'] = ('{}/onap-aai-aai-common/en/%s'.format(doc_url) % branch, None) +# + +html_last_updated_fmt = '%d-%b-%y %H:%M' def setup(app): app.add_css_file("css/ribbon.css") + +linkcheck_ignore = [ + r'http://localhost:\d+/', + r'http://:3904/events/org.onap.dmaap.mr.sprint/mygroup/mycus' +] diff --git a/docs/conf.yaml b/docs/conf.yaml deleted file mode 100644 index ab59281..0000000 --- a/docs/conf.yaml +++ /dev/null @@ -1,7 +0,0 @@ ---- -project_cfg: onap -project: onap - -# Change this to ReleaseBranchName to modify the header -default-version: latest -# diff --git a/docs/index.rst b/docs/index.rst index 0321395..566bea8 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -2,10 +2,8 @@ .. http://creativecommons.org/licenses/by/4.0 .. _master_index: -dmaap/message router(MR) -==================== - - +dmaap/message router (MR) +========================= .. toctree:: :maxdepth: 1 @@ -20,5 +18,3 @@ dmaap/message router(MR) administration/administration.rst humaninterfaces/humaninterfaces.rst release-notes/release-notes.rst - - diff --git a/docs/offeredapis/api.rst b/docs/offeredapis/api.rst index 906eb86..6027acf 100644 --- a/docs/offeredapis/api.rst +++ b/docs/offeredapis/api.rst @@ -22,7 +22,7 @@ considerations for each segment are as follows and are required for each of the specific transactions described in this section. HTTP URL -------- +-------- http[s]://serverBaseURL{/routing}{resourcePath} @@ -94,10 +94,12 @@ Request Parameters | partitionKey | | QueryParam | String | | N | String value | ?Partitionkey=123 | +--------------------+------------------------------+------------------+------------+-----------+-------------+--------------------------------+-----------------------------+ + **NOTE**: To publish data to the authenticated topic, Publisher must - have the AAF permission org.onap.dmaap.mr.topic|:topic.|pub. +have the AAF permission org.onap.dmaap.mr.topic|:topic.|pub. Publishers may use DMaaP BusController for provisoning the AAF permissions + Response Parameters =================== @@ -276,22 +278,23 @@ Response /Error Codes Sample Request: =============== -+----------------------------------------------------------------------------------------------------+ -| GET http://:3904/events/org.onap.dmaap.mr.sprint/mygroup/mycus | -| | -| Content-Type: application/json | -| | -| Example: | -| | -|curl -u XXX@csp.abc.com:MRDmap2016$ -X GET -d 'MyfirstMessage' | -| | -|http://10.12.7.22:3904/events/com.att.ecomp_test.crm.preDeo/myG/C1 | -| | -|[I am r sending first msg,I am R sending first msg] | -+----------------------------------------------------------------------------------------------------+ ++-----------------------------------------------------------------------------+ +| GET http ://{hostname}:3904/events/org.onap.dmaap.mr.sprint/mygroup/mycus | +| | +| Content-Type: application/json | +| | +| Example: | +| | +|curl -u XXX@csp.abc.com:MRDmap2016$ -X GET -d 'MyfirstMessage' | +| | +|http ://10.12.7.22:3904/events/com.att.ecomp_test.crm.preDeo/myG/C1 | +| | +|[I am r sending first msg,I am R sending first msg] | ++-----------------------------------------------------------------------------+ Provisioning ------------ + **Description**: To create, modify or delete the MessageRouter topics. These APIs can also be used by other applications to provision topics in MessageRouter. DMaaP BusController is recommended for topic and AAF permissions provisioning @@ -308,7 +311,7 @@ Request Parameters: +-------------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-----------------------------------+ | Name | Description | Param Type | datatype | MaxLen | Required | Format | Valid/Example Values | +===================+=================================+==================+============+==============+=============+=============+===================================+ -| Topicname | topicname to be created in MR | Body | String | 20 | Y | Json | org.onap.dmaap.mr.metrics | +| Topicname | topicname to be created in MR | Body | String | 20 | Y | Json | org.onap.dmaap.mr.metrics | +-------------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-----------------------------------+ | topicDescription | description for topic | Body | String | 15 | Y | | | +-------------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-----------------------------------+ @@ -372,26 +375,26 @@ Response Parameters Sample Request: =============== - .. code:: bash - - POST http://:3904/topics/create - Request Body - {"topicName":"org.onap.dmaap.mr.testtopic","description":"This is a test Topic ", - "partitionCount":"1","replicationCount":"3","transactionEnabled":"true"} - Content-Type: application/json - Example: - curl -u XXXc@csp.abc.com:xxxxx$ -H 'Content-Type:application/json' -X POST -d - @topicname.txt http://message-router:3904/topics/create - { - "writerAcl": { - "enabled": false, - "users": [] - }, - "description": "This is a TestTopic", - "name": "org.onap.dmaap.mr.testtopic", - "readerAcl": { - "enabled": false, - "users": [] + .. code:: bash + + POST http: //:3904/topics/create + Request Body + {"topicName":"org.onap.dmaap.mr.testtopic","description":"This is a test Topic ", + "partitionCount":"1","replicationCount":"3","transactionEnabled":"true"} + Content-Type: application/json + Example: + curl -u XXXc@csp.abc.com:xxxxx$ -H 'Content-Type:application/json' -X POST -d + @topicname.txt http: //message-router:3904/topics/create + { + "writerAcl": { + "enabled": false, + "users": [] + }, + "description": "This is a TestTopic", + "name": "org.onap.dmaap.mr.testtopic", + "readerAcl": { + "enabled": false, + "users": [] GetTopic Details @@ -448,9 +451,9 @@ Sample Request: =============== +-----------------------------------------------------------------------------------------------------------------------------------+ -| GET http://:3904/topic/org.onap.dmaap.mr.testtopic | +| GET http ://:3904/topic/org.onap.dmaap.mr.testtopic | | curl -u XXX@csp.abc.com:x$ -X | -| GET http://10.12.7.22:3904/topics | +| GET http ://10.12.7.22:3904/topics | | {"topics": [ | | {"txenabled": true,"description": "This is a TestTopic","owner": "XXXX@csp.abc.com","topicName": "org.onap.dmaap.mr.Load9" | | {"txenabled": false,"description": "", "owner": "", "topicName": "org.onap.dmaap.mr.Load1" | @@ -471,7 +474,7 @@ org.onap.dmaap.mr.topicFactory|:org.onap.dmaap.mr.topic:org.onap.dmaap.mr|destro Sample Request: =============== -ex: http://:3904/topics/org.onap.dmaap.mr.testopic +ex: http ://:3904/topics/org.onap.dmaap.mr.testopic +---------------------------+------------------------------------+ | Response statusCode | Response statusMessage | @@ -530,6 +533,3 @@ API Inventory | | from write | String consumerId) | {consumerId} | | | | | ACL on a Topic | | | | | +-----------+--------------------+-----------------------------------------+---------------------------------------+----------------+----------------------------------+ - - - diff --git a/docs/requirements-docs.txt b/docs/requirements-docs.txt index 3b3441a..ba3e0ec 100644 --- a/docs/requirements-docs.txt +++ b/docs/requirements-docs.txt @@ -1,3 +1,7 @@ -lfdocs-conf sphinx>=4.2.0 # BSD sphinx-rtd-theme>=1.0.0 # MIT +sphinxcontrib-blockdiag # BSD +sphinxcontrib-seqdiag # BSD +sphinxcontrib-swaggerdoc +sphinxcontrib-spelling +sphinxcontrib-plantuml \ No newline at end of file diff --git a/docs/tox.ini b/docs/tox.ini index edc90fd..b9d7eb4 100644 --- a/docs/tox.ini +++ b/docs/tox.ini @@ -1,6 +1,6 @@ [tox] minversion = 1.6 -envlist = docs, +envlist = docs,docs-linkcheck,docs-spellcheck skipsdist = true [testenv:docs] @@ -8,19 +8,24 @@ basepython = python3.8 deps = -r{toxinidir}/requirements-docs.txt -chttps://raw.githubusercontent.com/openstack/requirements/stable/yoga/upper-constraints.txt - -chttps://git.onap.org/doc/plain/etc/upper-constraints.onap.txt -whitelist_externals = - echo + -chttps://git.onap.org/doc/plain/etc/upper-constraints.onap.txt?h=master commands = - sphinx-build -b html -n -d {envtmpdir}/doctrees ./ {toxinidir}/_build/html - echo "Generated docs available in {toxinidir}/_build/html" + sphinx-build -W -q -b html -n -d {envtmpdir}/doctrees {toxinidir} {toxinidir}/_build/html [testenv:docs-linkcheck] basepython = python3.8 -# deps = -# -r{toxinidir}/requirements-docs.txt -# -chttps://git.onap.org/doc/plain/etc/upper-constraints.os.txt -# -chttps://git.onap.org/doc/plain/etc/upper-constraints.onap.txt -commands = echo "Link Checking not enforced" -# commands = sphinx-build -b linkcheck -d {envtmpdir}/doctrees ./ {toxinidir}/_build/linkcheck -whitelist_externals = echo +deps = + -r{toxinidir}/requirements-docs.txt + -chttps://raw.githubusercontent.com/openstack/requirements/stable/yoga/upper-constraints.txt + -chttps://git.onap.org/doc/plain/etc/upper-constraints.onap.txt?h=master +commands = + sphinx-build -W -q -b linkcheck -d {envtmpdir}/doctrees {toxinidir} {toxinidir}/_build/linkcheck + +[testenv:docs-spellcheck] +basepython = python3.8 +deps = + -r{toxinidir}/requirements-docs.txt + -chttps://raw.githubusercontent.com/openstack/requirements/stable/yoga/upper-constraints.txt + -chttps://git.onap.org/doc/plain/etc/upper-constraints.onap.txt?h=master +commands = + sphinx-build -W -q -b spelling -d {envtmpdir}/doctrees {toxinidir} {toxinidir}/_build/spellcheck -- 2.16.6