Merge "Distribution S3P tests results"
authorJim Hahn <jrh3@att.com>
Wed, 31 Mar 2021 14:11:04 +0000 (14:11 +0000)
committerGerrit Code Review <gerrit@onap.org>
Wed, 31 Mar 2021 14:11:04 +0000 (14:11 +0000)
docs/apex/APEX-DecisionMakerExample.rst
docs/apex/APEX-User-Manual.rst
docs/architecture/architecture.rst
docs/development/devtools/api-s3p.rst
docs/development/devtools/images/api-response-time-distribution_performance_H.png [new file with mode: 0644]
docs/development/devtools/images/api-response-time-overtime_performance_H.png [new file with mode: 0644]
docs/development/devtools/images/api-s3p-jm-2_H.png [new file with mode: 0644]
docs/release-notes.rst

index 57791ad..05aeefa 100644 (file)
@@ -39,7 +39,7 @@ APEX Examples Decision Maker
 
 .. |File > New to create a new Policy Model| image:: images/mfp/MyFirstPolicy_P1_newPolicyModel1.png
 .. |Create a new Policy Model| image:: images/mfp/MyFirstPolicy_P1_newPolicyModel2.png
-.. |ONAP| image:: ../../../images/logos.png
+.. |ONAP| image:: ../_static/logo_onap_2017.png
    :class: builtBy
    :target: http://www.onap.org/
 .. |apexConfigRESTClient_link| raw:: html
index 175f539..c48583a 100644 (file)
@@ -8,8 +8,8 @@ APEX User Manual
 .. contents::
     :depth: 3
 
-Installation
-^^^^^^^^^^^^
+Installation of Apex
+^^^^^^^^^^^^^^^^^^^^
 
 Requirements
 ------------
@@ -226,13 +226,14 @@ Build APEX
 +----------------------------------------------------------------------------------------------------------------+
 | Unix, Cygwin                                                                                                   |
 +================================================================================================================+
-| .. container::                                                                                                 |
+| .. container:: content                                                                                         |
 |                                                                                                                |
 | .. container:: listingblock                                                                                    |
 |                                                                                                                |
 | .. container:: content                                                                                         |
 |                                                                                                                |
 | .. code::                                                                                                      |
+|                                                                                                                |
 | :number-lines:                                                                                                 |
 |                                                                                                                |
 | -rwxrwx---+ 1 esvevan Domain Users       772 Sep  3 11:55 apex-pdp-package-full_2.0.0~SNAPSHOT_all.changes*    |
index 4c8b3f7..7edf9e0 100644 (file)
@@ -393,7 +393,7 @@ this information to automatically generate a policy.
 
 Note that SDC provides a wrapper for the SDC API as a Java Client and also provides a TOSCA parser. See the
 documentation for the `Policy Distribution Component
-<https://docs.onap.org/en/latest/submodules/policy/distribution.git/docs/index.html>`__.
+<https://docs.onap.org/projects/onap-policy-parent/en/latest/distribution/distribution.html>`__.
 
 In Step 4 above, the \ *PolicyDesign* must download the CSAR file. If the policy is to be composed from the TOSCA
 definition, it must also parse the TOSCA definition.
index 439719b..55867b4 100644 (file)
@@ -135,14 +135,54 @@ average a 10 seconds plus response time.
 Performance Test of Policy API
 ++++++++++++++++++++++++++++++
 
-A specific performance test was omitted in Honululu (as in Guilin).   The JMeter script used in the stability run injected
-back to back traffic with 5 parallel threads with no pauses between requests.   Since the JMeter threads operate
-in synchronous mode (waiting for a request's response before sending the next request), JMeter injection rates autoregulate
-because of the backpressure imposed by the response times.   Even though the response times are high, the
-"Response over Time" graph above indicates that they remain constant at large, throughout the duration of the test.
-This together with the absence of notorious spikes in the kubernetes node CPU utilization suggests that the API
-component is not strained.   A more enlightning set of tests, would plot jmeter threads (increasing load)
-against response times.   These tests have not been performed in this release.
+Introduction
+------------
+
+Performance test of policy-api has the goal of testing the min/avg/max processing time and rest call throughput for all the requests when the number of requests are large enough to saturate the resource and find the bottleneck.
+
+Setup Details
+-------------
+
+The performance test was performed on a default ONAP OOM installation in the Intel Wind River Lab environment.
+JMeter was installed on a separate VM to inject the traffic defined in the
+`API performace script
+<https://git.onap.org/policy/api/tree/testsuites/performance/src/main/resources/testplans/policy_api_performance.jmx>`_
+with the following command:
+
+.. code-block:: bash
+
+    jmeter.sh --nongui --testfile policy_api_performance.jmx --logfile result.jtl
+
+
+Test Plan
+---------
+
+Performance test plan is the same as stability test plan above.
+Only differences are, in performance test, we increase the number of threads up to 20 (simulating 20 users' behaviors at the same time) whereas reducing the test time down to 2.5 hours.
+
+Run Test
+--------
+
+Running/Triggering performance test will be the same as stability test. That is, launch JMeter pointing to corresponding *.jmx* test plan. The *API_HOST* and *API_PORT* are already set up in *.jmx*.
+
+**Test Statistics**
+
+=======================  =============  ===========  ===============================  ===============================  ===============================
+**Total # of requests**  **Success %**    **TPS**    **Avg. time taken per request**  **Min. time taken per request**  **Max. time taken per request**
+=======================  =============  ===========  ===============================  ===============================  ===============================
+    4082                     100%           0.45              1297 ms                              4 ms                          63612 ms
+=======================  =============  ===========  ===============================  ===============================  ===============================
+
+.. image:: images/api-s3p-jm-2_H.png
+
+Test Results
+------------
+
+The following graphs show the response time distributions.
+
+.. image:: images/api-response-time-distribution_performance_H.png
+.. image:: images/api-response-time-overtime_performance_H.png
+
 
 
 
diff --git a/docs/development/devtools/images/api-response-time-distribution_performance_H.png b/docs/development/devtools/images/api-response-time-distribution_performance_H.png
new file mode 100644 (file)
index 0000000..68a5c79
Binary files /dev/null and b/docs/development/devtools/images/api-response-time-distribution_performance_H.png differ
diff --git a/docs/development/devtools/images/api-response-time-overtime_performance_H.png b/docs/development/devtools/images/api-response-time-overtime_performance_H.png
new file mode 100644 (file)
index 0000000..63fc859
Binary files /dev/null and b/docs/development/devtools/images/api-response-time-overtime_performance_H.png differ
diff --git a/docs/development/devtools/images/api-s3p-jm-2_H.png b/docs/development/devtools/images/api-s3p-jm-2_H.png
new file mode 100644 (file)
index 0000000..da9d1e9
Binary files /dev/null and b/docs/development/devtools/images/api-s3p-jm-2_H.png differ
index 7e36b3e..852498d 100644 (file)
@@ -89,7 +89,7 @@ System Limitations
 The policy API component requires a fresh new database when migrating to the honolulu release.
 Therefore, upgrades require a fresh new database installation.
 Please see the
-`Installing or Upgrading Policy <https://onap.readthedocs.io/en/latest/submodules/policy/parent.git/docs/installation/oom.html#installing-or-upgrading-policy>`__ section for appropriate procedures.
+`Installing or Upgrading Policy <https://docs.onap.org/projects/onap-policy-parent/en/honolulu/installation/oom.html#installing-or-upgrading-policy>`__ section for appropriate procedures.
 
 Known Vulnerabilities
 ~~~~~~~~~~~~~~~~~~~~~
@@ -225,7 +225,7 @@ System Limitations
 The policy API component requires a fresh new database when migrating to the guilin release.
 Therefore, upgrades require a fresh new database installation.
 Please see the
-`Installing or Upgrading Policy <https://onap.readthedocs.io/en/latest/submodules/policy/parent.git/docs/installation/oom.html#installing-or-upgrading-policy>`__ section for appropriate procedures.
+`Installing or Upgrading Policy <https://docs.onap.org/projects/onap-policy-parent/en/guilin/installation/oom.html#installing-or-upgrading-policy>`__ section for appropriate procedures.
 
 Known Vulnerabilities
 ~~~~~~~~~~~~~~~~~~~~~
@@ -476,14 +476,14 @@ POLICY-APEX-PDP
 * Passing parameters from ApexConfig to policy logic.
    - TaskParameters can be used to pass parameters from ApexConfig to the policy logic. Consider a scenario where from CLAMP, serviceId or closedLoopId has to be passed to the policy, and this should be available to perform some logic or action within the policy. In the CLAMP UI, while configuring the APEX Policy, specifying taskParameters with these will enable this.
 
-   - More information about the usage of Task Parameters can be found here: https://onap.readthedocs.io/en/latest/submodules/policy/parent.git/docs/apex/APEX-User-Manual.html#configure-task-parameters
+   - More information about the usage of Task Parameters can be found here: https://docs.onap.org/projects/onap-policy-parent/en/frankfurt/apex/APEX-User-Manual.html#configure-task-parameters
 
    - In the taskLogic, taskParameters can be accessed as  executor.parameters.get("ParameterKey1"))
 
-   - More information can be found here: https://onap.readthedocs.io/en/latest/submodules/policy/parent.git/docs/apex/APEX-Policy-Guide.html#accessing-taskparameters
+   - More information can be found here: https://docs.onap.org/projects/onap-policy-parent/en/frankfurt/apex/APEX-Policy-Guide.html#accessing-taskparameters
 
 * GRPC support for APEX-CDS interaction.
-   - APEX-PDP now supports interaction with CDS over gRPC. Up through El Alto, CDS interaction was possible over REST only. A new plugin was developed in APEX for this feature. Refer the link for more details. https://onap.readthedocs.io/en/latest/submodules/policy/parent.git/docs/apex/APEX-User-Manual.html#grpc-io
+   - APEX-PDP now supports interaction with CDS over gRPC. Up through El Alto, CDS interaction was possible over REST only. A new plugin was developed in APEX for this feature. Refer the link for more details. https://docs.onap.org/projects/onap-policy-parent/en/frankfurt/apex/APEX-User-Manual.html#grpc-io
 
 POLICY-XACML-PDP
 ================
@@ -530,7 +530,7 @@ System Limitations
 The policy API component requires a fresh new database when migrating to the frankfurt release.
 Therefore, upgrades require a fresh new database installation.
 Please see the
-`Installing or Upgrading Policy <https://onap.readthedocs.io/en/frankfurt/submodules/policy/parent.git/docs/installation/oom.html#installing-or-upgrading-policy>`__ section for appropriate procedures.
+`Installing or Upgrading Policy <https://docs.onap.org/projects/onap-policy-parent/en/frankfurt/installation/oom.html#installing-or-upgrading-policy>`__ section for appropriate procedures.
 
 Known Vulnerabilities
 ~~~~~~~~~~~~~~~~~~~~~