Setup Details
-------------
-The stability test was performed on a default ONAP OOM installation in the Intel Wind River Lab environment.
+The stability test was performed on a default ONAP OOM installation in the Nordix Lab environment.
JMeter was installed on a separate VM to inject the traffic defined in the
`API stability script
<https://git.onap.org/policy/api/tree/testsuites/stability/src/main/resources/testplans/policy_api_stability.jmx>`_
.. code-block:: bash
- nohup ./apache-jmeter-5.4.1/bin/jmeter.sh -n -t policy_api_stability.jmx -l stabilityTestResultsPolicyApi.jtl
+ nohup apache-jmeter-5.5/bin/jmeter -n -t policy_api_stability.jmx -l stabilityTestResultsPolicyApi.jtl &
-The test was run in the background via “nohup”, to prevent it from being interrupted.
+The test was run in the background via “nohup” and “&”, to prevent it from being interrupted.
Test Plan
---------
**API Test Flow (5 threads running the same steps in the same loop)**
-- Get Policy Metrics
- Create a new Monitoring Policy Type with Version 6.0.#
- Create a new Monitoring Policy Type with Version 7.0.#
- Create a new Optimization Policy Type with Version 6.0.#
- Get Version 6.0.# of the new Native Drools Policy Type
- Get Version 6.0.# of the new Native XACML Policy Type
- Get the Latest Version of the New Monitoring Policy Type
+- Create Version 6.0.# of Node Template
- Create Monitoring Policy Ver 6.0.# w/Monitoring Policy Type Ver 6.0.#
- Create Monitoring Policy Ver 7.0.# w/Monitoring Policy Type Ver 7.0.#
- Create Optimization Policy Ver 6.0.# w/Optimization Policy Type Ver 6.0.#
- Create Native APEX Policy Ver 6.0.# w/Native APEX Policy Type Ver 6.0.#
- Create Native Drools Policy Ver 6.0.# w/Native Drools Policy Type Ver 6.0.#
- Create Native XACML Policy Ver 6.0.# w/Native XACML Policy Type Ver 6.0.#
+- Create Version 6.0.# of PNF Example Policy with Metadata
+- Get Node Template
+- Get All TCA Policies
+- Get All Versions of Monitoring Policy Type
- Get Version 6.0.# of the new Monitoring Policy
- Get Version 6.0.# of the new Optimization Policy
- Get Version 6.0.# of the new Guard Policy
- Delete Version 6.0.# of the new OptimizationPolicy
- Delete Version 6.0.# of the new Guard Policy
- Delete Version 6.0.# of the new Native APEX Policy
+- Delete Version 6.0.# of PNF Example Policy having Metadata
- Delete Version 6.0.# of the new Native Drools Policy
- Delete Version 6.0.# of the new Native XACML Policy
- Delete Monitoring Policy Type with Version 6.0.#
- Delete Native APEX Policy Type with Version 6.0.#
- Delete Native Drools Policy Type with Version 6.0.#
- Delete Native XACML Policy Type with Version 6.0.#
+- Delete Node Template
+- Get Policy Metrics
**TearDown Thread (will only be running after API Test Flow is completed)**
======================= ============= =========== =============================== =============================== ===============================
**Total # of requests** **Success %** **TPS** **Avg. time taken per request** **Min. time taken per request** **Max. time taken per request**
======================= ============= =========== =============================== =============================== ===============================
- 242277 100% 0.935 5340 ms 1 ms 736976 ms
+ 950839 100% 3.67 1351 ms 126 ms 16324 ms
======================= ============= =========== =============================== =============================== ===============================
-.. image:: api-s3p-results/api-s3p-jm-1_I.png
+.. image:: api-s3p-results/api-s3p-jm-1_J.png
**JMeter Results**
The following graphs show the response time distributions. The "Get Policy Types" API calls are the most expensive calls that
-average a 7 seconds plus response time.
+average a 13 seconds plus response time.
-.. image:: api-s3p-results/api-response-time-distribution_I.png
-.. image:: api-s3p-results/api-response-time-overtime_I.png
+.. image:: api-s3p-results/api-response-time-distribution_J.png
+.. image:: api-s3p-results/api-response-time-overtime_J.png
**Memory and CPU usage**
Memory and CPU usage before test execution:
-.. image:: api-s3p-results/api_top_before_72h.jpg
+.. image:: api-s3p-results/api_top_before_72h.png
Memory and CPU usage after test execution:
-.. image:: api-s3p-results/api_top_after_72h.jpg
+.. image:: api-s3p-results/api_top_after_72h.png
Performance Test of Policy API
Setup Details
-------------
-The performance test was performed on a default ONAP OOM installation in the Intel Wind River Lab environment.
+The performance test was performed on a default ONAP OOM installation in the Nordix Lab environment.
JMeter was installed on a separate VM to inject the traffic defined in the
`API performance script
<https://git.onap.org/policy/api/tree/testsuites/performance/src/main/resources/testplans/policy_api_performance.jmx>`_
.. code-block:: bash
- nohup ./apache-jmeter-5.4.1/bin/jmeter.sh -n -t policy_api_performance.jmx -l performanceTestResultsPolicyApi.jtl
+ nohup apache-jmeter-5.5/bin/jmeter -n -t policy_api_performance.jmx -l performanceTestResultsPolicyApi.jtl &
-The test was run in the background via “nohup”, to prevent it from being interrupted.
+The test was run in the background via “nohup” and “&”, to prevent it from being interrupted.
Test Plan
---------
======================= ============= =========== =============================== =============================== ===============================
**Total # of requests** **Success %** **TPS** **Avg. time taken per request** **Min. time taken per request** **Max. time taken per request**
======================= ============= =========== =============================== =============================== ===============================
- 2822 100% 0.31 63794 ms 2 ms 1183376 ms
+ 16212 100% 1.8 11109 ms 162 ms 237265 ms
======================= ============= =========== =============================== =============================== ===============================
-.. image:: api-s3p-results/api-s3p-jm-2_I.png
+.. image:: api-s3p-results/api-s3p-jm-2_J.png
Test Results
------------
The following graphs show the response time distributions.
-.. image:: api-s3p-results/api-response-time-distribution_performance_I.png
-.. image:: api-s3p-results/api-response-time-overtime_performance_I.png
+.. image:: api-s3p-results/api-response-time-distribution_performance_J.png
+.. image:: api-s3p-results/api-response-time-overtime_performance_J.png