.. code-block:: bash
- jmeter.sh --nongui --testfile policy_api_stability.jmx --logfile result.jtl
+ nohup ./apache-jmeter-5.4.1/bin/jmeter.sh -n -t policy_api_stability.jmx -l stabilityTestResultsPolicyApi.jtl
+The test was run in the background via “nohup”, to prevent it from being interrupted.
Test Plan
---------
**API Test Flow (5 threads running the same steps in the same loop)**
+- Get Policy Metrics
- Create a new Monitoring Policy Type with Version 6.0.#
- Create a new Monitoring Policy Type with Version 7.0.#
- Create a new Optimization Policy Type with Version 6.0.#
======================= ============= =========== =============================== =============================== ===============================
**Total # of requests** **Success %** **TPS** **Avg. time taken per request** **Min. time taken per request** **Max. time taken per request**
======================= ============= =========== =============================== =============================== ===============================
- 176407 100% 0.68 7340 ms 34 ms 49298 ms
+ 242277 100% 0.935 5340 ms 1 ms 736976 ms
======================= ============= =========== =============================== =============================== ===============================
+.. image:: api-s3p-results/api-s3p-jm-1_I.png
**JMeter Results**
The following graphs show the response time distributions. The "Get Policy Types" API calls are the most expensive calls that
-average a 10 seconds plus response time.
+average a 7 seconds plus response time.
-.. image:: images/api-response-time-distribution.png
-.. image:: images/api-response-time-overtime.png
+.. image:: api-s3p-results/api-response-time-distribution_I.png
+.. image:: api-s3p-results/api-response-time-overtime_I.png
+
+**Memory and CPU usage**
+
+The memory and CPU usage can be monitored by running "top" command in the policy-api pod.
+A snapshot is taken before and after test execution to monitor the changes in resource utilization.
+
+Memory and CPU usage before test execution:
+
+.. image:: api-s3p-results/api_top_before_72h.jpg
+
+Memory and CPU usage after test execution:
+
+.. image:: api-s3p-results/api_top_after_72h.jpg
Performance Test of Policy API
++++++++++++++++++++++++++++++
-A specific performance test was omitted in Guilin. The JMeter script used in the stability run injected
-back to back traffic with 5 parallel threads with no pauses between requests. Since the JMeter threads operate
-in synchronous mode (waiting for a request's response before sending the next request), JMeter injection rates autoregulate
-because of the backpressure imposed by the response times. Even though the response times are high, the
-"Response over Time" graph above indicates that they remain constant at large, throughout the duration of the test.
-This together with the absence of notorious spikes in the kubernetes node CPU utilization suggests that the API
-component is not strained. A more enlightning set of tests, would plot jmeter threads (increasing load)
-against response times. These tests have not been performed in this release.
+Introduction
+------------
+
+Performance test of policy-api has the goal of testing the min/avg/max processing time and rest call throughput for all the requests when the number of requests are large enough to saturate the resource and find the bottleneck.
+
+Setup Details
+-------------
+
+The performance test was performed on a default ONAP OOM installation in the Intel Wind River Lab environment.
+JMeter was installed on a separate VM to inject the traffic defined in the
+`API performace script
+<https://git.onap.org/policy/api/tree/testsuites/performance/src/main/resources/testplans/policy_api_performance.jmx>`_
+with the following command:
+
+.. code-block:: bash
+
+ nohup ./apache-jmeter-5.4.1/bin/jmeter.sh -n -t policy_api_performance.jmx -l performanceTestResultsPolicyApi.jtl
+
+The test was run in the background via “nohup”, to prevent it from being interrupted.
+
+Test Plan
+---------
+
+Performance test plan is the same as stability test plan above.
+Only differences are, in performance test, we increase the number of threads up to 20 (simulating 20 users' behaviors at the same time) whereas reducing the test time down to 2.5 hours.
+
+Run Test
+--------
+
+Running/Triggering performance test will be the same as stability test. That is, launch JMeter pointing to corresponding *.jmx* test plan. The *API_HOST* and *API_PORT* are already set up in *.jmx*.
+
+**Test Statistics**
+
+======================= ============= =========== =============================== =============================== ===============================
+**Total # of requests** **Success %** **TPS** **Avg. time taken per request** **Min. time taken per request** **Max. time taken per request**
+======================= ============= =========== =============================== =============================== ===============================
+ 2822 100% 0.31 63794 ms 2 ms 1183376 ms
+======================= ============= =========== =============================== =============================== ===============================
+
+.. image:: api-s3p-results/api-s3p-jm-2_I.png
+
+Test Results
+------------
+
+The following graphs show the response time distributions.
+
+.. image:: api-s3p-results/api-response-time-distribution_performance_I.png
+.. image:: api-s3p-results/api-response-time-overtime_performance_I.png
+
Code Review / policy / parent.git / blobdiff