.. toctree::
:maxdepth: 2
-Policy XACML PDP component
##########################
-Both the Performance and the Stability tests were executed by performing requests
-against the Policy RESTful APIs residing on the XACML PDP installed in the windriver
-lab. This was running on a kubernetes pod having the following configuration:
+Performance Test of Policy XACML PDP (Jakarta)
+**********************************************
-- 16GB RAM
-- 8 VCPU
-- 160GB Disk
+The Performance test was executed by performing requests
+against the Policy RESTful APIs.
-Both tests were run via jmeter, which was installed on a separate VM so-as not
-to impact the performance of the XACML-PDP being tested.
+A default ONAP installation in the Policy tenant in UNH was used to run the tests.
-Performance Test of Policy XACML PDP
-************************************
+The Agent VMs in this lab have the following configuration:
+
+- 16GB RAM
+- 8 VCPU
Summary
=======
-The Performance test was executed, and the result analyzed, via:
+The Performance test was executed, and the result analysed, via:
.. code-block:: bash
jmeter -Jduration=1200 -Jusers=10 \
-Jxacml_ip=$ip -Jpap_ip=$ip -Japi_ip=$ip \
- -Jxacml_port=31104 -Jpap_port=32425 -Japi_port=30709 \
- -n -t perf.jmx
-
- ./result.sh
+ -Jxacml_port=30111 -Jpap_port=30197 -Japi_port=30664 \
+ -n -t perf.jmx -l testresults.jtl
Note: the ports listed above correspond to port 6969 of the respective components.
-The performance test, perf.jmx, runs the following, all in parallel:
+The performance tests runs the following, all in parallel:
- Healthcheck, 10 simultaneous threads
- Statistics, 10 simultaneous threads
Results
=======
-The test was run for 20 minutes at a time, for different numbers of users (i.e.,
-threads), with the following results:
+The test was run for 20 minutes with 10 users (i.e., threads), with the following results:
.. csv-table::
:header: "Number of Users", "Throughput (requests/second)", "Average Latency (ms)"
- 10, 6064, 4.1
- 20, 6495, 7.2
- 40, 6457, 12.2
- 80, 5803, 21.3
+ 10, 4603, 2
+
+.. image:: images/s3p-perf-xacml.png
Stability Test of Policy XACML PDP
-************************************
+**********************************
+
+This test was run using jmeter on a default
+ONAP installation in the Policy tenant in UNH.
+
+The Agent VMs in this lab have the following configuration:
+
+- 16GB RAM
+- 8 VCPU
Summary
=======
-The Stability test was run with the same pods/VMs and uses the same jmeter script as the
-performance test, except that it was run for 72 hours instead of 20 minutes. In
-addition, it was run in the background via "nohup", to prevent it from being interrupted:
+The stability test was performed on a default ONAP OOM installation in the Policy tenant of the UNH lab.
+JMeter injected the traffic defined in the
+`XACML PDP stability script
+<https://git.onap.org/policy/xacml-pdp/tree/testsuites/stability/src/main/resources/testplans/stability.jmx>`_
+with the following command:
.. code-block:: bash
- nohup jmeter -Jduration=259200 \
- -Jxacml_ip=$ip -Jpap_ip=$ip -Japi_ip=$ip \
- -Jxacml_port=31104 -Jpap_port=32425 -Japi_port=30709 \
- -n -t perf.jmx &
-
-The memory and CPU usage can be monitored by running "top" on the xacml pod. By taking
-a snapshot before the test is started, and again when it completes, the total CPU used
-by all of the requests can be computed.
+ jmeter.sh -Jduration=259200 -Jusers=2 -Jxacml_ip=$ip -Jpap_ip=$ip -Japi_ip=$ip \
+ -Jxacml_port=30111 -Jpap_port=30197 -Japi_port=30664 --nongui --testfile stability.jmx
-Results
-=======
+Note: the ports listed above correspond to port 6969 of the respective components.
-The final output of the jmeter script is found in the nohup.out file:
+The default log level of the root and org.eclipse.jetty.server.RequestLog loggers in the logback.xml
+of the XACML PDP
+(om/kubernetes/policy/components/policy-xacml-pdp/resources/config/logback.xml)
+was set to WARN since the OOM installation did have log rotation enabled of the
+container logs in the kubernetes worker nodes.
-.. image:: images/xacml-s3p-jmeter.png
+The stability test, stability.jmx, runs the following, all in parallel:
-The final memory and CPU from "top":
+- Healthcheck, 2 simultaneous threads
+- Statistics, 2 simultaneous threads
+- Decisions, 2 simultaneous threads, each running the following tasks in sequence:
+ - Monitoring Decision
+ - Monitoring Decision, abbreviated
+ - Naming Decision
+ - Optimization Decision
+ - Default Guard Decision (always "Permit")
+ - Frequency Limiter Guard Decision
+ - Min/Max Guard Decision
-.. image:: images/xacml-s3p-top.png
+When the script starts up, it uses policy-api to create, and policy-pap to deploy
+the policies that are needed by the test. It assumes that the "naming" policy has
+already been created and deployed. Once the test completes, it undeploys and deletes
+the policies that it previously created.
-The through-put reported by jmeter was 4849 requests/second, with 0 errors. In addition,
-the memory usage observed via "top" indicated that the virtual memory and resident set
-sizes remained virtually unchanged through-out the test.
+Results
+=======
-Unfortunately, the initial CPU usage was not recorded, so the CPU time reported in
-the "top" screen-shot includes XACML-PDP start-up time as well as requests that were
-executed before the stability test was started. Nevertheless, even including that, we find:
+The stability summary results were reported by JMeter with the following summary line:
.. code-block:: bash
- 13,166 CPU minutes * 60sec/min * 1000ms/sec / 1,256,834,239 requests = 0.63ms/request
+ summary = 941639699 in 71:59:36 = 3633.2/s Avg: 1 Min: 0 Max: 842 Err: 0 (0.00%)
+
+The XACML PDP offered very good performance with JMeter for the traffic mix described above.
+The average transaction time is insignificant. The maximum transaction time of 842 ms.
+There was a Drools stability test running in parallel, hence the actual load was higher.
+
Code Review / policy / parent.git / blobdiff