----------
The stability and performance tests are performed on VM's running in the OpenStack cloud
-environment in the ONAP integration lab. There are two separate VMs, one for running backend policy
-services which policy distribution needs, and the other for the policy distribution service itself
-and Jmeter.
+environment in the ONAP integration lab.
-**OpenStack environment details**
+**Policy VM details**
-- Version: Windriver Titanium
-
-**Policy Backend VM details (VM1)**
-
-- OS: Ubuntu 18.04.5 LTS
-- CPU: 8 core, Intel Xeon E3-12xx v2 (Ivy Bridge), 2693.668 MHz, 16384 kB cache
-- RAM: 32 GB
-- HardDisk: 200 GB
-- Docker version 19.03.8, build afacb8b7f0
-- Java: openjdk 11.0.8 2020-07-14
+- OS: Ubuntu 18.04 LTS (GNU/Linux 4.15.0-151-generic x86_64)
+- CPU: 4 core
+- RAM: 15 GB
+- HardDisk: 39 GB
+- Docker version 20.10.7, build 20.10.7-0ubuntu1~18.04.2
+- Java: openjdk 11.0.11 2021-04-20
Common Setup
.. code-block:: bash
$ java --version
- openjdk 11.0.8 2020-07-14
- OpenJDK Runtime Environment (build 11.0.8+10-post-Ubuntu-0ubuntu118.04.1)
- OpenJDK 64-Bit Server VM (build 11.0.8+10-post-Ubuntu-0ubuntu118.04.1, mixed mode, sharing)
+ openjdk 11.0.11 2021-04-20
+ OpenJDK Runtime Environment (build 11.0.11+9-Ubuntu-0ubuntu2.18.04)
+ OpenJDK 64-Bit Server VM (build 11.0.11+9-Ubuntu-0ubuntu2.18.04, mixed mode)
Install Docker and Docker Compose
Connect to Distribution JMX Port.
- 1. Right click on "Local" in the left panel of the screen and select "Add JMX Connection"
+ 1. On the visualvm toolbar, click on "Add JMX Connection"
2. Enter the Distribution container IP and Port 9090. This is the JMX port exposed by the
distribution container
- 3. Double click on the newly added nodes under "Local" to start monitoring CPU, Memory & GC.
+ 3. Double click on the newly added nodes under "Remotes" to start monitoring CPU, Memory & GC.
Example Screenshot of visualVM
-.. image:: distribution-s3p-results/distribution-s3p-vvm-sample.png
+.. image:: distribution-s3p-results/distribution-visualvm-snapshot.png
Stability Test of Policy Distribution
- **Add CSAR** - Adds CSAR to the directory that distribution is watching
- **Get Healthcheck** - Ensures Healthcheck is returning 200 OK
- **Get Statistics** - Ensures Statistics is returning 200 OK
-- **CheckPDPGroupQuery** - Checks that PDPGroupQuery contains the deployed policy
-- **CheckPolicyDeployed** - Checks that the policy is deployed
-- **Undeploy Policy** - Undeploys the policy
-- **Delete Policy** - Deletes the Policy for the next loop
-- **Check PDP Group for Deletion** - Ensures the policy has been removed and does not exist
+- **Assert PDP Group Query** - Checks that PDPGroupQuery contains the deployed policy
+- **Assert PoliciesDeployed** - Checks that the policy is deployed
+- **Undeploy/Delete Policy** - Undeploys and deletes the Policy for the next loop
+- **Assert PDP Group Query for Deleted Policy** - Ensures the policy has been removed and does not exist
The following steps can be used to configure the parameters of the test plan.
Screenshot of Distribution stability test plan
-.. image:: distribution-s3p-results/distribution-s3p-testplan.png
+.. image:: distribution-s3p-results/distribution-jmeter-testcases.png
Running the Test Plan
script execution. If not, run the following commands to create folder and change folder permissions
to allow the testplan to insert the CSAR into the /tmp/policydistribution/distributionmount folder.
+.. note::
+ Make sure that only csar file is being loaded in the watched folder and log generation is in a
+ logs folder, as any sort of zip file can be understood by distribution as a policy file. A
+ logback.xml configuration file is available under setup/distribution folder.
+
.. code-block:: bash
sudo mkdir -p /tmp/policydistribution/distributionmount
**Test Statistics**
-.. image:: distribution-s3p-results/dist_stability_statistics.PNG
-.. image:: distribution-s3p-results/dist_stability_threshold.PNG
+.. image:: distribution-s3p-results/stability-statistics.png
+.. image:: distribution-s3p-results/stability-threshold.png
**VisualVM Screenshots**
-.. image:: distribution-s3p-results/dist_stability_monitor.PNG
-.. image:: distribution-s3p-results/dist_stability_threads.PNG
+.. image:: distribution-s3p-results/stability-monitor.png
+.. image:: distribution-s3p-results/stability-threads.png
Performance Test of Policy Distribution
CLAMP Policy Participant Smoke Tests
------------------------------------
-
1. Introduction
***************
The Smoke testing of the policy participant is executed in a local CLAMP/Policy environment. The CLAMP-Controlloop interfaces interact with the Policy Framework to perform actions based on the state of the policy participant. The goal of the Smoke tests is the ensure that CLAMP Policy Participant and Policy Framework work together as expected.
2.2 Assumptions
===============
+
- You are accessing the policy repositories through gerrit
- You are using "git review".
+
The following repositories are required for development in this project. These repositories should be present on your machine and you should run "mvn clean install" on all of them so that the packages are present in your .m2 repository.
+
- policy/parent
- policy/common
- policy/models
2.3 Setting up the components
=============================
+
2.3.1 MariaDB Setup
^^^^^^^^^^^^^^^^^^^
+
We will be using Docker to run our mariadb instance. It will have a total of two databases running in it.
+
- controlloop: the runtime-controlloop db
- policyadmin: the policy-api db
+
The easiest way to do this is to perform a small alteration on an SQL script provided by the clamp backend in the file "runtime/extra/sql/bulkload/create-db.sql"
+
.. code-block:: mysql
+
CREATE DATABASE `controlloop`;
USE `controlloop`;
DROP USER 'policy';
CREATE USER 'policy_user';
GRANT ALL on controlloop.* to 'policy_user' identified by 'policy_user' with GRANT OPTION;
FLUSH PRIVILEGES;
+
Once this has been done, we can run the bash script provided here: "runtime/extra/bin-for-dev/start-db.sh"
+
.. code-block:: bash
+
./start-db.sh
+
This will setup the two databases needed. The database will be exposed locally on port 3306 and will be backed by an anonymous docker volume.
2.3.2 DMAAP Simulator
For convenience, a dmaap simulator has been provided in the policy/models repository. To start the simulator, you can do the following:
1. Navigate to /models-sim/policy-models-simulators in the policy/models repository.
2. Add a configuration file to src/test/resources with the following contents:
+
.. code-block:: json
+
{
"dmaapProvider":{
"name":"DMaaP simulator",
}
]
}
+
3. You can then start dmaap with:
+
.. code-block:: bash
+
mvn exec:java -Dexec.mainClass=org.onap.policy.models.simulators.Main -Dexec.args="src/test/resources/YOUR_CONF_FILE.json"
+
At this stage the dmaap simulator should be running on your local machine on port 3904.
2.3.3 Policy API
^^^^^^^^^^^^^^^^
In the policy-api repo, you should find the file "src/main/resources/etc/defaultConfig.json". This file must be altered slightly - as below with the restServerParameters and databaseProviderParameters shown. Note how the database parameters match-up with what you setup in Mariadb:
+
.. code-block:: json
+
{
"restServerParameters": {
"host": "0.0.0.0",
"persistenceUnit": "PolicyMariaDb"
},
}
+
Next, navigate to the "/main" directory. You can then run the following command to start the policy api:
+
.. code-block:: bash
+
mvn exec:java -Dexec.mainClass=org.onap.policy.api.main.startstop.Main -Dexec.args=" -c ../packages/policy-api-tarball/src/main/resources/etc/defaultConfig.json"
2.3.4 Policy PAP
^^^^^^^^^^^^^^^^
In the policy-pap repo, you should find the file 'main/src/test/resources/parameters/PapConfigParameters.json'. This file may need to be altered slightly as below:
+
.. code-block:: json
+
{
"name": "PapGroup",
"restServerParameters": {
"basePath": "healthcheck"
}]
}
+
Next, navigate to the "/main" directory. You can then run the following command to start the policy pap
+
.. code-block:: bash
+
mvn -q -e clean compile exec:java -Dexec.mainClass="org.onap.policy.pap.main.startstop.Main" -Dexec.args="-c /src/test/resources/parameters/PapConfigParameters.json"
2.3.5 Controlloop Runtime
^^^^^^^^^^^^^^^^^^^^^^^^^
To start the controlloop runtime we need to go the "runtime-controlloop" directory in the clamp repo. There is a config file that is used, by default, for the controlloop runtime. That config file is here: "src/main/resources/application.yaml". For development in your local environment, it shouldn't need any adjustment and we can just run the controlloop runtime with:
+
.. code-block:: bash
+
mvn spring-boot:run
2.3.6 Controlloop Policy Participant
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To start the policy participant we need to go to the "participant-impl/participant-impl-policy" directory in the clamp repo. There is a config file under "src/main/resources/config/application.yaml". For development in your local environment, we will need to adjust this file slightly:
+
.. code-block:: yaml
+
server:
port: 8082
servers:
- ${topicServer:localhost}
topicCommInfrastructure: dmaap
+
Navigate to the participant-impl/particpant-impl-policy/main directory. We can then run the policy-participant with the following command:
+
.. code-block:: bash
+
mvn spring-boot:run -Dspring-boot.run.arguments="--server.port=8082 --topicServer=localhost"
3. Testing Procedure
====================
+
3.1 Testing Outline
^^^^^^^^^^^^^^^^^^^
To perform the Smoke testing of the policy-participant we will be verifying the behaviours of the participant when the control loop changes state. The scenarios are:
+
- UNINITIALISED to PASSIVE: participant creates policies and policyTypes specified in the ToscaServiceTemplate using policy-api
- PASSIVE to RUNNING: participant deploys created policies specified in the ToscaServiceTemplate
- RUNNING to PASSIVE: participant undeploys policies which have been deployed
************************
A Control Loop is created by commissioning a Tosca template with Control loop definitions and instantiating the Control Loop with the state "UNINITIALISED".
Using postman, commision a TOSCA template and instantiate using the following template:
-:download:'Tosca Service Template <tosca/tosca_service_template_pptnt_smoke.yaml>'
-:download:'Instantiate Controlloop <tosca/instantiation_pptnt_smoke.json>'
+
+:download:`Tosca Service Template <tosca/tosca_service_template_pptnt_smoke.yaml>`
+
+:download:`Instantiate Controlloop <tosca/instantiation_pptnt_smoke.json>`
+
To verify this, we check that the Controlloop has been created and is in state UNINITIALISED.
+
.. image:: images/pol-part-controlloop-creation-ver.png
Creation of policies and policyTypes:
*************************************
The Controlloop STATE is changed from UNINITIALISED to PASSIVE using postman:
-.. code-block::json
+
+.. code-block:: json
+
{
"orderedState": "PASSIVE",
"controlLoopIdentifierList": [
}
]
}
+
This state change will trigger the creation of policies and policyTypes using the policy-api. To verify this we will check, using policy-api endpoints, that the "Sirisha" policyType, which is specified in the service template, has been created.
+
.. image:: images/pol-part-controlloop-sirisha-ver.png
+
We can also check that the pm-control policy has been created.
+
.. image:: images/pol-part-controlloop-pmcontrol-ver.png
Deployment of policies:
***********************
The Controlloop STATE is changed from PASSIVE to RUNNING using postman:
-.. code-block::json
+
+.. code-block:: json
+
{
"orderedState": "RUNNING",
"controlLoopIdentifierList": [
}
]
}
+
This state change will trigger the deployment of the policies specified in the ToscaServiceTemplate. To verify this, we will check that the apex pmcontrol policy has been deployed to the defaultGroup. We check this using pap:
+
.. image:: images/pol-part-controlloop-pmcontrol-deploy-ver.png
Undeployment of policies:
*************************
-
The Controlloop STATE is changed from RUNNING to PASSIVE using postman:
-.. code-block::json
+
+.. code-block:: json
+
{
"orderedState": "PASSIVE",
"controlLoopIdentifierList": [
}
]
}
+
This state change will trigger the undeployment of the pmcontrol policy which was deployed previously. To verifiy this we do a PdpGroup Query as before and check that the pmcontrol policy has been undeployed and removed from the defaultGroup:
+
.. image:: images/pol-part-controlloop-pmcontrol-undep-ver.png
Deletion of policies and policyTypes:
*************************************
The Controlloop STATE is changed from PASSIVE to UNINITIALISED using postman:
-.. code-block::json
+
+.. code-block:: json
+
{
"orderedState": "UNINITIALISED",
"controlLoopIdentifierList": [
}
]
}
+
This state change will trigger the deletion of the previously created policies and policyTypes. To verify this, as before, we can check that the Sirisha policyType is not found this time and likewise for the pmcontrol policy:
+
.. image:: images/pol-part-controlloop-sirisha-nf.png
+
.. image:: images/pol-part-controlloop-pmcontrol-nf.png
+
Code Review / policy / parent.git / commitdiff
