docs/development/devtools/clamp-smoke.rst

   1 .. This work is licensed under a
   2 .. Creative Commons Attribution 4.0 International License.
   3 .. http://creativecommons.org/licenses/by/4.0
   4
   5 .. _policy-clamp-runtime-smoke-label:
   6
   7 CLAMP Automation Composition Smoke Tests
   8 ########################################
   9
  10 .. contents::
  11     :depth: 3
  12
  13
  14 This article explains how to build the CLAMP automation composition for development purposes and how to run smoke tests for automation composition. To start, the developer should consult the latest ONAP Wiki to familiarize themselves with developer best practices and how-tos to setup their environment, see `https://wiki.onap.org/display/DW/Developer+Best+Practices`.
  15
  16
  17 This article assumes that:
  18
  19 * You are using a *\*nix* operating system such as linux or macOS.
  20 * You are using a directory called *git* off your home directory *(~/git)* for your git repositories
  21 * Your local maven repository is in the location *~/.m2/repository*
  22 * You have copied the settings.xml from oparent to *~/.m2/* directory
  23 * You have added settings to access the ONAP Nexus to your M2 configuration, see `Maven Settings Example <https://wiki.onap.org/display/DW/Setting+Up+Your+Development+Environment>`_ (bottom of the linked page)
  24
  25 The procedure documented in this article has been verified using Ubuntu 20.04 LTS VM.
  26
  27 Cloning CLAMP automation composition and all dependency
  28 *******************************************************
  29
  30 Run a script such as the script below to clone the required modules from the `ONAP git repository <https://gerrit.onap.org/r/admin/repos/q/filter:policy>`_. This script clones CLAMP automation composition and all dependency.
  31
  32 ONAP Policy Framework has dependencies to the ONAP Parent *oparent* module, the ONAP ECOMP SDK *ecompsdkos* module, and the A&AI Schema module.
  33
  34
  35 .. code-block:: bash
  36    :caption: Typical ONAP Policy Framework Clone Script
  37    :linenos:
  38
  39     #!/usr/bin/env bash
  40
  41     ## script name for output
  42     MOD_SCRIPT_NAME='basename $0'
  43
  44     ## the ONAP clone directory, defaults to "onap"
  45     clone_dir="onap"
  46
  47     ## the ONAP repos to clone
  48     onap_repos="\
  49     policy/parent \
  50     policy/common \
  51     policy/models \
  52     policy/clamp \
  53     policy/docker "
  54
  55     ##
  56     ## Help screen and exit condition (i.e. too few arguments)
  57     ##
  58     Help()
  59     {
  60         echo ""
  61         echo "$MOD_SCRIPT_NAME - clones all required ONAP git repositories"
  62         echo ""
  63         echo "       Usage:  $MOD_SCRIPT_NAME [-options]"
  64         echo ""
  65         echo "       Options"
  66         echo "         -d          - the ONAP clone directory, defaults to '.'"
  67         echo "         -h          - this help screen"
  68         echo ""
  69         exit 255;
  70     }
  71
  72     ##
  73     ## read command line
  74     ##
  75     while [ $# -gt 0 ]
  76     do
  77         case $1 in
  78             #-d ONAP clone directory
  79             -d)
  80                 shift
  81                 if [ -z "$1" ]; then
  82                     echo "$MOD_SCRIPT_NAME: no clone directory"
  83                     exit 1
  84                 fi
  85                 clone_dir=$1
  86                 shift
  87             ;;
  88
  89             #-h prints help and exists
  90             -h)
  91                 Help;exit 0;;
  92
  93             *)    echo "$MOD_SCRIPT_NAME: undefined CLI option - $1"; exit 255;;
  94         esac
  95     done
  96
  97     if [ -f "$clone_dir" ]; then
  98         echo "$MOD_SCRIPT_NAME: requested clone directory '$clone_dir' exists as file"
  99         exit 2
 100     fi
 101     if [ -d "$clone_dir" ]; then
 102         echo "$MOD_SCRIPT_NAME: requested clone directory '$clone_dir' exists as directory"
 103         exit 2
 104     fi
 105
 106     mkdir $clone_dir
 107     if [ $? != 0 ]
 108     then
 109         echo cannot clone ONAP repositories, could not create directory '"'$clone_dir'"'
 110         exit 3
 111     fi
 112
 113     for repo in $onap_repos
 114     do
 115         repoDir=`dirname "$repo"`
 116         repoName=`basename "$repo"`
 117
 118         if [ ! -z $dirName ]
 119         then
 120             mkdir "$clone_dir/$repoDir"
 121             if [ $? != 0 ]
 122             then
 123                 echo cannot clone ONAP repositories, could not create directory '"'$clone_dir/repoDir'"'
 124                 exit 4
 125             fi
 126         fi
 127
 128         git clone https://gerrit.onap.org/r/${repo} $clone_dir/$repo
 129     done
 130
 131     echo ONAP has been cloned into '"'$clone_dir'"'
 132
 133
 134 Execution of the script above results in the following directory hierarchy in your *~/git* directory:
 135
 136     *  ~/git/onap
 137     *  ~/git/onap/policy
 138     *  ~/git/onap/policy/parent
 139     *  ~/git/onap/policy/common
 140     *  ~/git/onap/policy/models
 141     *  ~/git/onap/policy/clamp
 142     *  ~/git/onap/policy/docker
 143
 144
 145 Building CLAMP automation composition and all dependency
 146 ********************************************************
 147
 148 **Step 1:** Optionally, for a completely clean build, remove the ONAP built modules from your local repository.
 149
 150     .. code-block:: bash
 151
 152         rm -fr ~/.m2/repository/org/onap
 153
 154
 155 **Step 2:**  A pom such as the one below can be used to build the ONAP Policy Framework modules. Create the *pom.xml* file in the directory *~/git/onap/policy*.
 156
 157 .. code-block:: xml
 158   :caption: Typical pom.xml to build the ONAP Policy Framework
 159   :linenos:
 160
 161     <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
 162         <modelVersion>4.0.0</modelVersion>
 163         <groupId>org.onap</groupId>
 164         <artifactId>onap-policy</artifactId>
 165         <version>1.0.0-SNAPSHOT</version>
 166         <packaging>pom</packaging>
 167         <name>${project.artifactId}</name>
 168         <inceptionYear>2017</inceptionYear>
 169         <organization>
 170             <name>ONAP</name>
 171         </organization>
 172
 173         <modules>
 174             <module>parent</module>
 175             <module>common</module>
 176             <module>models</module>
 177             <module>clamp</module>
 178         </modules>
 179     </project>
 180
 181
 182 **Step 3:** You can now build the Policy framework.
 183
 184 Build java artifacts only:
 185
 186     .. code-block:: bash
 187
 188        cd ~/git/onap/policy
 189        mvn -pl '!org.onap.policy.clamp:policy-clamp-runtime' install
 190
 191 Build with docker images:
 192
 193     .. code-block:: bash
 194
 195        cd ~/git/onap/policy/clamp/packages/
 196        mvn clean install -P docker
 197
 198 Running MariaDb and DMaaP Simulator
 199 ***********************************
 200
 201 Running a MariaDb Instance
 202 ++++++++++++++++++++++++++
 203
 204 Assuming you have successfully built the codebase using the instructions above. There are two requirements for the Clamp automation composition component to run, one of them is a
 205 running MariaDb database instance. The easiest way to do this is to run the docker image locally.
 206
 207 A sql such as the one below can be used to build the SQL initialization. Create the *mariadb.sql* file in the directory *~/git*.
 208
 209     .. code-block:: SQL
 210
 211        create database clampacm;
 212        CREATE USER 'policy'@'%' IDENTIFIED BY 'P01icY';
 213        GRANT ALL PRIVILEGES ON clampacm.* TO 'policy'@'%';
 214
 215
 216 Execution of the command above results in the creation and start of the *mariadb-smoke-test* container.
 217
 218     .. code-block:: bash
 219
 220        cd ~/git
 221        docker run --name mariadb-smoke-test  \
 222         -p 3306:3306 \
 223         -e MYSQL_ROOT_PASSWORD=my-secret-pw  \
 224         --mount type=bind,source=~/git/mariadb.sql,target=/docker-entrypoint-initdb.d/data.sql \
 225         mariadb:10.5.8
 226
 227
 228 Running the DMaaP Simulator during Development
 229 ++++++++++++++++++++++++++++++++++++++++++++++
 230 The second requirement for the Clamp automation composition component to run is to run the DMaaP simulator. You can run it from the command line using Maven.
 231
 232
 233 Change the local configuration file *src/test/resources/simParameters.json* using the below code:
 234
 235 .. code-block:: json
 236
 237    {
 238      "dmaapProvider": {
 239        "name": "DMaaP simulator",
 240        "topicSweepSec": 900
 241      },
 242      "restServers": [
 243        {
 244          "name": "DMaaP simulator",
 245          "providerClass": "org.onap.policy.models.sim.dmaap.rest.DmaapSimRestControllerV1",
 246          "host": "localhost",
 247          "port": 3904,
 248          "https": false
 249        }
 250      ]
 251    }
 252
 253 Run the following commands:
 254
 255    .. code-block:: bash
 256
 257       cd ~/git/onap/policy/models/models-sim/policy-models-simulators
 258       mvn exec:java  -Dexec.mainClass=org.onap.policy.models.simulators.Main -Dexec.args="src/test/resources/simParameters.json"
 259
 260
 261 Developing and Debugging CLAMP automation composition
 262 *****************************************************
 263
 264 Running on the Command Line using Maven
 265 +++++++++++++++++++++++++++++++++++++++
 266
 267 Once the mariadb and DMaap simulator are up and running, run the following commands:
 268
 269    .. code-block:: bash
 270
 271       cd ~/git/onap/policy/clamp/runtime-acm
 272       mvn spring-boot:run
 273
 274
 275 Running on the Command Line
 276 +++++++++++++++++++++++++++
 277
 278    .. code-block:: bash
 279
 280       cd ~/git/onap/policy/clamp/runtime-acm
 281       java -jar target/policy-clamp-runtime-acm-6.2.2-SNAPSHOT.jar
 282
 283
 284 Running in Eclipse
 285 ++++++++++++++++++
 286
 287 1. Check out the policy models repository
 288 2. Go to the *policy-clamp-runtime-acm* module in the clamp repo
 289 3. Specify a run configuration using the class *org.onap.policy.clamp.acm.runtime.Application* as the main class
 290 4. Run the configuration
 291
 292 Swagger UI of Automation composition is available at *http://localhost:6969/onap/policy/clamp/acm/swagger-ui/*, and swagger JSON at *http://localhost:6969/onap/policy/clamp/acm/v2/api-docs/*
 293
 294
 295 Running one or more participant simulators
 296 ++++++++++++++++++++++++++++++++++++++++++
 297
 298 Into *docker\csit\clamp\tests\data* you can find a test case with policy-participant. In order to use that test you can use particpant-simulator.
 299 Copy the file *src/main/resources/config/application.yaml* and paste into *src/test/resources/*, after that change *participantId* and *participantType* as showed below:
 300
 301    .. code-block:: yaml
 302
 303       participantId:
 304         name: org.onap.PM_Policy
 305         version: 1.0.0
 306       participantType:
 307         name: org.onap.policy.clamp.acm.PolicyParticipant
 308         version: 2.3.1
 309
 310 Run the following commands:
 311
 312    .. code-block:: bash
 313
 314       cd ~/git/onap/policy/clamp/participant/participant-impl/participant-impl-simulator
 315       java -jar target/policy-clamp-participant-impl-simulator-6.2.2-SNAPSHOT.jar --spring.config.location=src/main/resources/config/application.yaml
 316
 317
 318 Creating self-signed certificate
 319 ++++++++++++++++++++++++++++++++
 320
 321 There is an additional requirement for the Clamp automation composition docker image to run, is creating the SSL self-signed certificate.
 322
 323 Run the following commands:
 324
 325    .. code-block:: bash
 326
 327       cd ~/git/onap/policy/docker/csit/
 328       ./gen_truststore.sh
 329       ./gen_keystore.sh
 330
 331 Execution of the commands above results additional files into the following directory *~/git/onap/policy/docker/csit/config*:
 332
 333     *  ~/git/onap/policy/docker/csit/config/cakey.pem
 334     *  ~/git/onap/policy/docker/csit/config/careq.pem
 335     *  ~/git/onap/policy/docker/csit/config/caroot.cer
 336     *  ~/git/onap/policy/docker/csit/config/ks.cer
 337     *  ~/git/onap/policy/docker/csit/config/ks.csr
 338     *  ~/git/onap/policy/docker/csit/config/ks.jks
 339
 340
 341 Running the CLAMP automation composition docker image
 342 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 343
 344 Run the following command:
 345
 346    .. code-block:: bash
 347
 348       docker run --name runtime-smoke-test \
 349        -p 6969:6969 \
 350        -e mariadb.host=host.docker.internal \
 351        -e topicServer=host.docker.internal \
 352        --mount type=bind,source=~/git/onap/policy/docker/csit/config/ks.jks,target=/opt/app/policy/clamp/etc/ssl/policy-keystore  \
 353        --mount type=bind,source=~/git/onap/policy/clamp/runtime-acm/src/main/resources/application.yaml,target=/opt/app/policy/clamp/etc/AcRuntimeParameters.yaml  \
 354        onap/policy-clamp-runtime-acm
 355
 356
 357 Swagger UI of automation composition is available at *https://localhost:6969/onap/policy/clamp/acm/swagger-ui/*, and swagger JSON at *https://localhost:6969/onap/policy/clamp/acm/v2/api-docs/*
 358
 359
 360 Using CLAMP runtime to connect to CLAMP automation composition
 361 **************************************************************
 362
 363 Build CLAMP runtime image:
 364
 365     .. code-block:: bash
 366
 367        cd ~/git/onap/policy/clamp/runtime
 368        mvn clean install -P docker -DskipTests
 369
 370
 371 Run the following docker composition:
 372
 373    .. code-block:: yaml
 374
 375       version: '3.1'
 376
 377       services:
 378         db:
 379           image: mariadb:10.5.8
 380           volumes:
 381              - "~/git/onap/policy/clamp/runtime/extra/sql/:/docker-entrypoint-initdb.d:rw"
 382           environment:
 383             - MYSQL_ROOT_PASSWORD=strong_pitchou
 384           ports:
 385             - "3306:3306"
 386
 387         policy-clamp-backend:
 388           image: onap/policy-clamp-backend
 389           depends_on:
 390             - db
 391             - third-party-proxy
 392           environment:
 393             - SPRING_DATASOURCE_URL=jdbc:mariadb:sequential://db:3306/cldsdb4?autoReconnect=true&connectTimeout=10000&socketTimeout=10000&retriesAllDown=3
 394             - SPRING_PROFILES_ACTIVE=clamp-default,clamp-default-user,clamp-sdc-controller,clamp-ssl-config,clamp-policy-controller,default-dictionary-elements
 395             - CLAMP_CONFIG_POLICY_API_URL=http://third-party-proxy:8085
 396             - CLAMP_CONFIG_ACM_RUNTIME_URL=http://host.docker.internal:6969
 397             - CLAMP_CONFIG_POLICY_PAP_URL=http://third-party-proxy:8085
 398             - CLAMP_CONFIG_DCAE_INVENTORY_URL=http://third-party-proxy:8085
 399             - CLAMP_CONFIG_DCAE_DEPLOYMENT_URL=http://third-party-proxy:8085
 400             - SPRING_CONFIG_LOCATION=classpath:/application.properties
 401           ports:
 402             - "10443:8443"
 403
 404         third-party-proxy:
 405           image: python:2-slim
 406           volumes:
 407             - "~/git/onap/policy/clamp/runtime/src/test/resources/http-cache/example/:/thirdparty:rw"
 408             - "~/git/onap/policy/clamp/runtime/src/test/resources/http-cache/:/script/:ro"
 409           ports:
 410             - "8085:8085"
 411           command: /bin/sh -c "pip install --no-cache-dir requests &&  pip install --no-cache-dir simplejson && python -u /script/third_party_proxy.py -v true --port 8085 --root /thirdparty --proxyaddress third-party-proxy:8085"
 412
 413
 414 Run DMaaP simulator, and then run CLAMP Acm using java.