docs/sections/installation_manual.rst

   1 DCAE Installation
   2 =================
   3
   4 The below steps covers manual setup of DCAE VM’s and DCAE service
   5 components.
   6
   7 VESCollector
   8 ------------
   9
  10
  11
  12 DCAE VES Collector can be configured on VM with ubuntu-16.04 image
  13 (m1.small should suffice if this is only service) and 20Gb cinder
  14 storage
  15
  16 1. Install docker
  17
  18 .. code-block:: bash
  19
  20    sudo apt-get update
  21    sudo apt install `docker.io <http://docker.io/>`__
  22
  23 2. Pull the latest container from onap nexus
  24
  25 .. code-block:: bash
  26
  27    sudo docker login -u docker -p docker
  28    `nexus.onap.org <http://nexus.onap.org/>`__:10001
  29
  30    sudo docker pull
  31    `nexus.onap.org <http://nexus.onap.org/>`__:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.1
  32
  33 3. Start the VESCollector with below command
  34
  35 .. code-block:: bash
  36
  37    sudo docker run -d --name vescollector -p 8080:8080/tcp -p
  38    8443:8443/tcp -P -e DMAAPHOST='<dmaap IP>'
  39    `nexus.onap.org <http://nexus.onap.org/>`__:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.1
  40
  41 .. Note:  Change the dmaaphost to required DMAAP ip. To change the
  42    dmaap information for a running container,  stop the active
  43    container and rerun above command changing the dmaap IP.
  44
  45 4. Verification
  46
  47 i.  Check logs under container /opt/app/VESCollector/logs/collector.log
  48         for errors
  49
  50 ii. If no active feed, you can simulate an event into collector via curl
  51
  52 .. code-block:: bash
  53
  54    curl -i  -X POST -d @<sampleves> --header "Content-Type:application/json" -k http://localhost:8080/eventListener/v5
  55
  56 .. Note: If DMAAPHOST provided is invalid, you will see exception
  57    around publish on the collector.logs (collector queues and attempts
  58    to resend the event hence exceptions reported will be periodic).
  59
  60 i. Below two topic configuration are pre-set into this container.  When
  61        valid DMAAP instance ip was provided and VES events are received,
  62        the collector will post to below topics.
  63
  64     Fault -
  65      http://<dmaaphost>:3904/events/unauthenticated.SEC\_FAULT\_OUTPUT
  66
  67     Measurement
  68     -http://<dmaaphost>:3904/events/unauthenticated.SEC\_MEASUREMENT\_OUTPUT
  69
  70 VM Init
  71 ~~~~~~~
  72
  73 To address windriver server in-stability, the below **init.sh** script
  74 was used to start the container on VM restart.
  75
  76 .. code-block:: bash
  77
  78    #!/bin/sh
  79    sudo docker ps | grep “vescollector”
  80    if [ $? -ne 0 ]; then
  81           sudo docker login -u docker -p docker nexus.onap.org:10001
  82           sudo docker pull nexus.onap.org:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.1
  83           sudo docker rm -f vescollector
  84           echo “Collector process not running - $(date)” >> /home/ubuntu/startuplog
  85           sudo docker run -d –name vescollector -p 8080:8080/tcp -p 8443:8443/tcp -P -e DMAAPHOST=‘10.12.25.96’ nexus.onap.org:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.1
  86    else
  87           echo “Collector process running - $(date)” >> /home/ubuntu/startuplog
  88    fi
  89
  90
  91 This script was invoked via VM init script (rc.d).
  92
  93 .. code-block:: bash
  94
  95    ln -s /home/ubuntu/init.sh /etc/init.d/init.sh
  96    sudo  update-rc.d init.sh start 2
  97
  98
  99
 100 ThresholdCrossingAnalysis (TCA/CDAP)
 101 ------------------------------------
 102
 103 The platform deploys CDAP as cluster and instantiates TCA. For the
 104 manual setup, we will leverage the CDAP SDK docker container to deploy
 105 TCA instances.  To setup TCA, choose VM with ubuntu-16.04 image,
 106 m1.medium size and 50gb cinder volumes.
 107
 108 1. Install docker
 109
 110 .. code-block:: bash
 111
 112    sudo apt-get update
 113    sudo apt install `docker.io <http://docker.io/>`__
 114
 115 2. Pull CDAP SDK container
 116
 117 .. code-block:: bash
 118
 119    sudo docker pull caskdata/cdap-standalone:4.1.2
 120
 121 3. Deploy and run the CDAP container
 122
 123 .. code-block:: bash
 124
 125    sudo docker run -d --name cdap-sdk-2 -p 11011:11011 -p 11015:11015
 126    caskdata/cdap-standalone:4.1.2
 127
 128 4. Create Namespace on CDAP application
 129
 130 .. code-block:: bash
 131
 132    curl -X PUT http://localhost:11015/v3/namespaces/cdap_tca_hi_lo
 133
 134 5. Create TCA app config file - "tca\_app\_config.json" under ~ubuntu as below
 135
 136 .. code-block:: json
 137
 138  {
 139    "artifact": {
 140      "name": "dcae-analytics-cdap-tca",
 141      "version": "2.0.0",
 142      "scope": "user"
 143    },
 144
 145    "config": {
 146      "appName": "dcae-tca",
 147      "appDescription": "DCAE Analytics Threshold Crossing Alert Application",
 148      "tcaVESMessageStatusTableName": "TCAVESMessageStatusTable",
 149      "tcaVESMessageStatusTableTTLSeconds": 86400.0,
 150      "tcaAlertsAbatementTableName": "TCAAlertsAbatementTable",
 151      "tcaAlertsAbatementTableTTLSeconds": 1728000.0,
 152      "tcaVESAlertsTableName": "TCAVESAlertsTable",
 153      "tcaVESAlertsTableTTLSeconds": 1728000.0,
 154      "thresholdCalculatorFlowletInstances": 2.0,
 155      "tcaSubscriberOutputStreamName": "TCASubscriberOutputStream"
 156    }
 157  }
 158
 159
 160 6. Create TCA app preference file under ~ubuntu as below
 161
 162 .. code-block:: json
 163
 164  {
 165   "publisherContentType" : "application/json",
 166   "publisherHostName" : "10.12.25.96",
 167   "publisherHostPort" : "3904",
 168   "publisherMaxBatchSize" : "1",
 169   "publisherMaxRecoveryQueueSize" : "100000",
 170   "publisherPollingInterval" : "20000",
 171   "publisherProtocol" : "http",
 172   "publisherTopicName" : "unauthenticated.DCAE_CL_OUTPUT",
 173   "subscriberConsumerGroup" : "OpenDCAE-c1",
 174   "subscriberConsumerId" : "c1",
 175   "subscriberContentType" : "application/json",
 176   "subscriberHostName" : "10.12.25.96",
 177   "subscriberHostPort" : "3904",
 178   "subscriberMessageLimit" : "-1",
 179   "subscriberPollingInterval" : "20000",
 180   "subscriberProtocol" : "http",
 181   "subscriberTimeoutMS" : "-1",
 182   "subscriberTopicName" : "unauthenticated.SEC_MEASUREMENT_OUTPUT",
 183   "enableAAIEnrichment" : false,
 184   "aaiEnrichmentHost" : "10.12.25.72",
 185   "aaiEnrichmentPortNumber" : 8443,
 186   "aaiEnrichmentProtocol" : "https",
 187   "aaiEnrichmentUserName" : "DCAE",
 188   "aaiEnrichmentUserPassword" : "DCAE",
 189   "aaiEnrichmentIgnoreSSLCertificateErrors" : false,
 190   "aaiVNFEnrichmentAPIPath" : "/aai/v11/network/generic-vnfs/generic-vnf",
 191   "aaiVMEnrichmentAPIPath" :  "/aai/v11/search/nodes-query",
 192   "tca_policy" : {
 193         "domain": "measurementsForVfScaling",
 194         "metricsPerEventName": [{
 195                 "eventName": "vFirewallBroadcastPackets",
 196                 "controlLoopSchemaType": "VNF",
 197                 "policyScope": "DCAE",
 198                 "policyName": "DCAE.Config_tca-hi-lo",
 199                 "policyVersion": "v0.0.1",
 200                 "thresholds": [{
 201                         "closedLoopControlName": "ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a",
 202                         "version": "1.0.2",
 203                         "fieldPath": "$.event.measurementsForVfScalingFields.vNicUsageArray[*].receivedTotalPacketsDelta",
 204                         "thresholdValue": 300,
 205                         "direction": "LESS_OR_EQUAL",
 206                         "severity": "MAJOR",
 207                         "closedLoopEventStatus": "ONSET"
 208                 }, {
 209                         "closedLoopControlName": "ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a",
 210                         "version": "1.0.2",
 211                         "fieldPath": "$.event.measurementsForVfScalingFields.vNicUsageArray[*].receivedTotalPacketsDelta",
 212                         "thresholdValue": 700,
 213                         "direction": "GREATER_OR_EQUAL",
 214                         "severity": "CRITICAL",
 215                         "closedLoopEventStatus": "ONSET"
 216                 }]
 217         }, {
 218                 "eventName": "vLoadBalancer",
 219                 "controlLoopSchemaType": "VM",
 220                 "policyScope": "DCAE",
 221                 "policyName": "DCAE.Config_tca-hi-lo",
 222                 "policyVersion": "v0.0.1",
 223                 "thresholds": [{
 224                         "closedLoopControlName": "ControlLoop-vDNS-6f37f56d-a87d-4b85-b6a9-cc953cf779b3",
 225                         "version": "1.0.2",
 226                         "fieldPath": "$.event.measurementsForVfScalingFields.vNicUsageArray[*].receivedTotalPacketsDelta",
 227                         "thresholdValue": 300,
 228                         "direction": "GREATER_OR_EQUAL",
 229                         "severity": "CRITICAL",
 230                         "closedLoopEventStatus": "ONSET"
 231                 }]
 232         }, {
 233                 "eventName": "Measurement_vGMUX",
 234                 "controlLoopSchemaType": "VNF",
 235                 "policyScope": "DCAE",
 236                 "policyName": "DCAE.Config_tca-hi-lo",
 237                 "policyVersion": "v0.0.1",
 238                 "thresholds": [{
 239                         "closedLoopControlName": "ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e",
 240                         "version": "1.0.2",
 241                         "fieldPath": "$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].value",
 242                         "thresholdValue": 0,
 243                         "direction": "EQUAL",
 244                         "severity": "MAJOR",
 245                         "closedLoopEventStatus": "ABATED"
 246                 }, {
 247                         "closedLoopControlName": "ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e",
 248                         "version": "1.0.2",
 249                         "fieldPath": "$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].value",
 250                         "thresholdValue": 0,
 251                         "direction": "GREATER",
 252                         "severity": "CRITICAL",
 253                         "closedLoopEventStatus": "ONSET"
 254                 }]
 255         }]
 256   }
 257  }
 258
 259
 260 .. Note: Dmaap configuration are specified on this file on
 261    publisherHostName and subscriberHostName. To be changed as
 262    required\*\*
 263
 264 7. Copy below script to CDAP server (this gets latest image from nexus and deploys TCA application) and execute it
 265
 266 .. code-block:: bash
 267
 268  #!/bin/sh
 269  TCA_JAR=dcae-analytics-cdap-tca-2.0.0.jar
 270  rm -f /home/ubuntu/$TCA_JAR
 271  cd /home/ubuntu/
 272  wget https://nexus.onap.org/service/local/repositories/staging/content/org/onap/dcaegen2/analytics/tca/dcae-analytics-cdap-tca/2.0.0/$TCA_JAR
 273  if [ $? -eq 0 ]; then
 274         if [ -f /home/ubuntu/$TCA_JAR ]; then
 275                 echo “Restarting TCA CDAP application using $TCA_JAR artifact”
 276         else
 277                 echo “ERROR: $TCA_JAR missing”
 278                 exit 1
 279         fi
 280  else
 281         echo “ERROR: $TCA_JAR not found in nexus”
 282         exit 1
 283  fi
 284  # stop programs
 285  curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/workers/TCADMaaPMRPublisherWorker/stop
 286  curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/workers/TCADMaaPMRSubscriberWorker/stop
 287  curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/flows/TCAVESCollectorFlow/stop
 288  # delete application
 289  curl -X DELETE http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca
 290  # delete artifact
 291  curl -X DELETE http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/artifacts/dcae-analytics-cdap-tca/versions/2.0.0
 292  # load artifact
 293  curl -X POST –data-binary @/home/ubuntu/$TCA_JAR http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/artifacts/dcae-analytics-cdap-tca
 294  # create app
 295  curl -X PUT -d @/home/ubuntu/tca_app_config.json http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca
 296  # load preferences
 297  curl -X PUT -d @/home/ubuntu/tca_app_preferences.json http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/preferences
 298  # start programs
 299  curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/workers/TCADMaaPMRPublisherWorker/start
 300  curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/workers/TCADMaaPMRSubscriberWorker/start
 301  curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/flows/TCAVESCollectorFlow/start
 302  echo
 303  # get status of programs
 304  curl http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/workers/TCADMaaPMRPublisherWorker/status
 305  curl http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/workers/TCADMaaPMRSubscriberWorker/status
 306  curl http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/flows/TCAVESCollectorFlow/status
 307  echo
 308
 309
 310 8. Verify TCA application and logs via CDAP GUI processes
 311
 312     The overall flow can be checked here
 313
 314 TCA Configuration Change
 315 ~~~~~~~~~~~~~~~~~~~~~~~~
 316
 317 Typical configuration changes include changing DMAAP host and/or Policy configuration. If necessary, modify the file on step #6 and run the script noted as step #7 to redeploy TCA with updated configuration.
 318
 319 VM Init
 320 ~~~~~~~
 321
 322 To address windriver server in-stability, the below **init.sh** script
 323 was used to restart the container on VM restart.  This script was
 324 invoked via VM init script (rc.d).
 325
 326 .. code-block:: bash
 327
 328    #!/bin/sh
 329    #docker run -d –name cdap-sdk -p 11011:11011 -p 11015:11015 caskdata/cdap-standalone:4.1.2
 330    sudo docker restart cdap-sdk-2
 331    sleep 30
 332    # start program
 333    curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/workers/TCADMaaPMRPublisherWorker/start
 334    curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/workers/TCADMaaPMRSubscriberWorker/start
 335    curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/flows/TCAVESCollectorFlow/start
 336
 337
 338 This script was invoked via VM init script (rc.d).
 339
 340 .. code-block:: bash
 341
 342    ln -s /home/ubuntu/init.sh /etc/init.d/init.sh
 343    sudo  update-rc.d init.sh start 2