4 The below steps covers manual setup of DCAE VM’s and DCAE service
12 DCAE VES Collector can be configured on VM with ubuntu-16.04 image
13 (m1.small should suffice if this is only service) and 20Gb cinder
20 sudo apt install `docker.io <http://docker.io/>`__
22 2. Pull the latest container from onap nexus
24 sudo docker login -u docker -p docker
25 `nexus.onap.org <http://nexus.onap.org/>`__:10001
28 `nexus.onap.org <http://nexus.onap.org/>`__:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.1
30 3. Start the VESCollector with below command
32 sudo docker run -d --name vescollector -p 8080:8080/tcp -p
33 8443:8443/tcp -P -e DMAAPHOST='<dmaap IP>'
34 `nexus.onap.org <http://nexus.onap.org/>`__:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.1
36 Note: Change the dmaaphost to required DMAAP ip. To change the
37 dmaap information for a running container, stop the active
38 container and rerun above command changing the dmaap IP.
42 i. Check logs under container /opt/app/VESCollector/logs/collector.log
45 ii. If no active feed, you can simulate an event into collector via curl
47 curl -i -X POST -d @<sampleves> --header "Content-Type:
48 application/json" http://localhost:8080/eventListener/v5 -k
50 Note: If DMAAPHOST provided is invalid, you will see exception
51 around publish on the collector.logs (collector queues and attempts
52 to resend the event hence exceptions reported will be periodic).
54 i. Below two topic configuration are pre-set into this container. When
55 valid DMAAP instance ip was provided and VES events are received,
56 the collector will post to below topics.
59 http://<dmaaphost>:3904/events/unauthenticated.SEC\_FAULT\_OUTPUT
62 -http://<dmaaphost>:3904/events/unauthenticated.SEC\_MEASUREMENT\_OUTPUT
67 To address windriver server in-stability, the below **init.sh** script
68 was used to start the container on VM restart.
70 +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
73 | sudo docker ps \| grep "vescollector" |
75 | if [ $? -ne 0 ]; then |
77 | sudo docker login -u docker -p docker nexus.onap.org:10001 |
79 | sudo docker pull nexus.onap.org:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.1 |
81 | sudo docker rm -f vescollector |
83 | echo "Collector process not running - $(date)" >> /home/ubuntu/startuplog |
85 | sudo docker run -d --name vescollector -p 8080:8080/tcp -p 8443:8443/tcp -P -e DMAAPHOST='10.12.25.96' nexus.onap.org:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.1 |
89 | echo "Collector process running - $(date)" >> /home/ubuntu/startuplog |
92 +==============================================================================================================================================================================================+
93 +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
95 This script was invoked via VM init script (rc.d).
97 ln -s /home/ubuntu/init.sh /etc/init.d/init.sh
99 sudo update-rc.d init.sh start 2
103 ThresholdCrossingAnalysis (TCA/CDAP)
104 ------------------------------------
106 The platform deploys CDAP as cluster and instantiates TCA. For the
107 manual setup, we will leverage the CDAP SDK docker container to deploy
108 TCA instances. To setup TCA, choose VM with ubuntu-16.04 image,
109 m1.medium size and 50gb cinder volumes.
115 sudo apt install `docker.io <http://docker.io/>`__
117 2. Pull CDAP SDK container
119 sudo docker pull caskdata/cdap-standalone:4.1.2
121 3. Deploy and run the CDAP container
123 sudo docker run -d --name cdap-sdk-2 -p 11011:11011 -p 11015:11015
124 caskdata/cdap-standalone:4.1.2
126 4. Create Namespace on CDAP application
128 curl -X PUT http://localhost:11015/v3/namespaces/cdap_tca_hi_lo
130 5. Create TCA app config file - "tca\_app\_config.json" under ~ubuntu as
133 +------------------------------------------------------------------------------+
138 | "name": "dcae-analytics-cdap-tca", |
140 | "version": "2.0.0", |
148 | "appName": "dcae-tca", |
150 | "appDescription": "DCAE Analytics Threshold Crossing Alert Application", |
152 | "tcaVESMessageStatusTableName": "TCAVESMessageStatusTable", |
154 | "tcaVESMessageStatusTableTTLSeconds": 86400.0, |
156 | "tcaAlertsAbatementTableName": "TCAAlertsAbatementTable", |
158 | "tcaAlertsAbatementTableTTLSeconds": 1728000.0, |
160 | "tcaVESAlertsTableName": "TCAVESAlertsTable", |
162 | "tcaVESAlertsTableTTLSeconds": 1728000.0, |
164 | "thresholdCalculatorFlowletInstances": 2.0, |
166 | "tcaSubscriberOutputStreamName": "TCASubscriberOutputStream" |
171 +==============================================================================+
172 +------------------------------------------------------------------------------+
174 6. Create TCA app preference file under ~ubuntu as below
176 +--------------------------------------------------------------------------------------------------------------------------------------------+
179 | "publisherContentType" : "application/json", |
181 | "publisherHostName" : "10.12.25.96", |
183 | "publisherHostPort" : "3904", |
185 | "publisherMaxBatchSize" : "1", |
187 | "publisherMaxRecoveryQueueSize" : "100000", |
189 | "publisherPollingInterval" : "20000", |
191 | "publisherProtocol" : "http", |
193 | "publisherTopicName" : "unauthenticated.DCAE\_CL\_OUTPUT", |
195 | "subscriberConsumerGroup" : "OpenDCAE-c1", |
197 | "subscriberConsumerId" : "c1", |
199 | "subscriberContentType" : "application/json", |
201 | "subscriberHostName" : "10.12.25.96", |
203 | "subscriberHostPort" : "3904", |
205 | "subscriberMessageLimit" : "-1", |
207 | "subscriberPollingInterval" : "20000", |
209 | "subscriberProtocol" : "http", |
211 | "subscriberTimeoutMS" : "-1", |
213 | "subscriberTopicName" : "unauthenticated.SEC\_MEASUREMENT\_OUTPUT", |
215 | "enableAAIEnrichment" : false, |
217 | "aaiEnrichmentHost" : "10.12.25.72", |
219 | "aaiEnrichmentPortNumber" : 8443, |
221 | "aaiEnrichmentProtocol" : "https", |
223 | "aaiEnrichmentUserName" : "DCAE", |
225 | "aaiEnrichmentUserPassword" : "DCAE", |
227 | "aaiEnrichmentIgnoreSSLCertificateErrors" : false, |
229 | "aaiVNFEnrichmentAPIPath" : "/aai/v11/network/generic-vnfs/generic-vnf", |
231 | "aaiVMEnrichmentAPIPath" : "/aai/v11/search/nodes-query", |
233 | "tca\_policy" : "{ |
235 | \\"domain\\": \\"measurementsForVfScaling\\", |
237 | \\"metricsPerEventName\\": [{ |
239 | \\"eventName\\": \\"vFirewallBroadcastPackets\\", |
241 | \\"controlLoopSchemaType\\": \\"VNF\\", |
243 | \\"policyScope\\": \\"DCAE\\", |
245 | \\"policyName\\": \\"DCAE.Config\_tca-hi-lo\\", |
247 | \\"policyVersion\\": \\"v0.0.1\\", |
249 | \\"thresholds\\": [{ |
251 | \\"closedLoopControlName\\": \\"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a\\", |
253 | \\"version\\": \\"1.0.2\\", |
255 | \\"fieldPath\\": \\"$.event.measurementsForVfScalingFields.vNicUsageArray[\*].receivedTotalPacketsDelta\\", |
257 | \\"thresholdValue\\": 300, |
259 | \\"direction\\": \\"LESS\_OR\_EQUAL\\", |
261 | \\"severity\\": \\"MAJOR\\", |
263 | \\"closedLoopEventStatus\\": \\"ONSET\\" |
267 | \\"closedLoopControlName\\": \\"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a\\", |
269 | \\"version\\": \\"1.0.2\\", |
271 | \\"fieldPath\\": \\"$.event.measurementsForVfScalingFields.vNicUsageArray[\*].receivedTotalPacketsDelta\\", |
273 | \\"thresholdValue\\": 700, |
275 | \\"direction\\": \\"GREATER\_OR\_EQUAL\\", |
277 | \\"severity\\": \\"CRITICAL\\", |
279 | \\"closedLoopEventStatus\\": \\"ONSET\\" |
285 | \\"eventName\\": \\"vLoadBalancer\\", |
287 | \\"controlLoopSchemaType\\": \\"VM\\", |
289 | \\"policyScope\\": \\"DCAE\\", |
291 | \\"policyName\\": \\"DCAE.Config\_tca-hi-lo\\", |
293 | \\"policyVersion\\": \\"v0.0.1\\", |
295 | \\"thresholds\\": [{ |
297 | \\"closedLoopControlName\\": \\"ControlLoop-vDNS-6f37f56d-a87d-4b85-b6a9-cc953cf779b3\\", |
299 | \\"version\\": \\"1.0.2\\", |
301 | \\"fieldPath\\": \\"$.event.measurementsForVfScalingFields.vNicUsageArray[\*].receivedTotalPacketsDelta\\", |
303 | \\"thresholdValue\\": 300, |
305 | \\"direction\\": \\"GREATER\_OR\_EQUAL\\", |
307 | \\"severity\\": \\"CRITICAL\\", |
309 | \\"closedLoopEventStatus\\": \\"ONSET\\" |
315 | \\"eventName\\": \\"Measurement\_vGMUX\\", |
317 | \\"controlLoopSchemaType\\": \\"VNF\\", |
319 | \\"policyScope\\": \\"DCAE\\", |
321 | \\"policyName\\": \\"DCAE.Config\_tca-hi-lo\\", |
323 | \\"policyVersion\\": \\"v0.0.1\\", |
325 | \\"thresholds\\": [{ |
327 | \\"closedLoopControlName\\": \\"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e\\", |
329 | \\"version\\": \\"1.0.2\\", |
331 | \\"fieldPath\\": \\"$.event.measurementsForVfScalingFields.additionalMeasurements[\*].arrayOfFields[0].value\\", |
333 | \\"thresholdValue\\": 0, |
335 | \\"direction\\": \\"EQUAL\\", |
337 | \\"severity\\": \\"MAJOR\\", |
339 | \\"closedLoopEventStatus\\": \\"ABATED\\" |
343 | \\"closedLoopControlName\\": \\"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e\\", |
345 | \\"version\\": \\"1.0.2\\", |
347 | \\"fieldPath\\": \\"$.event.measurementsForVfScalingFields.additionalMeasurements[\*].arrayOfFields[0].value\\", |
349 | \\"thresholdValue\\": 0, |
351 | \\"direction\\": \\"GREATER\\", |
353 | \\"severity\\": \\"CRITICAL\\", |
355 | \\"closedLoopEventStatus\\": \\"ONSET\\" |
364 +============================================================================================================================================+
365 +--------------------------------------------------------------------------------------------------------------------------------------------+
367 Note: Dmaap configuration are specified on this file on
368 publisherHostName and subscriberHostName. To be changed as
371 7. Copy below script to CDAP server (this gets latest image from nexus
372 and deploys TCA application) and execute it
374 +--------------------------------------------------------------------------------------------------------------------------------------------------+
377 | TCA\_JAR=dcae-analytics-cdap-tca-2.0.0.jar |
379 | rm -f /home/ubuntu/$TCA\_JAR |
383 | wget https://nexus.onap.org/service/local/repositories/staging/content/org/onap/dcaegen2/analytics/tca/dcae-analytics-cdap-tca/2.0.0/$TCA\_JAR |
385 | if [ $? -eq 0 ]; then |
387 | if [ -f /home/ubuntu/$TCA\_JAR ]; then |
389 | echo "Restarting TCA CDAP application using $TCA\_JAR artifact" |
393 | echo "ERROR: $TCA\_JAR missing" |
401 | echo "ERROR: $TCA\_JAR not found in nexus" |
409 | curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/workers/TCADMaaPMRPublisherWorker/stop |
411 | curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/workers/TCADMaaPMRSubscriberWorker/stop |
413 | curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/flows/TCAVESCollectorFlow/stop |
415 | # delete application |
417 | curl -X DELETE http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca |
419 | # delete artifact |
421 | curl -X DELETE http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/artifacts/dcae-analytics-cdap-tca/versions/2.0.0 |
425 | curl -X POST --data-binary @/home/ubuntu/$TCA\_JAR http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/artifacts/dcae-analytics-cdap-tca |
429 | curl -X PUT -d @/home/ubuntu/tca\_app\_config.json http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca |
431 | # load preferences |
433 | curl -X PUT -d @/home/ubuntu/tca\_app\_preferences.json http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/preferences |
437 | curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/workers/TCADMaaPMRPublisherWorker/start |
439 | curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/workers/TCADMaaPMRSubscriberWorker/start |
441 | curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/flows/TCAVESCollectorFlow/start |
445 | # get status of programs |
447 | curl http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/workers/TCADMaaPMRPublisherWorker/status |
449 | curl http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/workers/TCADMaaPMRSubscriberWorker/status |
451 | curl http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/flows/TCAVESCollectorFlow/status |
454 +==================================================================================================================================================+
455 +--------------------------------------------------------------------------------------------------------------------------------------------------+
457 8. Verify TCA application and logs via CDAP GUI processes
459 The overall flow can be checked here
461 TCA Configuration Change
462 ~~~~~~~~~~~~~~~~~~~~~~~
464 Typical configuration changes include changing DMAAP host and/or Policy configuration. If necessary, modify the file on step #6 and run the script noted as step #7 to redeploy TCA with updated configuration.
465 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
470 To address windriver server in-stability, the below **init.sh** script
471 was used to restart the container on VM restart. This script was
472 invoked via VM init script (rc.d).
474 +------------------------------------------------------------------------------------------------------------------------------+
477 | #docker run -d --name cdap-sdk -p 11011:11011 -p 11015:11015 caskdata/cdap-standalone:4.1.2 |
479 | sudo docker restart cdap-sdk-2 |
485 | curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/workers/TCADMaaPMRPublisherWorker/start |
487 | curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/workers/TCADMaaPMRSubscriberWorker/start |
489 | curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/flows/TCAVESCollectorFlow/start |
490 +==============================================================================================================================+
491 +------------------------------------------------------------------------------------------------------------------------------+
495 This script was invoked via VM init script (rc.d).
497 ln -s /home/ubuntu/init.sh /etc/init.d/init.sh
499 sudo update-rc.d init.sh start 2