4 The below steps covers manual setup of DCAE VM’s and DCAE service
12 DCAE VES Collector can be configured on VM with ubuntu-16.04 image
13 (m1.small should suffice if this is only service) and 20Gb cinder
20 sudo apt install `docker.io <http://docker.io/>`__
22 1. Pull the latest container from onap nexus
24 sudo docker login -u docker -p docker
25 `nexus.onap.org <http://nexus.onap.org/>`__:10001
28 `nexus.onap.org <http://nexus.onap.org/>`__:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.1
30 1. Start the VESCollector with below command
32 sudo docker run -d --name vescollector -p 8080:8080/tcp -p
33 8443:8443/tcp -P -e DMAAPHOST='<dmaap IP>'
34 `nexus.onap.org <http://nexus.onap.org/>`__:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.1
36 Note: Change the dmaaphost to required DMAAP ip. To change the
37 dmaap information for a running container, stop the active
38 container and rerun above command changing the dmaap IP.
42 i. Check logs under container /opt/app/VESCollector/logs/collector.log
45 ii. If no active feed, you can simulate an event into collector via curl
47 curl -i -X POST -d @<sampleves> --header "Content-Type:
48 application/json" http://localhost:8080/eventListener/v5 -k
50 Note: If DMAAPHOST provided is invalid, you will see exception
51 around publish on the collector.logs (collector queues and attempts
52 to resend the event hence exceptions reported will be periodic).
54 i. Below two topic configuration are pre-set into this container. When
55 valid DMAAP instance ip was provided and VES events are received,
56 the collector will post to below topics.
59 http://<dmaaphost>:3904/events/unauthenticated.SEC\_FAULT\_OUTPUT
62 -http://<dmaaphost>:3904/events/unauthenticated.SEC\_MEASUREMENT\_OUTPUT
67 To address windriver server in-stability, the below **init.sh** script
68 was used to start the container on VM restart.
70 +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
73 | sudo docker ps \| grep "vescollector" |
75 | if [ $? -ne 0 ]; then |
77 | sudo docker login -u docker -p docker nexus.onap.org:10001 |
79 | sudo docker pull nexus.onap.org:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.1 |
81 | sudo docker rm -f vescollector |
83 | echo "Collector process not running - $(date)" >> /home/ubuntu/startuplog |
85 | sudo docker run -d --name vescollector -p 8080:8080/tcp -p 8443:8443/tcp -P -e DMAAPHOST='10.12.25.96' nexus.onap.org:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.1 |
89 | echo "Collector process running - $(date)" >> /home/ubuntu/startuplog |
92 +==============================================================================================================================================================================================+
93 +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
95 This script was invoked via VM init script (rc.d).
97 ln -s /home/ubuntu/init.sh /etc/init.d/init.sh
99 sudo update-rc.d init.sh start 2
103 ThresholdCrossingAnalysis (TCA/CDAP)
104 ------------------------------------
106 The platform deploys CDAP as cluster and instantiates TCA. For the
107 manual setup, we will leverage the CDAP SDK docker container to deploy
108 TCA instances. To setup TCA, choose VM with ubuntu-16.04 image,
109 m1.medium size and 50gb cinder volumes.
115 sudo apt install `docker.io <http://docker.io/>`__
117 1. Pull CDAP SDK container
119 sudo docker pull caskdata/cdap-standalone:4.1.2
121 1. Deploy and run the CDAP container
123 sudo docker run -d --name cdap-sdk-2 -p 11011:11011 -p 11015:11015
124 caskdata/cdap-standalone:4.1.2
126 1. Create Namespace on CDAP application
128 curl -X PUT http://localhost:11015/v3/namespaces/cdap_tca_hi_lo
130 1. Create TCA app preference file under ~ubuntu as below
132 +--------------------------------------------------------------------------------------------------------------------------------------------+
135 | "publisherContentType" : "application/json", |
137 | "publisherHostName" : "10.12.25.96", |
139 | "publisherHostPort" : "3904", |
141 | "publisherMaxBatchSize" : "1", |
143 | "publisherMaxRecoveryQueueSize" : "100000", |
145 | "publisherPollingInterval" : "20000", |
147 | "publisherProtocol" : "http", |
149 | "publisherTopicName" : "unauthenticated.DCAE\_CL\_OUTPUT", |
151 | "subscriberConsumerGroup" : "OpenDCAE-c1", |
153 | "subscriberConsumerId" : "c1", |
155 | "subscriberContentType" : "application/json", |
157 | "subscriberHostName" : "10.12.25.96", |
159 | "subscriberHostPort" : "3904", |
161 | "subscriberMessageLimit" : "-1", |
163 | "subscriberPollingInterval" : "20000", |
165 | "subscriberProtocol" : "http", |
167 | "subscriberTimeoutMS" : "-1", |
169 | "subscriberTopicName" : "unauthenticated.SEC\_MEASUREMENT\_OUTPUT", |
171 | "enableAAIEnrichment" : false, |
173 | "aaiEnrichmentHost" : "10.12.25.72", |
175 | "aaiEnrichmentPortNumber" : 8443, |
177 | "aaiEnrichmentProtocol" : "https", |
179 | "aaiEnrichmentUserName" : "DCAE", |
181 | "aaiEnrichmentUserPassword" : "DCAE", |
183 | "aaiEnrichmentIgnoreSSLCertificateErrors" : false, |
185 | "aaiVNFEnrichmentAPIPath" : "/aai/v11/network/generic-vnfs/generic-vnf", |
187 | "aaiVMEnrichmentAPIPath" : "/aai/v11/search/nodes-query", |
189 | "tca\_policy" : "{ |
191 | \\"domain\\": \\"measurementsForVfScaling\\", |
193 | \\"metricsPerEventName\\": [{ |
195 | \\"eventName\\": \\"vFirewallBroadcastPackets\\", |
197 | \\"controlLoopSchemaType\\": \\"VNF\\", |
199 | \\"policyScope\\": \\"DCAE\\", |
201 | \\"policyName\\": \\"DCAE.Config\_tca-hi-lo\\", |
203 | \\"policyVersion\\": \\"v0.0.1\\", |
205 | \\"thresholds\\": [{ |
207 | \\"closedLoopControlName\\": \\"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a\\", |
209 | \\"version\\": \\"1.0.2\\", |
211 | \\"fieldPath\\": \\"$.event.measurementsForVfScalingFields.vNicUsageArray[\*].receivedTotalPacketsDelta\\", |
213 | \\"thresholdValue\\": 300, |
215 | \\"direction\\": \\"LESS\_OR\_EQUAL\\", |
217 | \\"severity\\": \\"MAJOR\\", |
219 | \\"closedLoopEventStatus\\": \\"ONSET\\" |
223 | \\"closedLoopControlName\\": \\"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a\\", |
225 | \\"version\\": \\"1.0.2\\", |
227 | \\"fieldPath\\": \\"$.event.measurementsForVfScalingFields.vNicUsageArray[\*].receivedTotalPacketsDelta\\", |
229 | \\"thresholdValue\\": 700, |
231 | \\"direction\\": \\"GREATER\_OR\_EQUAL\\", |
233 | \\"severity\\": \\"CRITICAL\\", |
235 | \\"closedLoopEventStatus\\": \\"ONSET\\" |
241 | \\"eventName\\": \\"vLoadBalancer\\", |
243 | \\"controlLoopSchemaType\\": \\"VM\\", |
245 | \\"policyScope\\": \\"DCAE\\", |
247 | \\"policyName\\": \\"DCAE.Config\_tca-hi-lo\\", |
249 | \\"policyVersion\\": \\"v0.0.1\\", |
251 | \\"thresholds\\": [{ |
253 | \\"closedLoopControlName\\": \\"ControlLoop-vDNS-6f37f56d-a87d-4b85-b6a9-cc953cf779b3\\", |
255 | \\"version\\": \\"1.0.2\\", |
257 | \\"fieldPath\\": \\"$.event.measurementsForVfScalingFields.vNicUsageArray[\*].receivedTotalPacketsDelta\\", |
259 | \\"thresholdValue\\": 300, |
261 | \\"direction\\": \\"GREATER\_OR\_EQUAL\\", |
263 | \\"severity\\": \\"CRITICAL\\", |
265 | \\"closedLoopEventStatus\\": \\"ONSET\\" |
271 | \\"eventName\\": \\"Measurement\_vGMUX\\", |
273 | \\"controlLoopSchemaType\\": \\"VNF\\", |
275 | \\"policyScope\\": \\"DCAE\\", |
277 | \\"policyName\\": \\"DCAE.Config\_tca-hi-lo\\", |
279 | \\"policyVersion\\": \\"v0.0.1\\", |
281 | \\"thresholds\\": [{ |
283 | \\"closedLoopControlName\\": \\"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e\\", |
285 | \\"version\\": \\"1.0.2\\", |
287 | \\"fieldPath\\": \\"$.event.measurementsForVfScalingFields.additionalMeasurements[\*].arrayOfFields[0].value\\", |
289 | \\"thresholdValue\\": 0, |
291 | \\"direction\\": \\"EQUAL\\", |
293 | \\"severity\\": \\"MAJOR\\", |
295 | \\"closedLoopEventStatus\\": \\"ABATED\\" |
299 | \\"closedLoopControlName\\": \\"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e\\", |
301 | \\"version\\": \\"1.0.2\\", |
303 | \\"fieldPath\\": \\"$.event.measurementsForVfScalingFields.additionalMeasurements[\*].arrayOfFields[0].value\\", |
305 | \\"thresholdValue\\": 0, |
307 | \\"direction\\": \\"GREATER\\", |
309 | \\"severity\\": \\"CRITICAL\\", |
311 | \\"closedLoopEventStatus\\": \\"ONSET\\" |
320 +============================================================================================================================================+
321 +--------------------------------------------------------------------------------------------------------------------------------------------+
323 Note: Dmaap configuration are specified on this file on
324 publisherHostName and subscriberHostName. To be changed as
327 1. Copy below script to CDAP server (this gets latest image from nexus
328 and deploys TCA application) and execute it
330 +---------------------------------------------------------------------------------------------------------------------------------------------------------------+
333 | TCA\_JAR=dcae-analytics-cdap-tca-2.0.0.jar |
335 | rm -f /home/ubuntu/$TCA\_JAR |
339 | wget https://nexus.onap.org/service/local/repositories/staging/content/org/onap/dcaegen2/analytics/tca/dcae-analytics-cdap-tca/2.0.0/$TCA\_JAR |
341 | if [ $? -eq 0 ]; then |
343 | if [ -f /home/ubuntu/$TCA\_JAR ]; then |
345 | echo "Restarting TCA CDAP application using $TCA\_JAR artifact" |
349 | echo "ERROR: $TCA\_JAR missing" |
357 | echo "ERROR: $TCA\_JAR not found in nexus" |
365 | curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/workers/TCADMaaPMRPublisherWorker/stop |
367 | curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/workers/TCADMaaPMRSubscriberWorker/stop |
369 | curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/flows/TCAVESCollectorFlow/stop |
371 | # delete application |
373 | curl -X DELETE http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca# delete artifact |
375 | curl -X DELETE http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/artifacts/dcae-analytics-cdap-tca/versions/2.0.0# load artifact |
377 | curl -X POST --data-binary @/home/ubuntu/$TCA\_JAR http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/artifacts/dcae-analytics-cdap-tcaecho# create app |
379 | curl -X PUT -d @/home/ubuntu/tca\_app\_config.json http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca# load preferences |
381 | curl -X PUT -d @/home/ubuntu/tca\_app\_preferences.json http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/preferences# start programs |
383 | curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/workers/TCADMaaPMRPublisherWorker/start |
385 | curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/workers/TCADMaaPMRSubscriberWorker/start |
387 | curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/flows/TCAVESCollectorFlow/startecho# get status of programs |
389 | curl http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/workers/TCADMaaPMRPublisherWorker/status |
391 | curl http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/workers/TCADMaaPMRSubscriberWorker/status |
393 | curl http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/flows/TCAVESCollectorFlow/statusecho |
394 +===============================================================================================================================================================+
395 +---------------------------------------------------------------------------------------------------------------------------------------------------------------+
397 1. Verify TCA application and logs via CDAP GUI processes
399 The overall flow can be checked here
401 TCA Configuration Change
402 ~~~~~~~~~~~~~~~~~~~~~~~
404 Typical configuration changes include changing DMAAP host and/or Policy
405 configuration. If necessary, modify the file on step #5 and run the
406 script noted as step #6 to redeploy TCA with updated configuration.
411 To address windriver server in-stability, the below **init.sh** script
412 was used to restart the container on VM restart. This script was
413 invoked via VM init script (rc.d).
415 +------------------------------------------------------------------------------------------------------------------------------+
418 | #docker run -d --name cdap-sdk -p 11011:11011 -p 11015:11015 caskdata/cdap-standalone:4.1.2 |
420 | sudo docker restart cdap-sdk-2 |
426 | curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/workers/TCADMaaPMRPublisherWorker/start |
428 | curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/workers/TCADMaaPMRSubscriberWorker/start |
430 | curl -X POST http://localhost:11015/v3/namespaces/cdap\_tca\_hi\_lo/apps/dcae-tca/flows/TCAVESCollectorFlow/start |
431 +==============================================================================================================================+
432 +------------------------------------------------------------------------------------------------------------------------------+
436 This script was invoked via VM init script (rc.d).
438 ln -s /home/ubuntu/init.sh /etc/init.d/init.sh
440 sudo update-rc.d init.sh start 2