1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
2 .. http://creativecommons.org/licenses/by/4.0
6 Policy-controlled Video Streaming (pcvs) with APEX
7 **************************************************
15 .. container:: sectionbody
17 .. container:: paragraph
19 This module contains several demos for
20 Policy-controlled Video Streaming (PCVS). Each demo
21 defines a policy using AVRO and Javascript (or other
22 scripting languages for the policy logic). To run the
23 demo, a vanilla Ubuntu server with some extra software packages is required:
27 - Mininet as network simulator
29 - Floodlight as SDN controller
31 - Kafka as messaging system
33 - Zookeeper for Kafka configuration
35 - APEX for policy control
37 Install Ubuntu Server and SW
38 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
42 .. rubric:: Install Demo
45 .. container:: sectionbody
47 .. container:: paragraph
53 - Ubuntu server: 1.4 GB
55 - Ubuntu with Xubuntu Desktop, git, Firefox: 2.3 GB
57 - Ubuntu with all, system updated: 3 GB
59 - With ZK, Kafka, VLC, Mininet, Floodlight, Python:
62 - APEX Build (M2 and built): M2 ~ 2 GB, APEX ~3.5 GB
64 - APEX install (not build locally): ~ 300 MB
66 .. container:: paragraph
68 On a Ubuntu OS (install a stable or LTS server first)
70 .. container:: listingblock
72 .. container:: content
76 # pre for Ubuntu, tools and X
77 sudo apt-get -y install --no-install-recommends software-properties-common
78 sudo apt-get -y install --no-install-recommends build-essential
79 sudo apt-get -y install --no-install-recommends git
80 sudo aptitude -y install --no-install-recommends xubuntu-desktop
81 sudo apt-get -y install --no-install-recommends firefox
85 sudo add-apt-repository ppa:webupd8team/java
87 sudo apt-get -y install --no-install-recommends oracle-java8-installer
91 # reboot system, run system update, then continue
93 # if VBox additions are needed, install and reboot
94 sudo (cd /usr/local/share; wget https://www.virtualbox.org/download/testcase/VBoxGuestAdditions_5.2.7-120528.iso)
95 sudo mount /usr/local/share/VBoxGuestAdditions_5.2.7-120528.iso /media/cdrom
96 sudo (cd /media/cdrom;VBoxLinuxAdditions.run)
102 # if APEX is build from source, install maven and rpm
103 sudo apt-get install maven rpm
106 sudo apt-get install zookeeperd
109 (cd /tmp;wget http://ftp.heanet.ie/mirrors/www.apache.org/dist/kafka/1.0.0/kafka_2.12-1.0.0.tgz --show-progress)
110 sudo mkdir /opt/Kafka
111 sudo tar -xvf /tmp/kafka_2.12-1.0.0.tgz -C /opt/Kafka/
115 sudo git clone https://github.com/mininet/mininet.git
116 (cd mininet;util/install.sh -a)
118 # install floodlight, requires ant
119 sudo apt-get install ant
121 sudo wget --no-check-certificate https://github.com/floodlight/floodlight/archive/master.zip
122 sudo unzip master.zip
125 sudo mkdir /var/lib/floodlight
126 sudo chmod 777 /var/lib/floodlight
129 sudo apt-get install python-pip
131 # install kafka-python (need newer version from github)
133 sudo git clone https://github.com/dpkp/kafka-python
134 sudo pip install ./kafka-python
137 sudo apt-get install vlc
139 .. container:: paragraph
141 Install APEX either from source or from a distribution
142 package. See the APEX documentation for details. We
143 assume that APEX is installed in
144 ``/opt/ericsson/apex/apex``
146 .. container:: paragraph
148 Copy the LinkMonitor file to Kafka-Python
150 .. container:: listingblock
152 .. container:: content
156 sudo cp /opt/ericsson/apex/apex/examples/scripts/pcvs/vpnsla/LinkMonitor.py /usr/local/src/kafka-python
158 .. container:: paragraph
160 Change the Logback configuration in APEX to logic
163 .. container:: listingblock
165 .. container:: content
169 (cd /opt/ericsson/apex/apex/etc; sudo cp logback-logic.xml logback.xml)
173 .. rubric:: Get the Demo Video
174 :name: get_the_demo_video
176 .. container:: sectionbody
180 - For all download options of the movie please visit
181 http://bbb3d.renderfarming.net/download.html
183 - For lower-res downloads and mirrors see
184 https://peach.blender.org/download
186 .. container:: listingblock
188 .. container:: content
192 sudo mkdir /usr/local/src/videos
194 .. container:: paragraph
196 Standard 720p (recommended)
198 .. container:: listingblock
200 .. container:: content
204 (cd /usr/local/src/videos; sudo curl -o big_buck_bunny_480p_surround.avi http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_480p_surround-fix.avi)
206 .. container:: paragraph
210 .. container:: listingblock
212 .. container:: content
216 (cd videos; sudo curl -o bbb_sunflower_1080p_60fps_normal.mp4 http://distribution.bbb3d.renderfarming.net/video/mp4/bbb_sunflower_1080p_60fps_normal.mp4)
225 .. container:: sectionbody
227 .. container:: paragraph
229 This demo uses a network with several central office
230 and core switches, over which two VPNs are run. A
231 customer ``A`` has two location ``A1`` and ``A2`` and
232 a VPN between them. A customer ``B`` has two location
233 ``B1`` and ``B2`` and a VPN between them.
235 .. container:: imageblock
237 .. container:: content
239 |VPN SLA Architecture|
241 .. container:: paragraph
243 The architecture above shows the scenario. The
244 components are realized in this demo as follows:
248 - *CEP / Analytics* - a simple Python script taking
249 events from Kafka and sending them to APEX
251 - *APEX / Policy* - the APEX engine running the VPA
254 - *Controller* - A vanilla Floodlight controller
255 taking events from the Link Monitor and configuring
258 - *Network* - A network created using Mininet
260 .. container:: paragraph
262 The demo requires to start some software (detailed
263 below). To show actual video streams, we use ``VLC``.
264 If you do not want to show video streams, but only the
265 policy, skip the ``VLC`` section.
267 .. container:: paragraph
269 All shown scripts are available in a full APEX
271 ``$APEX_HOME/examples/scripts/pcvs/vpnsla``.
275 .. rubric:: Start all Software
276 :name: start_all_software
278 .. container:: paragraph
280 Create environment variables in a file, say
281 ``env.sh``. In each new Xterm
285 - Source these environment settings, e.g.
288 - Run the commands below as root (``sudo`` per
289 command or ``sudo -i`` for interactive mode as
292 .. container:: listingblock
294 .. container:: content
300 export src_dir=/usr/local/src
301 export APEX_HOME=/opt/ericsson/apex/apex
302 export APEX_USER=apexuser
304 .. container:: paragraph
306 In a new Xterm, start Floodlight
308 .. container:: listingblock
310 .. container:: content
316 cd $src_dir/floodlight-master && java -jar target/floodlight.jar
318 .. container:: paragraph
320 In a new Xterm start Mininet
322 .. container:: listingblock
324 .. container:: content
330 mn -c && python $APEX_HOME/examples/scripts/pcvs/vpnsla/MininetTopology.py
332 .. container:: paragraph
334 In a new Xterm, start Kafka
336 .. container:: listingblock
338 .. container:: content
344 /opt/Kafka/kafka_2.12-1.0.0/bin/kafka-server-start.sh /opt/Kafka/kafka_2.12-1.0.0/config/server.properties
346 .. container:: paragraph
348 In a new Xerm start APEX with the Kafka
349 configuration for this demo
351 .. container:: listingblock
353 .. container:: content
358 ./bin/apexApps.sh engine -c examples/config/pcvs/vpnsla/kafka2kafka.json
360 .. container:: paragraph
362 In a new Xterm start the Link Monitor. The Link
363 Monitor has a 30 second sleep to slow down the
364 demonstration. So the first action of it comes 30
365 seconds after start. Every new action in 30 second
368 .. container:: listingblock
370 .. container:: content
377 xterm -hold -e 'python3 $src_dir/kafka-python/LinkMonitor.py' &
379 .. container:: paragraph
381 Now all software should be started and the demo is
382 running. The Link Monitor will send link up events,
383 picked up by APEX which triggers the policy. Since
384 there is no problem, the policy will do nothing.
388 .. rubric:: Create 2 Video Streams with VLC
389 :name: create_2_video_streams_with_vlc
391 .. container:: paragraph
393 In the Mininet console, type ``xterm A1 A2`` and
394 ``xterm B1 B2`` to open terminals on these nodes.
396 .. container:: paragraph
398 ``A2`` and ``B2`` are the receiving nodes. In these
399 terminals, run ``vlc-wrapper``. In each opened VLC
404 - Click Media → Open Network Stream
406 - Give the URL as ``rtp://@:5004``
408 .. container:: paragraph
410 ``A1`` and ``B1`` are the sending nodes (sending
411 the video stream) In these terminals, run
412 ``vlc-wrapper``. In each opened VLC window do
416 - Click Media → Stream
418 - Add the video (from ``/usr/local/src/videos``)
424 - Change the destination
425 ``RTP / MPEG Transport Stream`` and click
428 - Change the address and type to ``10.0.0.2`` in
429 ``A1`` and to ``10.0.0.4`` in ``B1``
431 - Turn off ``Active Transcoding`` (this is
432 important to minimize CPU load)
438 .. container:: paragraph
440 The video should be streaming across the network
441 from ``A1`` to ``A2`` and from ``B1`` to ``B2``. If
442 the video streams a slow or interrupted the CPU
443 load is too high. In these cases either try a
444 better machine or use a different (lower quality)
449 .. rubric:: Take out L09 and let the Policy do it’s
451 :name: take_out_l09_and_let_the_policy_do_it_s_magic
453 .. container:: paragraph
455 Now it is time to take out the link ``L09``. This
456 will be picked up by the Link Monitor, which sends
457 a new event (L09 DOWN) to the policy. The policy
458 then will calculate which customer should be
459 impeded (throttled). This will continue, until SLAs
460 are violated, then a priority calculation will kick
461 in (Customer ``A`` is prioritized in the setup).
463 .. container:: paragraph
465 To initiate this, simply type ``link s5 s6 down``
466 in the Mininet console followed by ``exit``.
468 .. container:: paragraph
470 If you have the video streams running, you will see
471 one or the other struggeling, depending on the
476 .. rubric:: Reset the Demo
477 :name: reset_the_demo
479 .. container:: paragraph
481 If you want to reset the demo, simple stop (in this
482 order) the following process
494 .. container:: paragraph
496 Then restart them in this order
510 .. rubric:: Monitor the Demo
511 :name: monitor_the_demo
513 .. container:: paragraph
515 Floodlight and APEX provide REST interfaces for
520 - Floodlight: see `Floodlight
521 Docs <https://floodlight.atlassian.net/wiki/spaces/floodlightcontroller/pages/40403023/Web+GUI>`__
522 for details on how to access the monitoring. In
523 a standard installation as we use here, pointing
525 ``http://localhost:8080/ui/pages/index.html``
526 should work on the same host
528 - APEX please see the APEX documentation for
530 Client <https://ericsson.github.io/apex-docs/user-manual/engine-apps/um-engapps-eng-monitoring.html>`__
532 Client <https://ericsson.github.io/apex-docs/user-manual/engine-apps/um-engapps-full-client.html>`__
533 for details on how to monitor APEX.
539 .. container:: sectionbody
541 .. container:: paragraph
543 The VPN SLA policy is designed as a MEDA policy. The
544 first state (M = Match) takes the trigger event (a
545 link up or down) and checks if this is a change to the
546 known topology. The second state (E = Establish) takes
547 all available information (trigger event, local
548 context) and defines what situation we have. The third
549 state (D = Decide) takes the situation and selects
550 which algorithm is best to process it. This state can
551 select between ``none`` (nothing to do), ``solved`` (a
552 problem is solved now), ``sla`` (compare the current
553 customer SLA situation and select one to impede), and
554 ``priority`` (impede non-priority customers). The
555 fourth and final state (A = Act) selects the right
556 action for the taken decision and creates the response
557 event sent to the orchestrator.
559 .. container:: paragraph
561 We have added three more policies to set the local
562 context: one for adding nodes, one for adding edges
563 (links), and one for adding customers. These policies
564 do not realize any action, they are only here for
565 updating the local context. This mechanism is the
566 fasted way to update local context, and it is
567 independent of any context plugin.
569 .. container:: paragraph
571 The policy uses data defined in Avro, so we have a
572 number of Avro schema definitions.
579 .. container:: sectionbody
581 .. container:: paragraph
583 The context schemas are for the local context. We
584 model edges and nodes for the topology, customers, and
585 problems with all information on detected problems.
589 - |avroSchemaEdges_link|
590 - |avroSchemaNodes_link|
591 - |avroSchemaCustomers_link|
599 .. container:: sectionbody
601 .. container:: paragraph
603 The trigger event provides a status as ``UP`` or
604 ``DOWN``. To avoid tests for these strings in the
605 logic, we defined an Avro schema for an enumeration (|avroSchemaLinkStatus_link|).
606 This does not impact the trigger system (it can still
607 send the strings), but makes the task logic simpler.
616 .. container:: sectionbody
618 .. container:: paragraph
620 The node context logic simply takes the trigger event
621 (for context) and creates a new node in the local
622 context topology (|cntxtNodes_link|).
629 .. container:: sectionbody
631 .. container:: paragraph
633 The edge context logic simply takes the trigger event
634 (for context) and creates a new edge in the local
635 context topology (|cntxtEdges_link|).
638 Context Logic Customer
639 ----------------------
643 .. container:: sectionbody
645 .. container:: paragraph
647 The customer context logic simply takes the trigger
648 event (for context) and creates a new customer in the
649 local context topology (|cntxtCustomer_link|).
657 .. container:: sectionbody
659 .. container:: paragraph
661 This is the logic for the match state. It is kept very
662 simple. Beside taking the trigger event, it also
663 creates a timestamp. This timestamp is later used for
664 SLA and downtime calculations as well as for some
665 performance information of the policy .
666 Sample |taskMatch_link|
669 Logic: Policy Establish State
670 -----------------------------
674 .. container:: sectionbody
676 .. container:: paragraph
678 This is the logic for the establish state. It is the
679 most complicated logic, since establishing a situation
680 for a decision is the most important part of any
681 policy. First, the policy describes what we find (the
682 switch block), in terms of 8 normal situations and 1
685 .. container:: paragraph
687 If required, it creates local context information for
688 the problem (if it is new) or updates it (if the
689 problem still exists). It also calculates customer SLA
690 downtime and checks for any SLA violations. Finally,
691 it creates a situation object.
692 Sample |taskEstablish_link|
694 Logic: Policy Decide State
695 --------------------------
699 .. container:: sectionbody
701 .. container:: paragraph
703 The decide state can select between different
704 algorithms depending on the situation. So it needs a
705 Task Selection Logic (TSL). This TSL select a task in
706 the current policy execution (i.e. potentially a
707 different one per execution).
708 Sample |tslDecide_Link|
710 .. container:: paragraph
712 The actual task logic are then ``none``, ``solved``,
713 ``sla``, and ``priority``.
714 Sample task logic are as given below :
718 - |taskDecideNone_link|
719 - |taskDecideSolved_Link|
720 - |taskDecideSla_link|
721 - |taskDecidePriority_Link|
724 Logic: Policy Act State
725 ------------------------
729 .. container:: sectionbody
731 .. container:: paragraph
733 This is the logic for the act state. It is simply
734 selecting an action, and creating the repsonse event
735 for the orchestrator (the output of the policy).
736 Sample |taskAct_link|
743 .. rubric:: Complete Policy Definition
744 :name: complete_policy_definition
746 .. container:: sectionbody
748 .. container:: paragraph
750 The complete policy definition is realized using the
751 APEX CLI Editor. The script below shows the actual
752 policy specification. All logic and schemas are
753 included (as macro file).
754 Sample |vpnsla-apex_link|
762 .. container:: sectionbody
764 .. container:: paragraph
766 The following events create all nodes of the topology.
770 - |ctxtNodesJson_link|
778 .. container:: sectionbody
780 .. container:: paragraph
782 The following events create all edges of the topology.
786 - |ctxtEdgesJson_link|
789 Context Events Customers
790 ------------------------
794 .. container:: sectionbody
796 .. container:: paragraph
798 The following events create all customers of the topology.
802 - |ctxtCustomerJson_link|
810 .. container:: sectionbody
812 .. container:: paragraph
814 The following events are examples for trigger events
818 - |triggerEdgeJson_link|
825 .. container:: sectionbody
827 .. container:: paragraph
829 The Link Monitor is a Python script. At startup, it
830 sends the context events to APEX to initialize the
831 topology and the customers. Then it takes events from
832 Kafka and sends them to APEX.
833 Sample |linkMonitor_link|
840 .. container:: sectionbody
842 .. container:: paragraph
844 The topology is realized using Mininet. This
845 script is used to establish the topology and to realize
846 network configurations.
847 Sample |mininetTopology_link|
854 Last updated 2020-04-03 16:04:24 IST
857 .. |ONAP| image:: ../../../images/logos.png
859 :target: http://www.onap.org/
861 .. |VPN SLA Architecture| image:: images/pcvs/vpnsla-arch.png
863 .. |avroSchemaEdges_link| raw:: html
865 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/avro/topology-edges.avsc" target="_blank">AVRO Schema Edges</a>
866 .. |avroSchemaNodes_link| raw:: html
868 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src//main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/avro/topology-nodes.avsc" target="_blank">AVRO Schema Nodes</a>
869 .. |avroSchemaCustomers_link| raw:: html
871 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/avro/customers.avsc" target="_blank">AVRO Schema Customers</a>
872 .. |avroSchemaLinkStatus_link| raw:: html
874 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/avro/link-status.avsc" target="_blank">AVRO Schema Link Status</a>
875 .. |cntxtNodes_link| raw:: html
877 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/logic/ctxt-nodes.js" target="_blank">Logic Node Context</a>
878 .. |cntxtEdges_link| raw:: html
880 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/logic/ctxt-edges.js" target="_blank">Logic Edge Context</a>
881 .. |cntxtCustomer_link| raw:: html
883 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/logic/ctxt-customer.js" target="_blank">Logic Customer Context</a>
884 .. |taskMatch_link| raw:: html
886 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/logic/task-match.js" target="_blank">Logic Policy Match State</a>
887 .. |taskEstablish_link| raw:: html
889 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/logic/task-establish.js" target="_blank">Logic Policy Establish State</a>
890 .. |tslDecide_Link| raw:: html
892 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/logic/tsl-decide.js" target="_blank">JS Logic Policy Decide State - TSL</a>
893 .. |taskDecideNone_link| raw:: html
895 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/logic/task-decide-none.js" target="_blank">Logic: Decide None</a>
896 .. |taskDecideSolved_Link| raw:: html
898 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/logic/task-decide-solved.js" target="_blank">Logic: Decide Solved</a>
899 .. |taskDecideSla_link| raw:: html
901 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/logic/task-decide-sla.js" target="_blank">Logic: Decide SLA</a>
902 .. |taskDecidePriority_Link| raw:: html
904 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/logic/task-decide-priority.js" target="_blank">Logic: Decide Priority</a>
905 .. |taskAct_link| raw:: html
907 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/logic/task-act.js" target="_blank">Logic Policy Act State</a>
908 .. |vpnsla-apex_link| raw:: html
910 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/vpnsla.apex" target="_blank">APEX VPN SLA Policy Specification</a>
911 .. |ctxtNodesJson_link| raw:: html
913 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/examples/events/pcvs/vpnsla/CtxtNodes.json" target="_blank">Context Events Nodes</a>
914 .. |ctxtEdgesJson_link| raw:: html
916 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/examples/events/pcvs/vpnsla/CtxtEdges.json" target="_blank">Context Events Edges</a>
917 .. |ctxtCustomerJson_link| raw:: html
919 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/examples/events/pcvs/vpnsla/CtxtCustomers.json" target="_blank">Context Events Customers</a>
920 .. |triggerEdgeJson_link| raw:: html
922 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/examples/events/pcvs/vpnsla/Trigger_Edge09.json" target="_blank">Trigger Examples</a>
923 .. |linkMonitor_link| raw:: html
925 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/examples/scripts/pcvs/vpnsla/LinkMonitor.py" target="_blank">Link Monitor</a>
926 .. |mininetTopology_link| raw:: html
928 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/examples/scripts/pcvs/vpnsla/MininetTopology.py" target="_blank">Mininet Topology</a>