1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
2 .. http://creativecommons.org/licenses/by/4.0
6 Policy-controlled Video Streaming (pcvs) with APEX
7 **************************************************
15 .. container:: sectionbody
17 .. container:: paragraph
19 This module contains several demos for
20 Policy-controlled Video Streaming (PCVS). Each demo
21 defines a policy using AVRO and Javascript (or other
22 scripting languages for the policy logic). To run the
23 demo, a vanilla Ubuntu server with some extra software packages is required:
27 - Mininet as network simulator
29 - Floodlight as SDN controller
31 - Kafka as messaging system
33 - Zookeeper for Kafka configuration
35 - APEX for policy control
37 Install Ubuntu Server and SW
38 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
42 .. rubric:: Install Demo
45 .. container:: sectionbody
47 .. container:: paragraph
53 - Ubuntu server: 1.4 GB
55 - Ubuntu with Xubuntu Desktop, git, Firefox: 2.3 GB
57 - Ubuntu with all, system updated: 3 GB
59 - With ZK, Kafka, VLC, Mininet, Floodlight, Python:
62 - APEX Build (M2 and built): M2 ~ 2 GB, APEX ~3.5 GB
64 - APEX install (not build locally): ~ 300 MB
66 .. container:: paragraph
68 On a Ubuntu OS (install a stable or LTS server first)
70 .. container:: listingblock
72 .. container:: content
76 # pre for Ubuntu, tools and X
77 sudo apt-get -y install --no-install-recommends software-properties-common
78 sudo apt-get -y install --no-install-recommends build-essential
79 sudo apt-get -y install --no-install-recommends git
80 sudo aptitude -y install --no-install-recommends xubuntu-desktop
81 sudo apt-get -y install --no-install-recommends firefox
85 sudo add-apt-repository ppa:webupd8team/java
87 sudo apt-get -y install --no-install-recommends oracle-java8-installer
91 # reboot system, run system update, then continue
93 # if VBox additions are needed, install and reboot
94 sudo (cd /usr/local/share; wget https://www.virtualbox.org/download/testcase/VBoxGuestAdditions_5.2.7-120528.iso)
95 sudo mount /usr/local/share/VBoxGuestAdditions_5.2.7-120528.iso /media/cdrom
96 sudo (cd /media/cdrom;VBoxLinuxAdditions.run)
102 # if APEX is build from source, install maven and rpm
103 sudo apt-get install maven rpm
106 sudo apt-get install zookeeperd
109 (cd /tmp;wget http://ftp.heanet.ie/mirrors/www.apache.org/dist/kafka/1.0.0/kafka_2.12-1.0.0.tgz --show-progress)
110 sudo mkdir /opt/Kafka
111 sudo tar -xvf /tmp/kafka_2.12-1.0.0.tgz -C /opt/Kafka/
115 sudo git clone https://github.com/mininet/mininet.git
116 (cd mininet;util/install.sh -a)
118 # install floodlight, requires ant
119 sudo apt-get install ant
121 sudo wget --no-check-certificate https://github.com/floodlight/floodlight/archive/master.zip
122 sudo unzip master.zip
125 sudo mkdir /var/lib/floodlight
126 sudo chmod 777 /var/lib/floodlight
129 sudo apt-get install python-pip
131 # install kafka-python (need newer version from github)
133 sudo git clone https://github.com/dpkp/kafka-python
134 sudo pip install ./kafka-python
137 sudo apt-get install vlc
139 .. container:: paragraph
141 Install APEX either from source or from a distribution
142 package. See the APEX documentation for details. We
143 assume that APEX is installed in
144 ``/opt/ericsson/apex/apex``
146 .. container:: paragraph
148 Copy the LinkMonitor file to Kafka-Python
150 .. container:: listingblock
152 .. container:: content
156 sudo cp /opt/ericsson/apex/apex/examples/scripts/pcvs/vpnsla/LinkMonitor.py /usr/local/src/kafka-python
158 .. container:: paragraph
160 Change the Logback configuration in APEX to logic
163 .. container:: listingblock
165 .. container:: content
169 (cd /opt/ericsson/apex/apex/etc; sudo cp logback-logic.xml logback.xml)
173 .. rubric:: Get the Demo Video
174 :name: get_the_demo_video
176 .. container:: sectionbody
180 - For all download options of the movie please visit
181 http://bbb3d.renderfarming.net/download.html
183 - For lower-res downloads and mirrors see
184 https://peach.blender.org/download
186 .. container:: listingblock
188 .. container:: content
192 sudo mkdir /usr/local/src/videos
194 .. container:: paragraph
196 Standard 720p (recommended)
198 .. container:: listingblock
200 .. container:: content
204 (cd /usr/local/src/videos; sudo curl -o big_buck_bunny_480p_surround.avi http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_480p_surround-fix.avi)
206 .. container:: paragraph
210 .. container:: listingblock
212 .. container:: content
216 (cd videos; sudo curl -o bbb_sunflower_1080p_60fps_normal.mp4 http://distribution.bbb3d.renderfarming.net/video/mp4/bbb_sunflower_1080p_60fps_normal.mp4)
225 .. container:: sectionbody
227 .. container:: paragraph
229 This demo uses a network with several central office
230 and core switches, over which two VPNs are run. A
231 customer ``A`` has two location ``A1`` and ``A2`` and
232 a VPN between them. A customer ``B`` has two location
233 ``B1`` and ``B2`` and a VPN between them.
235 .. container:: imageblock
237 .. container:: content
239 |VPN SLA Architecture|
241 .. container:: paragraph
243 The architecture above shows the scenario. The
244 components are realized in this demo as follows:
248 - *CEP / Analytics* - a simple Python script taking
249 events from Kafka and sending them to APEX
251 - *APEX / Policy* - the APEX engine running the VPA
254 - *Controller* - A vanilla Floodlight controller
255 taking events from the Link Monitor and configuring
258 - *Network* - A network created using Mininet
260 .. container:: paragraph
262 The demo requires to start some software (detailed
263 below). To show actual video streams, we use ``VLC``.
264 If you do not want to show video streams, but only the
265 policy, skip the ``VLC`` section.
267 .. container:: paragraph
269 All shown scripts are available in a full APEX
271 ``$APEX_HOME/examples/scripts/pcvs/vpnsla``.
275 .. rubric:: Start all Software
276 :name: start_all_software
278 .. container:: paragraph
280 Create environment variables in a file, say
281 ``env.sh``. In each new Xterm
285 - Source these environment settings, e.g.
288 - Run the commands below as root (``sudo`` per
289 command or ``sudo -i`` for interactive mode as
292 .. container:: listingblock
294 .. container:: content
300 export src_dir=/usr/local/src
301 export APEX_HOME=/opt/ericsson/apex/apex
302 export APEX_USER=apexuser
304 .. container:: paragraph
306 In a new Xterm, start Floodlight
308 .. container:: listingblock
310 .. container:: content
316 cd $src_dir/floodlight-master && java -jar target/floodlight.jar
318 .. container:: paragraph
320 In a new Xterm start Mininet
322 .. container:: listingblock
324 .. container:: content
330 mn -c && python $APEX_HOME/examples/scripts/pcvs/vpnsla/MininetTopology.py
332 .. container:: paragraph
334 In a new Xterm, start Kafka
336 .. container:: listingblock
338 .. container:: content
344 /opt/Kafka/kafka_2.12-1.0.0/bin/kafka-server-start.sh /opt/Kafka/kafka_2.12-1.0.0/config/server.properties
346 .. container:: paragraph
348 In a new Xerm start APEX with the Kafka
349 configuration for this demo
351 .. container:: listingblock
353 .. container:: content
358 ./bin/apexApps.sh engine -c examples/config/pcvs/vpnsla/kafka2kafka.json
360 .. container:: paragraph
362 In a new Xterm start the Link Monitor. The Link
363 Monitor has a 30 second sleep to slow down the
364 demonstration. So the first action of it comes 30
365 seconds after start. Every new action in 30 second
368 .. container:: listingblock
370 .. container:: content
377 xterm -hold -e 'python3 $src_dir/kafka-python/LinkMonitor.py' &
379 .. container:: paragraph
381 Now all software should be started and the demo is
382 running. The Link Monitor will send link up events,
383 picked up by APEX which triggers the policy. Since
384 there is no problem, the policy will do nothing.
388 .. rubric:: Create 2 Video Streams with VLC
389 :name: create_2_video_streams_with_vlc
391 .. container:: paragraph
393 In the Mininet console, type ``xterm A1 A2`` and
394 ``xterm B1 B2`` to open terminals on these nodes.
396 .. container:: paragraph
398 ``A2`` and ``B2`` are the receiving nodes. In these
399 terminals, run ``vlc-wrapper``. In each opened VLC
404 - Click Media → Open Network Stream
406 - Give the URL as ``rtp://@:5004``
408 .. container:: paragraph
410 ``A1`` and ``B1`` are the sending nodes (sending
411 the video stream) In these terminals, run
412 ``vlc-wrapper``. In each opened VLC window do
416 - Click Media → Stream
418 - Add the video (from ``/usr/local/src/videos``)
424 - Change the destination
425 ``RTP / MPEG Transport Stream`` and click
428 - Change the address and type to ``10.0.0.2`` in
429 ``A1`` and to ``10.0.0.4`` in ``B1``
431 - Turn off ``Active Transcoding`` (this is
432 important to minimize CPU load)
438 .. container:: paragraph
440 The video should be streaming across the network
441 from ``A1`` to ``A2`` and from ``B1`` to ``B2``. If
442 the video streams a slow or interrupted the CPU
443 load is too high. In these cases either try a
444 better machine or use a different (lower quality)
449 .. rubric:: Take out L09 and let the Policy do it’s
451 :name: take_out_l09_and_let_the_policy_do_it_s_magic
453 .. container:: paragraph
455 Now it is time to take out the link ``L09``. This
456 will be picked up by the Link Monitor, which sends
457 a new event (L09 DOWN) to the policy. The policy
458 then will calculate which customer should be
459 impeded (throttled). This will continue, until SLAs
460 are violated, then a priority calculation will kick
461 in (Customer ``A`` is prioritized in the setup).
463 .. container:: paragraph
465 To initiate this, simply type ``link s5 s6 down``
466 in the Mininet console followed by ``exit``.
468 .. container:: paragraph
470 If you have the video streams running, you will see
471 one or the other struggeling, depending on the
476 .. rubric:: Reset the Demo
477 :name: reset_the_demo
479 .. container:: paragraph
481 If you want to reset the demo, simple stop (in this
482 order) the following process
494 .. container:: paragraph
496 Then restart them in this order
510 .. rubric:: Monitor the Demo
511 :name: monitor_the_demo
513 .. container:: paragraph
515 Floodlight and APEX provide REST interfaces for
520 - Floodlight: see `Floodlight
521 Docs <https://floodlight.atlassian.net/wiki/spaces/floodlightcontroller/pages/40403023/Web+GUI>`__
522 for details on how to access the monitoring. In
523 a standard installation as we use here, pointing
525 ``http://localhost:8080/ui/pages/index.html``
526 should work on the same host
531 .. container:: sectionbody
533 .. container:: paragraph
535 The VPN SLA policy is designed as a MEDA policy. The
536 first state (M = Match) takes the trigger event (a
537 link up or down) and checks if this is a change to the
538 known topology. The second state (E = Establish) takes
539 all available information (trigger event, local
540 context) and defines what situation we have. The third
541 state (D = Decide) takes the situation and selects
542 which algorithm is best to process it. This state can
543 select between ``none`` (nothing to do), ``solved`` (a
544 problem is solved now), ``sla`` (compare the current
545 customer SLA situation and select one to impede), and
546 ``priority`` (impede non-priority customers). The
547 fourth and final state (A = Act) selects the right
548 action for the taken decision and creates the response
549 event sent to the orchestrator.
551 .. container:: paragraph
553 We have added three more policies to set the local
554 context: one for adding nodes, one for adding edges
555 (links), and one for adding customers. These policies
556 do not realize any action, they are only here for
557 updating the local context. This mechanism is the
558 fasted way to update local context, and it is
559 independent of any context plugin.
561 .. container:: paragraph
563 The policy uses data defined in Avro, so we have a
564 number of Avro schema definitions.
571 .. container:: sectionbody
573 .. container:: paragraph
575 The context schemas are for the local context. We
576 model edges and nodes for the topology, customers, and
577 problems with all information on detected problems.
581 - |avroSchemaEdges_link|
582 - |avroSchemaNodes_link|
583 - |avroSchemaCustomers_link|
591 .. container:: sectionbody
593 .. container:: paragraph
595 The trigger event provides a status as ``UP`` or
596 ``DOWN``. To avoid tests for these strings in the
597 logic, we defined an Avro schema for an enumeration (|avroSchemaLinkStatus_link|).
598 This does not impact the trigger system (it can still
599 send the strings), but makes the task logic simpler.
608 .. container:: sectionbody
610 .. container:: paragraph
612 The node context logic simply takes the trigger event
613 (for context) and creates a new node in the local
614 context topology (|cntxtNodes_link|).
621 .. container:: sectionbody
623 .. container:: paragraph
625 The edge context logic simply takes the trigger event
626 (for context) and creates a new edge in the local
627 context topology (|cntxtEdges_link|).
630 Context Logic Customer
631 ----------------------
635 .. container:: sectionbody
637 .. container:: paragraph
639 The customer context logic simply takes the trigger
640 event (for context) and creates a new customer in the
641 local context topology (|cntxtCustomer_link|).
649 .. container:: sectionbody
651 .. container:: paragraph
653 This is the logic for the match state. It is kept very
654 simple. Beside taking the trigger event, it also
655 creates a timestamp. This timestamp is later used for
656 SLA and downtime calculations as well as for some
657 performance information of the policy .
658 Sample |taskMatch_link|
661 Logic: Policy Establish State
662 -----------------------------
666 .. container:: sectionbody
668 .. container:: paragraph
670 This is the logic for the establish state. It is the
671 most complicated logic, since establishing a situation
672 for a decision is the most important part of any
673 policy. First, the policy describes what we find (the
674 switch block), in terms of 8 normal situations and 1
677 .. container:: paragraph
679 If required, it creates local context information for
680 the problem (if it is new) or updates it (if the
681 problem still exists). It also calculates customer SLA
682 downtime and checks for any SLA violations. Finally,
683 it creates a situation object.
684 Sample |taskEstablish_link|
686 Logic: Policy Decide State
687 --------------------------
691 .. container:: sectionbody
693 .. container:: paragraph
695 The decide state can select between different
696 algorithms depending on the situation. So it needs a
697 Task Selection Logic (TSL). This TSL select a task in
698 the current policy execution (i.e. potentially a
699 different one per execution).
700 Sample |tslDecide_Link|
702 .. container:: paragraph
704 The actual task logic are then ``none``, ``solved``,
705 ``sla``, and ``priority``.
706 Sample task logic are as given below :
710 - |taskDecideNone_link|
711 - |taskDecideSolved_Link|
712 - |taskDecideSla_link|
713 - |taskDecidePriority_Link|
716 Logic: Policy Act State
717 -----------------------
721 .. container:: sectionbody
723 .. container:: paragraph
725 This is the logic for the act state. It is simply
726 selecting an action, and creating the repsonse event
727 for the orchestrator (the output of the policy).
728 Sample |taskAct_link|
735 .. rubric:: Complete Policy Definition
736 :name: complete_policy_definition
738 .. container:: sectionbody
740 .. container:: paragraph
742 The complete policy definition is realized using the
743 APEX CLI Editor. The script below shows the actual
744 policy specification. All logic and schemas are
745 included (as macro file).
746 Sample |vpnsla-apex_link|
754 .. container:: sectionbody
756 .. container:: paragraph
758 The following events create all nodes of the topology.
762 - |ctxtNodesJson_link|
770 .. container:: sectionbody
772 .. container:: paragraph
774 The following events create all edges of the topology.
778 - |ctxtEdgesJson_link|
781 Context Events Customers
782 ------------------------
786 .. container:: sectionbody
788 .. container:: paragraph
790 The following events create all customers of the topology.
794 - |ctxtCustomerJson_link|
802 .. container:: sectionbody
804 .. container:: paragraph
806 The following events are examples for trigger events
810 - |triggerEdgeJson_link|
817 .. container:: sectionbody
819 .. container:: paragraph
821 The Link Monitor is a Python script. At startup, it
822 sends the context events to APEX to initialize the
823 topology and the customers. Then it takes events from
824 Kafka and sends them to APEX.
825 Sample |linkMonitor_link|
832 .. container:: sectionbody
834 .. container:: paragraph
836 The topology is realized using Mininet. This
837 script is used to establish the topology and to realize
838 network configurations.
839 Sample |mininetTopology_link|
846 Last updated 2020-04-03 16:04:24 IST
849 .. |VPN SLA Architecture| image:: images/pcvs/vpnsla-arch.png
851 .. |avroSchemaEdges_link| raw:: html
853 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/avro/topology-edges.avsc" target="_blank">AVRO Schema Edges</a>
854 .. |avroSchemaNodes_link| raw:: html
856 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src//main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/avro/topology-nodes.avsc" target="_blank">AVRO Schema Nodes</a>
857 .. |avroSchemaCustomers_link| raw:: html
859 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/avro/customers.avsc" target="_blank">AVRO Schema Customers</a>
860 .. |avroSchemaLinkStatus_link| raw:: html
862 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/avro/link-status.avsc" target="_blank">AVRO Schema Link Status</a>
863 .. |cntxtNodes_link| raw:: html
865 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/logic/ctxt-nodes.js" target="_blank">Logic Node Context</a>
866 .. |cntxtEdges_link| raw:: html
868 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/logic/ctxt-edges.js" target="_blank">Logic Edge Context</a>
869 .. |cntxtCustomer_link| raw:: html
871 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/logic/ctxt-customer.js" target="_blank">Logic Customer Context</a>
872 .. |taskMatch_link| raw:: html
874 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/logic/task-match.js" target="_blank">Logic Policy Match State</a>
875 .. |taskEstablish_link| raw:: html
877 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/logic/task-establish.js" target="_blank">Logic Policy Establish State</a>
878 .. |tslDecide_Link| raw:: html
880 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/logic/tsl-decide.js" target="_blank">JS Logic Policy Decide State - TSL</a>
881 .. |taskDecideNone_link| raw:: html
883 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/logic/task-decide-none.js" target="_blank">Logic: Decide None</a>
884 .. |taskDecideSolved_Link| raw:: html
886 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/logic/task-decide-solved.js" target="_blank">Logic: Decide Solved</a>
887 .. |taskDecideSla_link| raw:: html
889 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/logic/task-decide-sla.js" target="_blank">Logic: Decide SLA</a>
890 .. |taskDecidePriority_Link| raw:: html
892 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/logic/task-decide-priority.js" target="_blank">Logic: Decide Priority</a>
893 .. |taskAct_link| raw:: html
895 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/logic/task-act.js" target="_blank">Logic Policy Act State</a>
896 .. |vpnsla-apex_link| raw:: html
898 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/org/onap/policy/apex/examples/pcvs/vpnsla/vpnsla.apex" target="_blank">APEX VPN SLA Policy Specification</a>
899 .. |ctxtNodesJson_link| raw:: html
901 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/examples/events/pcvs/vpnsla/CtxtNodes.json" target="_blank">Context Events Nodes</a>
902 .. |ctxtEdgesJson_link| raw:: html
904 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/examples/events/pcvs/vpnsla/CtxtEdges.json" target="_blank">Context Events Edges</a>
905 .. |ctxtCustomerJson_link| raw:: html
907 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/examples/events/pcvs/vpnsla/CtxtCustomers.json" target="_blank">Context Events Customers</a>
908 .. |triggerEdgeJson_link| raw:: html
910 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/examples/events/pcvs/vpnsla/Trigger_Edge09.json" target="_blank">Trigger Examples</a>
911 .. |linkMonitor_link| raw:: html
913 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/examples/scripts/pcvs/vpnsla/LinkMonitor.py" target="_blank">Link Monitor</a>
914 .. |mininetTopology_link| raw:: html
916 <a href="https://github.com/onap/policy-apex-pdp/tree/master/examples/examples-pcvs/src/main/resources/examples/scripts/pcvs/vpnsla/MininetTopology.py" target="_blank">Mininet Topology</a>