docs/distribution/Distribution-User-Manual.rst

   1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
   2 .. http://creativecommons.org/licenses/by/4.0
   3
   4
   5 Policy Distribution User Manual
   6 *******************************
   7
   8 .. contents::
   9     :depth: 3
  10
  11 Installation
  12 ^^^^^^^^^^^^
  13
  14 Requirements
  15 ------------
  16
  17             .. container:: paragraph
  18
  19                Distribution is 100% written in Java and runs on any platform
  20                that supports a JVM, e.g. Windows, Unix, Cygwin.
  21
  22 Installation Requirements
  23 #########################
  24
  25                .. container:: ulist
  26
  27                   -  Downloaded distribution: JAVA runtime environment
  28                      (JRE, Java 11, Distribution is tested with the
  29                      OpenJDK)
  30
  31                   -  Building from source: JAVA development kit (JDK,
  32                      Java 11, Distribution is tested with the OpenJDK)
  33
  34                   -  Sufficient rights to install Distribution on the system
  35
  36                   -  Installation tools
  37
  38                      .. container:: ulist
  39
  40                         -  TAR and GZ to extract from that TAR.GZ
  41                            distribution
  42
  43                            .. container:: ulist
  44
  45                               -  Windows for instance
  46                                  `7Zip <http://www.7-zip.org/>`__
  47
  48                         -  `Docker <https://www.docker.com/>`__ to run Distribution
  49                            inside a Docker container
  50
  51
  52 Build (Install from Source) Requirements
  53 ########################################
  54
  55                .. container:: paragraph
  56
  57                   Installation from source requires a few development
  58                   tools
  59
  60                .. container:: ulist
  61
  62                   -  GIT to retrieve the source code
  63
  64                   -  Java SDK, Java version 8 or later
  65
  66                   -  Apache Maven 3 (the Distribution build environment)
  67
  68 Get the Distribution Source Code
  69 --------------------------------
  70
  71             .. container:: paragraph
  72
  73                The Distribution source code is hosted in ONAP as project distribution.
  74                The current stable version is in the master branch.
  75                Simply clone the master branch from ONAP using HTTPS.
  76
  77             .. container:: listingblock
  78
  79                .. container:: content
  80
  81                   .. code:: text
  82
  83                      git clone https://gerrit.onap.org/r/policy/distribution
  84
  85 Build Distribution
  86 ------------------
  87
  88    .. container:: paragraph
  89
  90       The examples in this document assume that the distribution source
  91       repositories are cloned to:
  92
  93    .. container:: ulist
  94
  95       -  Unix, Cygwin: ``/usr/local/src/distribution``
  96
  97       -  Windows: ``C:\dev\distribution``
  98
  99       -  Cygwin: ``/cygdrive/c/dev/distribution``
 100
 101    .. important::
 102       A Build requires ONAP Nexus
 103       Distribution has a dependency to ONAP parent projects. You might need to adjust your Maven M2 settings. The most current
 104       settings can be found in the ONAP oparent repo: `Settings <https://git.onap.org/oparent/plain/settings.xml>`__.
 105
 106    .. important::
 107       A Build needs Space
 108       Building distribution requires approximately 1-2 GB of hard disc space, 100 MB for the actual build with full
 109       distribution and around 1 GB for the downloaded dependencies.
 110
 111    .. important::
 112       A Build requires Internet (for first build)
 113       During the build, several (a lot) of Maven dependencies will be downloaded and stored in the configured local Maven
 114       repository. The first standard build (and any first specific build) requires Internet access to download those
 115       dependencies.
 116
 117    .. container:: paragraph
 118
 119       Use Maven for a standard build without any tests.
 120
 121       +-------------------------------------------------------+--------------------------------------------------------+
 122       | Unix, Cygwin                                          | Windows                                                |
 123       +=======================================================+========================================================+
 124       | .. container::                                        | .. container::                                         |
 125       |                                                       |                                                        |
 126       |    .. container:: content                             |    .. container:: content                              |
 127       |                                                       |                                                        |
 128       |       .. code:: text                                  |       .. code:: text                                   |
 129       |                                                       |                                                        |
 130       |         # cd /usr/local/src/distribution              |          >c:                                           |
 131       |         # mvn clean install -DskipTest                |          >cd \dev\distribution                         |
 132       |                                                       |          >mvn clean install -DskipTests                |
 133       +-------------------------------------------------------+--------------------------------------------------------+
 134
 135 .. container:: paragraph
 136
 137    The build takes 2-3 minutes on a standard development laptop. It
 138    should run through without errors, but with a lot of messages from
 139    the build process.
 140
 141 |
 142
 143 .. container:: paragraph
 144
 145    When Maven is finished with the build, the final screen should look
 146    similar to this (omitting some ``success`` lines):
 147
 148 .. container:: listingblock
 149
 150    .. container:: content
 151
 152       .. code:: text
 153
 154         [INFO] ------------------------------------------------------------------------
 155         [INFO] Reactor Summary:
 156         [INFO]
 157         [INFO] policy-distribution ................................ SUCCESS [  3.666 s]
 158         [INFO] distribution-model ................................. SUCCESS [ 11.234 s]
 159         [INFO] forwarding ......................................... SUCCESS [  7.611 s]
 160         [INFO] reception .......................................... SUCCESS [  7.072 s]
 161         [INFO] main ............................................... SUCCESS [ 21.017 s]
 162         [INFO] plugins ............................................ SUCCESS [  0.453 s]
 163         [INFO] forwarding-plugins ................................. SUCCESS [01:20 min]
 164         [INFO] reception-plugins .................................. SUCCESS [ 18.545 s]
 165         [INFO] Policy Distribution Packages ....................... SUCCESS [  0.419 s]
 166         [INFO] ------------------------------------------------------------------------
 167         [INFO] BUILD SUCCESS
 168         [INFO] ------------------------------------------------------------------------
 169         [INFO] Total time: 02:39 min
 170         [INFO] Finished at: 2018-11-15T13:59:09Z
 171         [INFO] Final Memory: 73M/1207M
 172         [INFO] ------------------------------------------------------------------------
 173
 174 .. container:: paragraph
 175
 176    The build will have created all artifacts required for distribution
 177    installation. The following example show how to change to the target
 178    directory and how it should look.
 179
 180 +----------------------------------------------------------------------------------------------------------------------------+
 181 | Unix, Cygwin                                                                                                               |
 182 +============================================================================================================================+
 183 | .. container::                                                                                                             |
 184 |                                                                                                                            |
 185 |    .. container:: listingblock                                                                                             |
 186 |                                                                                                                            |
 187 |       .. container:: content                                                                                               |
 188 |                                                                                                                            |
 189 |          .. code:: text                                                                                                    |
 190 |                                                                                                                            |
 191 |             -rw-r--r-- 1 user 1049089    10616 Oct 31 13:35 checkstyle-checker.xml                                         |
 192 |             -rw-r--r-- 1 user 1049089      609 Oct 31 13:35 checkstyle-header.txt                                          |
 193 |             -rw-r--r-- 1 user 1049089      245 Oct 31 13:35 checkstyle-result.xml                                          |
 194 |             -rw-r--r-- 1 user 1049089       89 Oct 31 13:35 checkstyle-cachefile                                           |
 195 |             drwxr-xr-x 1 user 1049089        0 Oct 31 13:35 maven-archiver/                                                |
 196 |             -rw-r--r-- 1 user 1049089     7171 Oct 31 13:35 policy-distribution-tarball-2.0.1-SNAPSHOT.jar                 |
 197 |             drwxr-xr-x 1 user 1049089        0 Oct 31 13:35 archive-tmp/                                                   |
 198 |             -rw-r--r-- 1 user 1049089 66296012 Oct 31 13:35 policy-distribution-tarball-2.0.1-SNAPSHOT-tarball.tar.gz      |
 199 |             drwxr-xr-x 1 user 1049089        0 Nov 12 10:56 test-classes/                                                  |
 200 |             drwxr-xr-x 1 user 1049089        0 Nov 20 14:31 classes/                                                       |
 201 +----------------------------------------------------------------------------------------------------------------------------+
 202
 203 +-------------------------------------------------------------------------------------------------------------------+
 204 | Windows                                                                                                           |
 205 +===================================================================================================================+
 206 | .. container::                                                                                                    |
 207 |                                                                                                                   |
 208 |    .. container:: listingblock                                                                                    |
 209 |                                                                                                                   |
 210 |       .. container:: content                                                                                      |
 211 |                                                                                                                   |
 212 |          .. code:: text                                                                                           |
 213 |                                                                                                                   |
 214 |                 11/12/2018  10:56 AM    <DIR>          .                                                          |
 215 |                 11/12/2018  10:56 AM    <DIR>          ..                                                         |
 216 |                 10/31/2018  01:35 PM    <DIR>          archive-tmp                                                |
 217 |                 10/31/2018  01:35 PM                89 checkstyle-cachefile                                       |
 218 |                 10/31/2018  01:35 PM            10,616 checkstyle-checker.xml                                     |
 219 |                 10/31/2018  01:35 PM               609 checkstyle-header.txt                                      |
 220 |                 10/31/2018  01:35 PM               245 checkstyle-result.xml                                      |
 221 |                 11/20/2018  02:31 PM    <DIR>          classes                                                    |
 222 |                 10/31/2018  01:35 PM    <DIR>          maven-archiver                                             |
 223 |                 10/31/2018  01:35 PM        66,296,012 policy-distribution-tarball-2.0.1-SNAPSHOT-tarball.tar.gz  |
 224 |                 10/31/2018  01:35 PM             7,171 policy-distribution-tarball-2.0.1-SNAPSHOT.jar             |
 225 |                 11/12/2018  10:56 AM    <DIR>          test-classes                                               |
 226 +-------------------------------------------------------------------------------------------------------------------+
 227
 228 Install Distribution
 229 --------------------
 230
 231    .. container:: paragraph
 232
 233       Distribution can be installed in different ways:
 234
 235    .. container:: ulist
 236
 237       -  Windows, Unix, Cygwin: manually from a ``.tar.gz`` archive
 238
 239       -  Windows, Unix, Cygwin: build from source using Maven, then
 240          install manually
 241
 242 Install Manually from Archive (Windows, 7Zip, GUI)
 243 ##################################################
 244
 245    .. container:: paragraph
 246
 247       Download a ``tar.gz`` archive and copy the file into the install
 248       folder (in this example ``C:\distribution``). Assuming you are using 7Zip,
 249       right click on the file and extract the ``tar`` archive.
 250
 251 |
 252
 253       .. container:: content
 254
 255          Extract the TAR archive
 256
 257    .. container:: paragraph
 258
 259       Then right-click on the new created TAR file and extract the actual
 260       distribution.
 261
 262 |
 263
 264       .. container:: content
 265
 266          Extract the distribution
 267
 268    .. container:: paragraph
 269
 270       Inside the new distribution folder you see the main directories: ``bin``,
 271       ``etc``and ``lib``
 272
 273 |
 274
 275    .. container:: paragraph
 276
 277       Once extracted, please rename the created folder to
 278       ``distribution-full-2.0.2-SNAPSHOT``. This will keep the directory name in
 279       line with the rest of this documentation.
 280
 281 Build from Source
 282 -----------------
 283
 284 Build and Install Manually (Unix, Windows, Cygwin)
 285 ##################################################
 286
 287       .. container:: paragraph
 288
 289          Clone the Distribution GIT repositories into a directory. Go to that
 290          directory. Use Maven to build Distribution (all details on building
 291          Distribution from source can be found in *Distribution HowTo: Build*).
 292
 293       .. container:: paragraph
 294
 295          Now, take the ``.tar.gz`` file and install distribution.
 296
 297 Installation Layout
 298 -------------------
 299
 300    .. container:: paragraph
 301
 302       A full installation of distribution comes with the following layout.
 303
 304    .. container:: listingblock
 305
 306       .. container:: content
 307
 308                 - bin
 309                 - etc
 310                 - lib
 311
 312 Running Distribution in Docker
 313 ------------------------------
 314
 315 Run in ONAP
 316 ###########
 317
 318       .. container:: paragraph
 319
 320          Running distribution from the ONAP docker repository only requires 2
 321          commands:
 322
 323       .. container:: olist arabic
 324
 325          #. Log into the ONAP docker repo
 326
 327       .. container:: listingblock
 328
 329          .. container:: content
 330
 331             ::
 332
 333                docker login -u docker -p docker nexus3.onap.org:10003
 334
 335       .. container:: olist arabic
 336
 337          #. Run the distribution docker image
 338
 339       .. container:: listingblock
 340
 341          .. container:: content
 342
 343             ::
 344
 345                docker run -it --rm  nexus3.onap.org:10003/onap/policy-distribution:latest
 346
 347 Build a Docker Image
 348 ####################
 349
 350       .. container:: paragraph
 351
 352          Alternatively, one can use the Dockerfile defined in the Docker
 353          package to build an image.
 354
 355 Distribution Configurations Explained
 356 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 357
 358 Introduction to Distribution Configuration
 359 ------------------------------------------
 360
 361          .. container:: paragraph
 362
 363             A distribution engine can be configured to use various combinations
 364             of policy reception handlers, policy decoders and policy forwarders.
 365             The system is built using a plugin architecture. Each configuration
 366             option is realized by a plugin, which can be loaded and
 367             configured when the engine is started. New plugins can be
 368             added to the system at any time, though to benefit from a
 369             new plugin, an engine will need to be restarted.
 370
 371     |
 372
 373          .. container:: paragraph
 374
 375             The distribution already comes with sdc reception handler,
 376             file reception handler, hpa optimization policy decoder, file in csar policy decoder,
 377             policy lifecycle api forwarder.
 378
 379 General Configuration Format
 380 ----------------------------
 381
 382          .. container:: paragraph
 383
 384             The distribution configuration file is a JSON file containing a few
 385             main blocks for different parts of the configuration. Each
 386             block then holds the configuration details. The following
 387             code shows the main blocks:
 388
 389          .. container:: listingblock
 390
 391             .. container:: content
 392
 393                .. code:: text
 394
 395                   {
 396                     "restServerParameters":{
 397                       ... (1)
 398                     },
 399                     "receptionHandlerParameters":{ (2)
 400                       "pluginHandlerParameters":{ (3)
 401                         "policyDecoders":{...}, (4)
 402                         "policyForwarders":{...} (5)
 403                       }
 404                     },
 405                     "receptionHandlerConfigurationParameters":{
 406                       ... (6)
 407                     }
 408                     ,
 409                     "policyForwarderConfigurationParameters":{
 410                       ... (7)
 411                     }
 412                     ,
 413                     "policyDecoderConfigurationParameters":{
 414                       ... (8)
 415                     }
 416                   }
 417
 418          .. container:: colist arabic
 419
 420             +-----------------------------------+-----------------------------------+
 421             | **1**                             | rest server configuration         |
 422             +-----------------------------------+-----------------------------------+
 423             | **2**                             | reception handler plugin          |
 424             |                                   | configurations                    |
 425             +-----------------------------------+-----------------------------------+
 426             | **3**                             | plugin handler parameters         |
 427             |                                   | configuration                     |
 428             +-----------------------------------+-----------------------------------+
 429             | **4**                             | policy decoder plugin             |
 430             |                                   | configuration                     |
 431             +-----------------------------------+-----------------------------------+
 432             | **5**                             | policy forwarder plugin           |
 433             |                                   | configuration                     |
 434             +-----------------------------------+-----------------------------------+
 435             | **6**                             | reception handler plugin          |
 436             |                                   | parameters                        |
 437             +-----------------------------------+-----------------------------------+
 438             | **7**                             | policy forwarder plugin           |
 439             |                                   | parameters                        |
 440             +-----------------------------------+-----------------------------------+
 441             | **8**                             | policy decoder plugin             |
 442             |                                   | parameters                        |
 443             +-----------------------------------+-----------------------------------+
 444
 445 A configuration example
 446 -----------------------
 447
 448          .. container:: paragraph
 449
 450             The following example loads HPA use case & general tosca policy related plug-ins.
 451
 452          .. container:: paragraph
 453
 454             Notifications are consumed from SDC through SDC client.
 455             Consumed artifacts format is CSAR.
 456
 457          .. container:: paragraph
 458
 459             Generated policies are forwarded to policy lifecycle api's for creation & deployment.
 460
 461          .. container:: listingblock
 462
 463             .. container:: content
 464
 465                .. code:: text
 466
 467                 {
 468                     "name":"SDCDistributionGroup",
 469                     "restServerParameters":{
 470                         "host":"0.0.0.0",
 471                         "port":6969,
 472                         "userName":"healthcheck",
 473                         "password":"zb!XztG34"
 474                       },
 475                     "receptionHandlerParameters":{
 476                          "SDCReceptionHandler":{
 477                             "receptionHandlerType":"SDC",
 478                             "receptionHandlerClassName":"org.onap.policy.distribution.reception.handling.sdc.SdcReceptionHandler",
 479                                 "receptionHandlerConfigurationName":"sdcConfiguration",
 480                             "pluginHandlerParameters":{
 481                                 "policyDecoders":{
 482                                     "HpaDecoder":{
 483                                         "decoderType":"HpaDecoder",
 484                                         "decoderClassName":"org.onap.policy.distribution.reception.decoding.hpa.PolicyDecoderCsarHpa",
 485                                         "decoderConfigurationName": "csarToOptimizationPolicyConfiguration"
 486                                     },
 487                                     "ToscaPolicyDecoder":{
 488                                         "decoderType":"ToscaPolicyDecoder",
 489                                         "decoderClassName":"org.onap.policy.distribution.reception.decoding.policy.file.PolicyDecoderFileInCsarToPolicy",
 490                                         "decoderConfigurationName": "toscaPolicyDecoderConfiguration"
 491                                     }
 492                                 },
 493                                 "policyForwarders":{
 494                                     "LifeCycleApiForwarder":{
 495                                         "forwarderType":"LifeCycleAPI",
 496                                         "forwarderClassName":"org.onap.policy.distribution.forwarding.lifecycle.api.LifecycleApiPolicyForwarder",
 497                                         "forwarderConfigurationName": "lifecycleApiConfiguration"
 498                                     }
 499                                 }
 500                             }
 501                         }
 502                     },
 503                     "receptionHandlerConfigurationParameters":{
 504                         "sdcConfiguration":{
 505                             "parameterClassName":"org.onap.policy.distribution.reception.handling.sdc.SdcReceptionHandlerConfigurationParameterGroup",
 506                             "parameters":{
 507                                 "asdcAddress": "sdc-be.onap:8443",
 508                                 "messageBusAddress": [
 509                                 "message-router.onap"
 510                                  ],
 511                                 "user": "policy",
 512                                 "password": "Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U",
 513                                 "pollingInterval":20,
 514                                 "pollingTimeout":30,
 515                                 "consumerId": "policy-id",
 516                                 "artifactTypes": [
 517                                 "TOSCA_CSAR",
 518                                 "HEAT"
 519                                 ],
 520                                 "consumerGroup": "policy-group",
 521                                 "environmentName": "AUTO",
 522                                 "keystorePath": "null",
 523                                 "keystorePassword": "null",
 524                                 "activeserverTlsAuth": false,
 525                                 "isFilterinEmptyResources": true,
 526                                 "isUseHttpsWithDmaap": true
 527                             }
 528                         }
 529                     },
 530                     "policyDecoderConfigurationParameters":{
 531                         "csarToOptimizationPolicyConfiguration":{
 532                             "parameterClassName":"org.onap.policy.distribution.reception.decoding.hpa.PolicyDecoderCsarHpaParameters",
 533                             "parameters":{
 534                                 "policyNamePrefix": "oofCasablanca",
 535                                 "onapName": "OOF",
 536                                 "version": "1.0",
 537                                 "priority": "3",
 538                                 "riskType": "Test",
 539                                 "riskLevel": "2"
 540                             }
 541                         },
 542                         "toscaPolicyDecoderConfiguration":{
 543                             "parameterClassName":"org.onap.policy.distribution.reception.decoding.policy.file.PolicyDecoderFileInCsarToPolicyParameterGroup",
 544                             "parameters":{
 545                                 "policyFileName": "tosca_policy",
 546                                 "policyTypeFileName": "tosca_policy_type"
 547                             }
 548                         }
 549                     },
 550                     "policyForwarderConfigurationParameters":{
 551                         "lifecycleApiConfiguration": {
 552                             "parameterClassName": "org.onap.policy.distribution.forwarding.lifecycle.api.LifecycleApiForwarderParameters",
 553                             "parameters": {
 554                                 "apiParameters": {
 555                                     "hostName": "policy-api",
 556                                     "port": 6969,
 557                                     "userName": "healthcheck",
 558                                     "password": "zb!XztG34"
 559                                 },
 560                                 "papParameters": {
 561                                     "hostName": "policy-pap",
 562                                     "port": 6969,
 563                                     "userName": "healthcheck",
 564                                     "password": "zb!XztG34"
 565                                 },
 566                                 "isHttps": true,
 567                                 "deployPolicies": true
 568                             }
 569                         }
 570                     }
 571                 }
 572
 573
 574 The Distribution Engine
 575 ^^^^^^^^^^^^^^^^^^^^^^^
 576
 577          .. container:: paragraph
 578
 579             The Distribution engine can be started using ``policy-dist.sh`` script.
 580             The script is located in the source code at
 581             *distribution/packages/policy-distribution-docker/src/main/docker*
 582             directory
 583
 584     |
 585
 586          .. container:: paragraph
 587
 588             On UNIX and Cygwin systems use ``policy-dist.sh`` script.
 589
 590     |
 591
 592          .. container:: paragraph
 593
 594             On Windows systems navigate to the distribution installation directory.
 595             Run the following command
 596             ``java -cp "etc:lib\*" org.onap.policy.distribution.main.startstop.Main -c <config-file-path>``
 597
 598     |
 599
 600          .. container:: paragraph
 601
 602             The Distribution engine comes with CLI arguments for setting
 603             configuration. The configuration file is always required.
 604             The option ``-h`` prints a help screen.
 605
 606          .. container:: listingblock
 607
 608             .. container:: content
 609
 610                .. code:: text
 611
 612                   usage: org.onap.policy.distribution.main.startstop.Main [options...]
 613                   options
 614                   -c,--config-file <CONFIG_FILE>  the full path to the configuration file to use, the configuration file must be a Json file
 615                                                   containing the distribution configuration parameters
 616                   -h,--help                       outputs the usage of this command
 617                   -v,--version                    outputs the version of distribution system
 618
 619
 620 The Distribution REST End-points
 621 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 622
 623          .. container:: paragraph
 624
 625             The distribution engine comes with built-in REST based
 626             endpoints for fetching health check status & statistical data
 627             of running distribution system.
 628
 629          .. container:: listingblock
 630
 631             .. container:: content
 632
 633                .. code:: text
 634
 635                   # Example Output from curl http -a '{user}:{password}' :6969/healthcheck
 636
 637                       HTTP/1.1 200 OK
 638                     Content-Length: XXX
 639                     Content-Type: application/json
 640                     Date: Tue, 17 Apr 2018 10:51:14 GMT
 641                     Server: Jetty(9.3.20.v20170531)
 642                     {
 643                          "code":200,
 644                          "healthy":true,
 645                          "message":"alive",
 646                          "name":"Policy SSD",
 647                          "url":"self"
 648                     }
 649
 650                   # Example Output from curl http -a '{user}:{password}' :6969/statistics
 651
 652                     HTTP/1.1 200 OK
 653                     Content-Length: XXX
 654                     Content-Type: application/json
 655                     Date: Tue, 17 Apr 2018 10:51:14 GMT
 656                     Server: Jetty(9.3.20.v20170531)
 657                     {
 658                          "code":200,
 659                          "distributions":10,
 660                          "distribution_complete_ok":8,
 661                          "distribution_complete_fail":2,
 662                          "downloads":15,
 663                          "downloads_ok"; 10,
 664                          "downloads_error": 5
 665                     }
 666