1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
2 .. http://creativecommons.org/licenses/by/4.0
15 The PDP-D Core Engine provides an infrastructure and services for `drools <https://www.drools.org/>`__ based applications
16 in the context of Policies and ONAP.
18 A PDP-D supports applications by means of *controllers*. A *controller* is a named
19 grouping of resources. These typically include references to communication endpoints,
20 maven artifact coordinates, and *coders* for message mapping.
22 *Controllers* use *communication endpoints* to interact
23 with remote networked entities typically using messaging (dmaap or ueb),
26 PDP-D Engine capabilities can be extended via *features*. Integration with other
27 Policy Framework components (API, PAP, and PDP-X) is through one of them (*feature-lifecycle*).
29 The PDP-D Engine infrastructure provides mechanisms for data migration, diagnostics, and application management.
34 Source Code repositories
35 ~~~~~~~~~~~~~~~~~~~~~~~~
37 The PDP-D software is mainly located in the `policy/drools repository <https://git.onap.org/policy/drools-pdp>`__ with the *communication endpoints* software residing in the `policy/common repository <https://git.onap.org/policy/common>`__ and Tosca policy models in the `policy/models repository <https://git.onap.org/policy/models>`__.
42 Check the *drools-pdp* `released versions <https://wiki.onap.org/display/DW/Policy+Framework+Project%3A+Component+Versions>`__ page for the latest versions.
43 At the time of this writing *1.8.2* is the latest version.
47 docker pull onap/policy-drools:1.8.2
49 A container instantiated from this image will run under the non-priviledged *policy* account.
51 The PDP-D root directory is located at the */opt/app/policy* directory (or *$POLICY_HOME*), with the
52 exception of the *$HOME/.m2* which contains the local maven repository.
53 The PDP-D configuration resides in the following directories:
55 - **/opt/app/policy/config**: (*$POLICY_HOME/config* or *$POLICY_CONFIG*) contains *engine*, *controllers*, and *endpoint* configuration.
56 - **/home/policy/.m2**: (*$HOME/.m2*) maven repository configuration.
57 - **/opt/app/policy/etc/**: (*$POLICY_HOME/etc*) miscellaneous configuration such as certificate stores.
59 The following command can be used to explore the directory layout.
63 docker run --rm -it nexus3.onap.org:10001/onap/policy-drools:1.8.2 -- bash
65 Communication Endpoints
66 =======================
68 PDP-D supports the following networked infrastructures. This is also referred to as
69 *communication infrastructures* in the source code.
77 The source code is located at
78 `the policy-endpoints module <https://git.onap.org/policy/common/tree/policy-endpoints>`__
79 in the *policy/commons* repository.
81 These network resources are *named* and typically have a *global* scope, therefore typically visible to
82 the PDP-D engine (for administration purposes), application *controllers*,
85 DMaaP, UEB, and NOOP are message-based communication infrastructures, hence the terminology of
86 source and sinks, to denote their directionality into or out of the *controller*, respectively.
88 An endpoint can either be *managed* or *unmanaged*. The default for an endpoint is to be *managed*,
89 meaning that they are globally accessible by name, and managed by the PDP-D engine.
90 *Unmanaged* topics are used when neither global visibility, or centralized PDP-D management is desired.
91 The software that uses *unmanaged* topics is responsible for their lifecycle management.
96 These are messaging enpoints that use DMaaP as the communication infrastructure.
98 Typically, a *managed* endpoint configuration is stored in the *<topic-name>-topic.properties* files.
101 `DCAE_TOPIC-topic.properties <https://git.onap.org/policy/drools-applications/tree/controlloop/common/feature-controlloop-management/src/main/feature/config/DCAE_TOPIC-topic.properties>`__ is defined as
105 dmaap.source.topics=DCAE_TOPIC
107 dmaap.source.topics.DCAE_TOPIC.effectiveTopic=${env:DCAE_TOPIC}
108 dmaap.source.topics.DCAE_TOPIC.servers=${env:DMAAP_SERVERS}
109 dmaap.source.topics.DCAE_TOPIC.consumerGroup=${env:DCAE_CONSUMER_GROUP}
110 dmaap.source.topics.DCAE_TOPIC.https=true
112 In this example, the generic name of the *source* endpoint
113 is *DCAE_TOPIC*. This is known as the *canonical* name.
114 The actual *topic* used in communication exchanges in a physical lab is contained
115 in the *$DCAE_TOPIC* environment variable. This environment variable is usually
116 set up by *devops* on a per installation basis to meet the needs of each
119 In the previous example, *DCAE_TOPIC* is a source-only topic.
121 Sink topics are similarly specified but indicating that are sink endpoints
122 from the perspective of the *controller*. For example, the *APPC-CL* topic
127 dmaap.source.topics=APPC-CL
128 dmaap.sink.topics=APPC-CL
130 dmaap.source.topics.APPC-CL.servers=${env:DMAAP_SERVERS}
131 dmaap.source.topics.APPC-CL.https=true
133 dmaap.sink.topics.APPC-CL.servers=${env:DMAAP_SERVERS}
134 dmaap.sink.topics.APPC-CL.https=true
136 Although not shown in these examples, additional configuration options are available such as *user name*,
137 *password*, *security keys*, *consumer group* and *consumer instance*.
142 Similary, UEB endpoints are messaging endpoints, similar to the DMaaP ones.
145 `DCAE_TOPIC-topic.properties <https://git.onap.org/policy/drools-applications/tree/controlloop/common/feature-controlloop-management/src/main/feature/config/DCAE_TOPIC-topic.properties>`__ can be converted to an *UEB* one, by replacing the
146 *dmaap* prefix with *ueb*. For example:
150 ueb.source.topics=DCAE_TOPIC
152 ueb.source.topics.DCAE_TOPIC.effectiveTopic=${env:DCAE_TOPIC}
153 ueb.source.topics.DCAE_TOPIC.servers=${env:DMAAP_SERVERS}
154 ueb.source.topics.DCAE_TOPIC.consumerGroup=${env:DCAE_CONSUMER_GROUP}
155 ueb.source.topics.DCAE_TOPIC.https=true
160 NOOP (no-operation) endpoints are messaging endpoints that don't have any network attachments.
161 They are used for testing convenience.
163 `DCAE_TOPIC-topic.properties <https://git.onap.org/policy/drools-applications/tree/controlloop/common/feature-controlloop-management/src/main/feature/config/DCAE_TOPIC-topic.properties>`__ to a *NOOP* endpoint, simply replace the *dmaap* prefix with *noop*:
167 noop.source.topics=DCAE_TOPIC
168 noop.source.topics.DCAE_TOPIC.effectiveTopic=${env:DCAE_TOPIC}
173 HTTP Clients are typically stored in files following the naming convention: *<name>-http-client.properties* convention.
175 the `AAI HTTP Client <https://git.onap.org/policy/drools-applications/tree/controlloop/common/feature-controlloop-management/src/main/feature/config/AAI-http-client.properties>`__:
179 http.client.services=AAI
181 http.client.services.AAI.managed=true
182 http.client.services.AAI.https=true
183 http.client.services.AAI.host=${envd:AAI_HOST}
184 http.client.services.AAI.port=${envd:AAI_PORT}
185 http.client.services.AAI.userName=${envd:AAI_USERNAME}
186 http.client.services.AAI.password=${envd:AAI_PASSWORD}
187 http.client.services.AAI.contextUriPath=${envd:AAI_CONTEXT_URI}
192 HTTP Servers are stored in files that follow a similar naming convention *<name>-http-server.properties*.
193 The following is an example of a server named *CONFIG*, getting most of its configuration from
194 environment variables.
198 http.server.services=CONFIG
200 http.server.services.CONFIG.host=${envd:TELEMETRY_HOST}
201 http.server.services.CONFIG.port=7777
202 http.server.services.CONFIG.userName=${envd:TELEMETRY_USER}
203 http.server.services.CONFIG.password=${envd:TELEMETRY_PASSWORD}
204 http.server.services.CONFIG.restPackages=org.onap.policy.drools.server.restful
205 http.server.services.CONFIG.managed=false
206 http.server.services.CONFIG.swagger=true
207 http.server.services.CONFIG.https=true
208 http.server.services.CONFIG.aaf=${envd:AAF:false}
210 *Endpoints* configuration resides in the *$POLICY_HOME/config* (or *$POLICY_CONFIG*) directory in a container.
215 *Controllers* are the means for the PDP-D to run *applications*. Controllers are
216 defined in *<name>-controller.properties* files.
219 `usecases controller configuration <https://git.onap.org/policy/drools-applications/tree/controlloop/common/feature-controlloop-usecases/src/main/feature/config/usecases-controller.properties>`__.
221 This configuration file has two sections: *a)* application maven coordinates, and *b)* endpoint references and coders.
226 The coordinates section (*rules*) points to the *controller-usecases* *kjar* artifact.
227 It is the *brain* of the control loop application.
231 controller.name=usecases
233 rules.groupId=${project.groupId}
234 rules.artifactId=controller-usecases
235 rules.version=${project.version}
238 This *kjar* contains the
239 `usecases DRL <https://git.onap.org/policy/drools-applications/tree/controlloop/common/controller-usecases/src/main/resources/usecases.drl>`__ file (there may be more than one DRL file included).
244 rule "NEW.TOSCA.POLICY"
246 $policy : ToscaPolicy()
251 ControlLoopParams params = ControlLoopUtils.toControlLoopParams($policy);
252 if (params != null) {
258 The DRL in conjuction with the dependent java libraries in the kjar
259 `pom <https://git.onap.org/policy/drools-applications/tree/controlloop/common/controller-usecases/pom.xml>`__
260 realizes the application's function. For intance, it realizes the
261 vFirewall, vCPE, and vDNS use cases in ONAP.
267 <groupId>org.onap.policy.models.policy-models-interactions.model-actors</groupId>
268 <artifactId>actor.appclcm</artifactId>
269 <version>${policy.models.version}</version>
270 <scope>provided</scope>
274 Endpoints References and Coders
275 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
277 The *usecases-controller.properties* configuration also contains a mix of
278 source (of incoming controller traffic) and sink (of outgoing controller traffic)
279 configuration. This configuration also contains specific
280 filtering and mapping rules for incoming and outgoing dmaap messages
286 dmaap.source.topics=DCAE_TOPIC,APPC-CL,APPC-LCM-WRITE,SDNR-CL-RSP
287 dmaap.sink.topics=APPC-CL,APPC-LCM-READ,POLICY-CL-MGT,SDNR-CL,DCAE_CL_RSP
290 dmaap.source.topics.APPC-LCM-WRITE.events=org.onap.policy.appclcm.AppcLcmDmaapWrapper
291 dmaap.source.topics.APPC-LCM-WRITE.events.org.onap.policy.appclcm.AppcLcmDmaapWrapper.filter=[?($.type == 'response')]
292 dmaap.source.topics.APPC-LCM-WRITE.events.custom.gson=org.onap.policy.appclcm.util.Serialization,gson
294 dmaap.sink.topics.APPC-CL.events=org.onap.policy.appc.Request
295 dmaap.sink.topics.APPC-CL.events.custom.gson=org.onap.policy.appc.util.Serialization,gsonPretty
298 In this example, the *coders* specify that incoming messages over the DMaaP endpoint
299 reference *APPC-LCM-WRITE*, that have a field called *type* under the root JSON object with
300 value *response* are allowed into the *controller* application. In this case, the incoming
301 message is converted into an object (fact) of type *org.onap.policy.appclcm.AppcLcmDmaapWrapper*.
302 The *coder* has attached a custom implementation provided by the *application* with class
303 *org.onap.policy.appclcm.util.Serialization*. Note that the *coder* filter is expressed in JSONPath notation.
305 Note that not all the communication endpoint references need to be explicitly referenced within the
306 *controller* configuration file. For example, *Http clients* do not.
307 The reasons are historical, as the PDP-D was initially intended to only communicate
308 through messaging-based protocols such as UEB or DMaaP in asynchronous unidirectional mode.
309 The introduction of *Http* with synchronous bi-directional communication with remote endpoints made
310 it more convenient for the application to manage each network exchange.
312 *Controllers* configuration resides in the *$POLICY_HOME/config* (or *$POLICY_CONFIG*) directory in a container.
314 Other Configuration Files
315 ~~~~~~~~~~~~~~~~~~~~~~~~~
317 There are other types of configuration files that *controllers* can use, for example *.environment* files
318 that provides a means to share data across applications. The
319 `controlloop.properties.environment <https://git.onap.org/policy/drools-applications/tree/controlloop/common/feature-controlloop-management/src/main/feature/config/controlloop.properties.environment>`__ is one such example.
325 PDP-D supports Tosca Policies through the *feature-lifecycle*. The *PDP-D* receives its policy set
326 from the *PAP*. A policy conforms to its Policy Type specification.
327 Policy Types and policy creation is done by the *API* component.
328 Policy deployments are orchestrated by the *PAP*.
330 All communication between *PAP* and PDP-D is over the DMaaP *POLICY-PDP-PAP* topic.
335 The PDP-D Engine supports two (native) Tosca policy types by means of the *lifecycle*
338 - *onap.policies.native.drools.Controller*
339 - *onap.policies.native.drools.Artifact*
341 These types can be used to dynamically deploy or undeploy application *controllers*,
342 assign policy types, and upgrade or downgrade their attached maven artifact versions.
345 `example native controller <https://git.onap.org/policy/drools-pdp/tree/feature-lifecycle/src/test/resources/tosca-policy-native-controller-example.json>`__ policy is shown below.
350 "tosca_definitions_version": "tosca_simple_yaml_1_0_0",
351 "topology_template": {
354 "example.controller": {
355 "type": "onap.policies.native.drools.Controller",
356 "type_version": "1.0.0",
358 "name": "example.controller",
360 "policy-id": "example.controller"
363 "controllerName": "lifecycle",
366 "topicName": "DCAE_TOPIC",
369 "eventClass": "java.util.HashMap",
370 "eventFilter": "[?($.closedLoopEventStatus == 'ONSET')]"
373 "eventClass": "java.util.HashMap",
374 "eventFilter": "[?($.closedLoopEventStatus == 'ABATED')]"
381 "topicName": "APPC-CL",
384 "eventClass": "java.util.HashMap",
385 "eventFilter": "[?($.CommonHeader && $.Status)]"
400 The actual application coordinates are provided with a policy of type onap.policies.native.drools.Artifact,
401 see the `example native artifact <https://git.onap.org/policy/drools-pdp/tree/feature-lifecycle/src/test/resources/tosca-policy-native-artifact-example.json>`__
406 "tosca_definitions_version": "tosca_simple_yaml_1_0_0",
407 "topology_template": {
410 "example.artifact": {
411 "type": "onap.policies.native.drools.Artifact",
412 "type_version": "1.0.0",
414 "name": "example.artifact",
416 "policy-id": "example.artifact"
420 "groupId": "org.onap.policy.drools.test",
421 "artifactId": "lifecycle",
434 Operational Policy Types
435 ~~~~~~~~~~~~~~~~~~~~~~~~
437 The PDP-D also recognizes Tosca Operational Policies, although it needs an
438 application *controller* that understands them to execute them. These are:
440 - *onap.policies.controlloop.operational.common.Drools*
442 A minimum of one application *controller* that supports these capabilities
443 must be installed in order to honor the *operational policy types*.
444 One such controller is the *usecases* controller residing in the
445 `policy/drools-applications <https://git.onap.org/policy/drools-applications>`__
448 Controller Policy Type Support
449 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
451 Note that a *controller* may support other policy types. A controller may declare them
452 explicitly in a native *onap.policies.native.drools.Controller* policy.
457 "controller.policy.types" : "policy.type.A"
460 The *controller* application could declare its supported policy types in the *kjar*.
461 For example, the *usecases controller* packages this information in the
462 `kmodule.xml <https://git.onap.org/policy/drools-applications/tree/controlloop/common/controller-usecases/src/main/resources/META-INF/kmodule.xml>`__. One advantage of this approach is that the PDP-D would only
463 commit to execute policies against these policy types if a supporting controller is up and running.
467 <kmodule xmlns="http://jboss.org/kie/6.0.0/kmodule">
468 <kbase name="onap.policies.controlloop.operational.common.Drools" default="false" equalsBehavior="equality"/>
469 <kbase name="onap.policies.controlloop.Operational" equalsBehavior="equality"
470 packages="org.onap.policy.controlloop" includes="onap.policies.controlloop.operational.common.Drools">
471 <ksession name="usecases"/>
475 Software Architecture
476 ======================
478 PDP-D is divided into 2 layers:
480 - core (`policy-core <https://git.onap.org/policy/drools-pdp/tree/policy-core>`__)
481 - management (`policy-management <https://git.onap.org/policy/drools-pdp/tree/policy-management>`__)
486 The core layer directly interfaces with the *drools* libraries with 2 main abstractions:
488 * `PolicyContainer <https://git.onap.org/policy/drools-pdp/tree/policy-core/src/main/java/org/onap/policy/drools/core/PolicyContainer.java>`__, and
489 * `PolicySession <https://git.onap.org/policy/drools-pdp/tree/policy-core/src/main/java/org/onap/policy/drools/core/PolicySession.java>`__.
491 Policy Container and Sessions
492 """""""""""""""""""""""""""""
494 The *PolicyContainer* abstracts the drools *KieContainer*, while a *PolicySession* abstracts a drools *KieSession*.
495 PDP-D uses stateful sessions in active mode (*fireUntilHalt*) (please visit the `drools <https://www.drools.org/>`__
496 website for additional documentation).
501 The management layer manages the PDP-D and builds on top of the *core* capabilities.
506 The PDP-D `PolicyEngine <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/java/org/onap/policy/drools/system/PolicyEngine.java>`__ is the top abstraction and abstracts away the PDP-D and all the
507 resources it holds. The reader looking at the source code can start looking at this component
508 in a top-down fashion. Note that the *PolicyEngine* abstraction should not be confused with the
509 sofware in the *policy/engine* repository, there is no relationship whatsoever other than in the naming.
511 The *PolicyEngine* represents the PDP-D, holds all PDP-D resources, and orchestrates activities among those.
513 The *PolicyEngine* manages applications via the `PolicyController <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/java/org/onap/policy/drools/system/PolicyController.java>`__ abstractions in the base code. The
514 relationship between the *PolicyEngine* and *PolicyController* is one to many.
516 The *PolicyEngine* holds other global resources such as a *thread pool*, *policies validator*, *telemetry* server,
517 and *unmanaged* topics for administration purposes.
519 The *PolicyEngine* has interception points that allow
520 `*features* <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/java/org/onap/policy/drools/features/PolicyEngineFeatureApi.java>`__
521 to observe and alter the default *PolicyEngine* behavior.
523 The *PolicyEngine* implements the `*Startable* <https://git.onap.org/policy/common/tree/capabilities/src/main/java/org/onap/policy/common/capabilities/Startable.java>`__ and `*Lockable* <https://git.onap.org/policy/common/tree/capabilities/src/main/java/org/onap/policy/common/capabilities/Lockable.java>`__ interfaces. These operations
524 have a cascading effect on the resources the *PolicyEngine* holds, as it is the top level entity, thus
525 affecting *controllers* and *endpoints*. These capabilities are intended to be used for extensions,
526 for example active/standby multi-node capabilities. This programmability is
527 exposed via the *telemetry* API, and *feature* hooks.
532 *PolicyEngine* related configuration is located in the
533 `engine.properties <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/server/config/engine.properties>`__,
534 and `engine-system.properties <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/server/config/engine.properties>`__.
536 The *engine* configuration files reside in the *$POLICY_CONFIG* directory.
541 A *PolicyController* represents an application. Each *PolicyController* has an instance of a
542 `DroolsController <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/java/org/onap/policy/drools/system/PolicyController.java>`__. The *PolicyController* provides the means to group application specific resources
543 into a single unit. Such resources include the application's *maven coordinates*, *endpoint references*, and *coders*.
545 A *PolicyController* uses a
546 `DroolsController <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/java/org/onap/policy/drools/controller/DroolsController.java>`__ to interface with the *core* layer (*PolicyContainer* and *PolicySession*).
548 The relationship between the *PolicyController* and the *DroolsController* is one-to-one.
549 The *DroolsController* currently supports 2 implementations, the
550 `MavenDroolsController <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/java/org/onap/policy/drools/controller/internal/MavenDroolsController.java>`__, and the
551 `NullDroolsController <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/java/org/onap/policy/drools/controller/internal/NullDroolsController.java>`__.
552 The *DroolsController*'s polymorphic behavior depends on whether a maven artifact is attached to the controller or not.
557 The *controllers* configuration resides in the *$POLICY_CONFIG* directory.
562 PDP-D is programmable through:
564 - Features and Event Listeners.
565 - Maven-Drools applications.
567 Using Features and Listeners
568 """"""""""""""""""""""""""""
570 Features hook into the interception points provided by the the *PDP-D* main entities.
572 *Endpoint Listeners*, see `here <https://git.onap.org/policy/common/tree/policy-endpoints/src/main/java/org/onap/policy/common/endpoints/event/comm/TopicListener.java>`__
573 and `here <https://git.onap.org/policy/common/tree/policy-endpoints/src/main/java/org/onap/policy/common/endpoints/listeners>`__, can be used in conjuction with features for additional capabilities.
575 Using Maven-Drools applications
576 """""""""""""""""""""""""""""""
578 Maven-based drools applications can run any arbitrary functionality structured with rules and java logic.
583 Whenever possible it is suggested that PDP-D related operations flow through the
584 *PolicyEngine* downwards in a top-down manner. This imposed order implies that
585 all the feature hooks are always invoked in a deterministic fashion. It is also
586 a good mechanism to safeguard against deadlocks.
591 It is recommended to *features* (extensions) to offer a diagnostics REST API
592 to integrate with the telemetry API. This is done by placing JAX-RS files under
593 the package *org.onap.policy.drools.server.restful*. The root context path
594 for all the telemetry services is */policy/pdp/engine*.
599 *Features* is an extension mechanism for the PDP-D functionality.
600 Features can be toggled on and off.
601 A feature is composed of:
604 - Scripts and configuration files.
609 Additional functionality can be provided in the form of java libraries that hook into the
610 *PolicyEngine*, *PolicyController*, *DroolsController*, and *PolicySession* interception
611 points to observe or alter the PDP-D logic.
613 See the Feature APIs available in the
614 `management <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/java/org/onap/policy/drools/features>`__
616 `core <https://git.onap.org/policy/drools-pdp/tree/policy-core/src/main/java/org/onap/policy/drools/core/PolicySessionFeatureApi.java>`__ layers.
618 The convention used for naming these extension modules are *api-<name>* for interfaces,
619 and *feature-<name>* for the actual java extensions.
624 Installation items such as scripts, SQL, maven artifacts, and configuration files.
626 The reader can refer to the `policy/drools-pdp repository <https://git.onap.org/policy/drools-pdp>`__
627 and the <https://git.onap.org/policy/drools-applications>`__ repository for miscellaneous feature
633 A feature is packaged in a *feature-<name>.zip* and has this internal layout:
637 # #######################################################################################
638 # Features Directory Layout:
642 # L─ <feature-name>*/
643 #    L─ [config]/
644 #    |  L─ <config-file>+
646 #    |  L─ <bin-file>+
648 #    | L─ [dependencies]/
649 #    | | L─ <dependent-jar>+
650 #    │ L─ feature/
651 #    │ L─ <feature-jar>
653 #    │  L─ <db-name>/+
654 #    │ L─ sql/
655 #    │ L─ <sql-scripts>*
656 #    L─ [artifacts]/
657 #     L─ <artifact>+
658 #    L─ [install]
659 #     L─ [enable]
660 #     L─ [disable]
661 #     L─ [other-directories-or-files]
663 # notes: [] = optional , * = 0 or more , + = 1 or more
664 # <feature-name> directory without "feature-" prefix.
665 # [config] feature configuration directory that contains all configuration
666 # needed for this features
667 # [config]/<config-file> preferably named with "feature-<feature-name>" prefix to
668 # precisely match it against the exact features, source code, and
669 # associated wiki page for configuration details.
670 # [bin] feature bin directory that contains helper scripts for this feature
671 # [bin]/<executable-file> preferably named with "feature-<feature-name>" prefix.
672 # lib jar libraries needed by this features
673 # lib/[dependencies] 3rd party jar dependencies not provided by base installation
674 # of pdp-d that are necessary for <feature-name> to operate
676 # lib/feature the single feature jar that implements the feature.
677 # [db] database directory, if the feature contains sql.
678 # [db]/<db-name> database to which underlying sql scripts should be applied.
679 # ideally, <db-name> = <feature-name> so it is easily to associate
680 # the db data with a feature itself. In addition, since a feature is
681 # a somewhat independent isolated unit of functionality,the <db-name>
682 # database ideally isolates all its data.
683 # [db]/<db-name>/sql directory with all the sql scripts.
684 # [db]/<db-name>/sql/<sql-scripts> for this feature, sql
685 # upgrade scripts should be suffixed with ".upgrade.sql"
686 # and downgrade scripts should be suffixed with ".downgrade.sql"
687 # [artifacts] maven artifacts to be deployed in a maven repository.
688 # [artifacts]/<artifact> maven artifact with identifiable maven coordinates embedded
690 # [install] custom installation directory where custom enable or disable scripts
691 # and other free form data is included to be used for the enable and
692 # and disable scripts.
693 # [install]/[enable] enable script executed when the enable operation is invoked in
695 # [install]/[disable] disable script executed when the disable operation is invoked in
697 # [install]/[other-directories-or-files] other executables, or data that can be used
698 # by the feature for any of its operations. The content is determined
699 # by the feature designer.
700 # ########################################################################################
702 The `features <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/server-gen/bin/features>`__
703 is the tool used for administration purposes:
707 Usage: features status
708 Get enabled/disabled status on all features
709 features enable <feature> ...
710 Enable the specified feature
711 features disable <feature> ...
712 Disable the specified feature
713 features install [ <feature> | <file-name> ] ...
714 Install the specified feature
715 features uninstall <feature> ...
716 Uninstall the specified feature
718 Features available in the Docker image
719 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
721 The only enabled feature in the *onap/policy-drools* image is:
723 - **lifecycle**: enables the lifecycle capability to integrate with the Policy Framework components.
725 The following features are included in the image but disabled.
727 - **distributed locking**: distributed resource locking.
728 - **healthcheck**: basic PDP-D Engine healthcheck.
733 The Healthcheck feature provides reports used to verify the health of *PolicyEngine.manager* in addition to the construction, operation, and deconstruction of HTTP server/client objects.
735 When enabled, the feature takes as input a properties file named "*feature-healtcheck.properties*.
736 This file should contain configuration properties necessary for the construction of HTTP client and server objects.
738 Upon initialization, the feature first constructs HTTP server and client objects using the properties
739 from its properties file. A healthCheck operation is then triggered. The logic of the healthCheck verifies
740 that *PolicyEngine.manager* is alive, and iteratively tests each HTTP server object by sending HTTP GET
741 requests using its respective client object. If a server returns a "200 OK" message, it is marked as "healthy"
742 in its individual report. Any other return code results in an "unhealthy" report.
744 After the testing of the server objects has completed, the feature returns a single consolidated report.
749 The "lifecycle" feature enables a PDP-D to work with the architectural framework introduced in the
752 The lifecycle feature maintains three states: TERMINATED, PASSIVE, and ACTIVE.
753 The PAP interacts with the lifecycle feature to put a PDP-D in PASSIVE or ACTIVE states.
754 The PASSIVE state allows for Tosca Operational policies to be deployed.
755 Policy execution is enabled when the PDP-D transitions to the ACTIVE state.
757 This feature can coexist side by side with the legacy mode of operation that pre-dates the Dublin release.
762 The Distributed Locking Feature provides locking of resources across a pool of PDP-D hosts.
763 The list of locks is maintained in a database, where each record includes a resource identifier,
764 an owner identifier, and an expiration time. Typically, a drools application will unlock the resource
765 when it's operation completes. However, if it fails to do so, then the resource will be automatically
766 released when the lock expires, thus preventing a resource from becoming permanently locked.
771 The following features have been contributed to the *policy/drools-pdp* but are either
772 unnecessary or have not been thoroughly tested:
777 feature_activestdbymgmt.rst
778 feature_controllerlogging.rst
780 feature_mdcfilters.rst
782 feature_sesspersist.rst
783 feature_statemgmt.rst
784 feature_testtransaction.rst
785 feature_nolocking.rst
790 PDP-D data is migrated across releases with the
791 `db-migrator <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/server-gen/bin/db-migrator>`__.
793 The migration occurs when different release data is detected. *db-migrator* will look under the
794 *$POLICY_HOME/etc/db/migration* for databases and SQL scripts to migrate.
798 $POLICY_HOME/etc/db/migration/<schema-name>/sql/<sql-file>
800 where *<sql-file>* is of the form:
804 <VERSION>-<pdp|feature-name>[-description](.upgrade|.downgrade).sql
806 The db-migrator tool syntax is
814 [-t <target-version>]
817 where <operations>=upgrade|downgrade|auto|version|erase|report
819 Configuration Options:
820 -s|--schema|--database: schema to operate on ('ALL' to apply on all)
821 -b|--basedir: overrides base DB migration directory
822 -f|--from: overrides current release version for operations
823 -t|--target: overrides target release to upgrade/downgrade
826 upgrade: upgrade operation
827 downgrade: performs a downgrade operation
828 auto: autonomous operation, determines upgrade or downgrade
829 version: returns current version, and in conjunction if '-f' sets the current version
830 erase: erase all data related <schema> (use with care)
831 report: migration detailed report on an schema
832 ok: is the migration status valid
835 `feature-distributed-locking sql directory <https://git.onap.org/policy/drools-pdp/tree/feature-distributed-locking/src/main/feature/db/pooling/sql>`__
836 for an example of upgrade/downgrade scripts.
838 The following command will provide a report on the upgrade or downgrade activies:
842 db-migrator -s ALL -o report
844 For example in the official guilin delivery:
848 policy@dev-drools-0:/tmp/policy-install$ db-migrator -s ALL -o report
849 +---------+---------+
851 +---------+---------+
853 +---------+---------+
854 +-------------------------------------+-----------+---------+---------------------+
855 | script | operation | success | atTime |
856 +-------------------------------------+-----------+---------+---------------------+
857 | 1804-distributedlocking.upgrade.sql | upgrade | 1 | 2020-05-22 19:33:09 |
858 | 1811-distributedlocking.upgrade.sql | upgrade | 1 | 2020-05-22 19:33:09 |
859 +-------------------------------------+-----------+---------+---------------------+
861 In order to use the *db-migrator* tool, the system must be configured with a database.
870 The drools libraries in the PDP-D uses maven to fetch rules artifacts and software dependencies.
872 The default *settings.xml* file specifies the repositories to search. This configuration
873 can be overriden with a custom copy that would sit in a mounted configuration
874 directory. See an example of the OOM override
875 `settings.xml <https://git.onap.org/oom/tree/kubernetes/policy/charts/drools/resources/configmaps/settings.xml>`__.
877 The default ONAP installation of the *control loop* child image *onap/policy-pdpd-cl:1.6.4* is *OFFLINE*.
878 In this configuration, the *rules* artifact and the *dependencies* retrieves all the artifacts from the local
879 maven repository. Of course, this requires that the maven dependencies are preloaded in the local
880 repository for it to work.
882 An offline configuration requires two items:
884 - *OFFLINE* environment variable set to true.
885 - override *settings.xml* customization, see
886 `settings.xml <https://git.onap.org/oom/tree/kubernetes/policy/charts/drools/resources/configmaps/settings.xml>`__.
888 The default mode in the *onap/policy-drools:1.6.3* is ONLINE instead.
890 In *ONLINE* mode, the *controller* initialization can take a significant amount of time.
892 The Policy ONAP installation includes a *nexus* repository component that can be used to host any arbitrary
893 artifacts that an PDP-D application may require.
894 The following environment variables configure its location:
898 SNAPSHOT_REPOSITORY_ID=policy-nexus-snapshots
899 SNAPSHOT_REPOSITORY_URL=http://nexus:8080/nexus/content/repositories/snapshots/
900 RELEASE_REPOSITORY_ID=policy-nexus-releases
901 RELEASE_REPOSITORY_URL=http://nexus:8080/nexus/content/repositories/releases/
902 REPOSITORY_OFFLINE=false
904 The *deploy-artifact* tool is used to deploy artifacts to the local or remote maven repositories.
905 It also allows for dependencies to be installed locally. The *features* tool invokes it when artifacts are
906 to be deployed as part of a feature. The tool can be useful for developers to test a new application
911 syntax: deploy-artifact
917 -f|--file-repo: deploy in the file repository
918 -l|--local-repo: install in the local repository
919 -d|--dependencies: install dependencies in the local repository
920 -s|--settings: custom settings.xml
921 -a|--artifact: file artifact (jar or pom) to deploy and/or install
926 Policy can talk to AAF for authorization requests. To enable AAF set
927 the following environment variables:
932 AAF_NAMESPACE=org.onap.policy
933 AAF_HOST=aaf-locate.onap
935 By default AAF is disabled.
940 The *policy* tool can be used to stop, start, and provide status on the PDP-D.
944 syntax: policy [--debug] status|start|stop
946 The *status* option provides generic status of the system.
950 [drools-pdp-controllers]
951 L []: Policy Management (pid 408) is running
952 0 cron jobs installed.
957 healthcheck 1.6.3 enabled
958 distributed-locking 1.6.3 enabled
959 lifecycle 1.6.3 enabled
960 controlloop-management 1.6.4 enabled
961 controlloop-utils 1.6.4 enabled
962 controlloop-trans 1.6.4 enabled
963 controlloop-usecases 1.6.4 enabled
968 It contains 3 sections:
970 - *PDP-D* running status
972 - Data migration status on a per database basis.
974 The *start* and *stop* commands are useful for developers testing functionality on a docker container instance.
979 *PDP-D* offers an ample set of REST APIs to debug, introspect, and change state on a running PDP-D. This is known as the
980 *telemetry* API. The *telemetry* shell wraps these APIs for shell-like access using
981 `http-prompt <http://http-prompt.com/>`__.
985 policy@dev-drools-0:~$ telemetry
987 https://localhost:9696/policy/pdp/engine> get controllers
990 Content-Type: application/json
991 Date: Thu, 04 Jun 2020 01:07:38 GMT
992 Server: Jetty(9.4.24.v20191120)
998 https://localhost:9696/policy/pdp/engine> exit
1000 policy@dev-drools-0:~$
1006 Refer to the *$POLICY_HOME/bin/* directory for additional tooling.
1008 PDP-D Docker Container Configuration
1009 ====================================
1011 Both the PDP-D *onap/policy-drools* and *onap/policy-pdpd-cl* images can be used without other components.
1013 There are 2 types of configuration data provided to the container:
1015 1. environment variables.
1016 2. configuration files and shell scripts.
1018 Environment variables
1019 ~~~~~~~~~~~~~~~~~~~~~
1021 As it was shown in the *controller* and *endpoint* sections, PDP-D configuration can rely
1022 on environment variables. In a container environment, these variables are set up by the user
1023 in the host environment.
1025 Configuration Files and Shell Scripts
1026 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1028 PDP-D is very flexible in its configuration.
1030 The following file types are recognized when mounted under */tmp/policy-install/config*.
1032 These are the configuration items that can reside externally and override the default configuration:
1034 - **settings.xml** if working with external nexus repositories.
1035 - **standalone-settings.xml** if an external *policy* nexus repository is not available.
1036 - ***.conf** files containing environment variables. This is an alternative to use environment variables,
1037 as these files will be sourced in before the PDP-D starts.
1038 - **features*.zip** to load any arbitrary feature not present in the image.
1039 - ***.pre.sh** scripts that will be executed before the PDP-D starts.
1040 - ***.post.sh** scripts that will be executed after the PDP-D starts.
1041 - **policy-keystore** to override the default PDP-D java keystore.
1042 - **policy-truststore** to override the default PDP-D java truststore.
1043 - **aaf-cadi.keyfile** to override the default AAF CADI Key generated by AAF.
1044 - ***.properties** to override or add any properties file for the PDP-D, this includes *controller*, *endpoint*,
1045 *engine* or *system* configurations.
1046 - **logback*.xml** to override the default logging configuration.
1047 - ***.xml** to override other .xml configuration that may be used for example by an *application*.
1048 - ***.json** *json* configuration that may be used by an *application*.
1051 Running PDP-D with a single container
1052 =====================================
1057 First create an environment file (in this example *env.conf*) to configure the PDP-D.
1059 .. code-block:: bash
1061 # SYSTEM software configuration
1063 POLICY_HOME=/opt/app/policy
1064 POLICY_LOGS=/var/log/onap/policy/pdpd
1065 KEYSTORE_PASSWD=Pol1cy_0nap
1066 TRUSTSTORE_PASSWD=Pol1cy_0nap
1068 # Telemetry credentials
1071 TELEMETRY_HOST=0.0.0.0
1072 TELEMETRY_USER=demo@people.osaaf.org
1073 TELEMETRY_PASSWORD=demo123456!
1077 SNAPSHOT_REPOSITORY_ID=
1078 SNAPSHOT_REPOSITORY_URL=
1079 RELEASE_REPOSITORY_ID=
1080 RELEASE_REPOSITORY_URL=
1081 REPOSITORY_USERNAME=
1082 REPOSITORY_PASSWORD=
1083 REPOSITORY_OFFLINE=true
1085 # Relational (SQL) DB access
1094 AAF_NAMESPACE=org.onap.policy
1095 AAF_HOST=aaf.api.simpledemo.onap.org
1097 # PDP-D DMaaP configuration channel
1099 PDPD_CONFIGURATION_TOPIC=PDPD-CONFIGURATION
1100 PDPD_CONFIGURATION_API_KEY=
1101 PDPD_CONFIGURATION_API_SECRET=
1102 PDPD_CONFIGURATION_CONSUMER_GROUP=
1103 PDPD_CONFIGURATION_CONSUMER_INSTANCE=
1104 PDPD_CONFIGURATION_PARTITION_KEY=
1106 # PAP-PDP configuration channel
1108 POLICY_PDP_PAP_TOPIC=POLICY-PDP-PAP
1109 POLICY_PDP_PAP_API_KEY=
1110 POLICY_PDP_PAP_API_SECRET=
1114 DMAAP_SERVERS=localhost
1116 Note that *SQL_HOST*, and *REPOSITORY* are empty, so the PDP-D does not attempt
1117 to integrate with those components.
1122 In order to avoid the noise in the logs that relate to dmaap configuration, a startup script (*noop.pre.sh*) is added
1123 to convert *dmaap* endpoints to *noop* in the host directory to be mounted.
1128 .. code-block:: bash
1132 sed -i "s/^dmaap/noop/g" $POLICY_HOME/config/*.properties
1138 To put the controller directly in active mode at initialization, place an *active.post.sh* script under the
1139 mounted host directory:
1141 .. code-block:: bash
1145 bash -c "http --verify=no -a ${TELEMETRY_USER}:${TELEMETRY_PASSWORD} PUT https://localhost:9696/policy/pdp/engine/lifecycle/state/ACTIVE"
1150 .. code-block:: bash
1152 docker run --rm -p 9696:9696 -v ${PWD}/config:/tmp/policy-install/config --env-file ${PWD}/env/env.conf -it --name PDPD -h pdpd nexus3.onap.org:10001/onap/policy-drools:1.6.3
1154 To run the container in detached mode, add the *-d* flag.
1156 Note that in this command, we are opening the *9696* telemetry API port to the outside world, the config directory
1157 (where the *noop.pre.sh* customization script resides) is mounted as /tmp/policy-install/config,
1158 and the customization environment variables (*env/env.conf*) are passed into the container.
1160 To open a shell into the PDP-D:
1162 .. code-block:: bash
1164 docker exec -it pdp-d bash
1166 Once in the container, run tools such as *telemetry*, *db-migrator*, *policy* to look at the system state:
1168 To run the *telemetry shell* and other tools from the host:
1170 .. code-block:: bash
1172 docker exec -it PDPD bash -c "/opt/app/policy/bin/telemetry"
1173 docker exec -it PDPD bash -c "/opt/app/policy/bin/policy status"
1174 docker exec -it PDPD bash -c "/opt/app/policy/bin/db-migrator -s ALL -o report"
1176 Controlled instantiation of the PDP-D
1177 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1179 Sometimes a developer may want to start and stop the PDP-D manually:
1181 .. code-block:: bash
1185 docker run --rm -p 9696:9696 -v ${PWD}/config:/tmp/policy-install/config --env-file ${PWD}/env/env.conf -it --name PDPD -h pdpd nexus3.onap.org:10001/onap/policy-drools:1.6.3 bash
1187 # use this command to start policy applying host customizations from /tmp/policy-install/config
1189 pdpd-entrypoint.sh vmboot
1191 # or use this command to start policy without host customization
1195 # at any time use the following command to stop the PDP-D
1199 # and this command to start the PDP-D back again
1203 Running PDP-D with nexus and mariadb
1204 ====================================
1206 *docker-compose* can be used to test the PDP-D with other components. This is an example configuration
1207 that brings up *nexus*, *mariadb* and the PDP-D (*docker-compose-pdp.yml*)
1209 docker-compose-pdp.yml
1210 ~~~~~~~~~~~~~~~~~~~~~~
1212 .. code-block:: bash
1217 image: mariadb:10.2.25
1218 container_name: mariadb
1220 command: ['--lower-case-table-names=1', '--wait_timeout=28800']
1224 - ${PWD}/db:/docker-entrypoint-initdb.d
1228 image: sonatype/nexus:2.14.8-01
1229 container_name: nexus
1234 image: nexus3.onap.org:10001/onap/policy-drools:1.6.3
1235 container_name: drools
1243 - ${PWD}/config:/tmp/policy-install/config
1245 - ${PWD}/env/env.conf
1247 with *${PWD}/db/db.conf*:
1252 .. code-block:: bash
1254 MYSQL_ROOT_PASSWORD=secret
1255 MYSQL_USER=policy_user
1256 MYSQL_PASSWORD=policy_user
1258 and *${PWD}/db/db.sh*:
1263 .. code-block:: bash
1265 for db in support onap_sdk log migration operationshistory10 pooling policyadmin operationshistory
1267 mysql -uroot -p"${MYSQL_ROOT_PASSWORD}" --execute "CREATE DATABASE IF NOT EXISTS ${db};"
1268 mysql -uroot -p"${MYSQL_ROOT_PASSWORD}" --execute "GRANT ALL PRIVILEGES ON \`${db}\`.* TO '${MYSQL_USER}'@'%' ;"
1271 mysql -uroot -p"${MYSQL_ROOT_PASSWORD}" --execute "FLUSH PRIVILEGES;"
1276 The environment file *env/env.conf* for *PDP-D* can be set up with appropriate variables to point to the *nexus* instance
1277 and the *mariadb* database:
1279 .. code-block:: bash
1281 # SYSTEM software configuration
1283 POLICY_HOME=/opt/app/policy
1284 POLICY_LOGS=/var/log/onap/policy/pdpd
1285 KEYSTORE_PASSWD=Pol1cy_0nap
1286 TRUSTSTORE_PASSWD=Pol1cy_0nap
1288 # Telemetry credentials
1291 TELEMETRY_HOST=0.0.0.0
1292 TELEMETRY_USER=demo@people.osaaf.org
1293 TELEMETRY_PASSWORD=demo123456!
1297 SNAPSHOT_REPOSITORY_ID=policy-nexus-snapshots
1298 SNAPSHOT_REPOSITORY_URL=http://nexus:8081/nexus/content/repositories/snapshots/
1299 RELEASE_REPOSITORY_ID=policy-nexus-releases
1300 RELEASE_REPOSITORY_URL=http://nexus:8081/nexus/content/repositories/releases/
1301 REPOSITORY_USERNAME=admin
1302 REPOSITORY_PASSWORD=admin123
1303 REPOSITORY_OFFLINE=false
1305 MVN_SNAPSHOT_REPO_URL=https://nexus.onap.org/content/repositories/snapshots/
1306 MVN_RELEASE_REPO_URL=https://nexus.onap.org/content/repositories/releases/
1308 # Relational (SQL) DB access
1311 SQL_USER=policy_user
1312 SQL_PASSWORD=policy_user
1317 AAF_NAMESPACE=org.onap.policy
1318 AAF_HOST=aaf.api.simpledemo.onap.org
1320 # PDP-D DMaaP configuration channel
1322 PDPD_CONFIGURATION_TOPIC=PDPD-CONFIGURATION
1323 PDPD_CONFIGURATION_API_KEY=
1324 PDPD_CONFIGURATION_API_SECRET=
1325 PDPD_CONFIGURATION_CONSUMER_GROUP=
1326 PDPD_CONFIGURATION_CONSUMER_INSTANCE=
1327 PDPD_CONFIGURATION_PARTITION_KEY=
1329 # PAP-PDP configuration channel
1331 POLICY_PDP_PAP_TOPIC=POLICY-PDP-PAP
1332 POLICY_PDP_PAP_API_KEY=
1333 POLICY_PDP_PAP_API_SECRET=
1337 DMAAP_SERVERS=localhost
1342 A pre-start script *config/prepare.pres.sh"can be added the custom config directory
1343 to prepare the PDP-D to activate the distributed-locking feature (using the database)
1344 and to use "noop" topics instead of *dmaap* topics:
1346 .. code-block:: bash
1350 bash -c "/opt/app/policy/bin/features enable distributed-locking"
1351 sed -i "s/^dmaap/noop/g" $POLICY_HOME/config/*.properties
1356 A post-start script *config/active.post.sh* can place PDP-D in *active* mode at initialization:
1358 .. code-block:: bash
1360 bash -c "http --verify=no -a ${TELEMETRY_USER}:${TELEMETRY_PASSWORD} PUT https://localhost:9696/policy/pdp/engine/lifecycle/state/ACTIVE"
1362 Bring up the PDP-D, nexus, and mariadb
1363 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1365 To bring up the containers:
1367 .. code-block:: bash
1369 docker-compose -f docker-compose-pdpd.yaml up -d
1373 .. code-block:: bash
1375 docker-compose -f docker-compose-pdpd.yaml down -v
1380 The reader can also look at the `policy/docker repository <https://github.com/onap/policy-docker/tree/master/csit>`__.
1381 More specifically, these directories have examples of other PDP-D configurations:
1383 * `plans <https://github.com/onap/policy-docker/tree/master/csit/drools-pdp/plans>`__: startup & teardown scripts.
1384 * `scripts <https://github.com/onap/policy-docker/blob/master/csit/docker-compose-all.yml>`__: docker-compose file.
1385 * `tests <https://github.com/onap/policy-docker/tree/master/csit/drools-pdp/tests>`__: test plan.
1387 Configuring the PDP-D in an OOM Kubernetes installation
1388 =======================================================
1390 The `PDP-D OOM chart <https://github.com/onap/oom/tree/master/kubernetes/policy/components/policy-drools-pdp>`__ can be
1391 customized at the following locations:
1393 * `values.yaml <https://github.com/onap/oom/blob/master/kubernetes/policy/components/policy-drools-pdp/values.yaml>`__: custom values for your installation.
1394 * `configmaps <https://github.com/onap/oom/tree/master/kubernetes/policy/components/policy-drools-pdp/resources/configmaps>`__: place in this directory any configuration extensions or overrides to customize the PDP-D that does not contain sensitive information.
1395 * `secrets <https://github.com/onap/oom/tree/master/kubernetes/policy/components/policy-drools-pdp/resources/secrets>`__: place in this directory any configuration extensions or overrides to customize the PDP-D that does contain sensitive information.
1397 The same customization techniques described in the docker sections for PDP-D, fully apply here, by placing the corresponding
1398 files or scripts in these two directories.
1400 Additional information
1401 ======================
1403 For additional information, please see the
1404 `Drools PDP Development and Testing (In Depth) <https://wiki.onap.org/display/DW/2020+Frankfurt+Tutorials>`__ page.