Code Review / cps.git / blobdiff
? search:
summary | shortlog | log | commit | commitdiff | review | tree
raw | inline | side by side
Merge "Part 1: Refactor CPS Delta code to utility class"
[cps.git] / docs / deployment.rst
diff --git a/docs/deployment.rst b/docs/deployment.rst
index 7ba163d..31673d6 100644 (file)
--- a/docs/deployment.rst
+++ b/docs/deployment.rst
@@ -1,6 +1,6 @@
 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
 .. http://creativecommons.org/licenses/by/4.0
 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
 .. http://creativecommons.org/licenses/by/4.0
-.. Copyright (C) 2021-2022 Nordix Foundation
+.. Copyright (C) 2021-2025 OpenInfra Foundation Europe. All rights reserved.
 .. Modifications Copyright (C) 2021 Bell Canada.
 
 .. DO NOT CHANGE THIS LABEL FOR RELEASE NOTES - EVEN THOUGH IT GIVES A WARNING
 .. Modifications Copyright (C) 2021 Bell Canada.
 
 .. DO NOT CHANGE THIS LABEL FOR RELEASE NOTES - EVEN THOUGH IT GIVES A WARNING
@@ -12,6 +12,41 @@ CPS Deployment
 .. contents::
     :depth: 2
 
 .. contents::
     :depth: 2
 
+Database configuration
+======================
+CPS uses PostgreSQL database. As per the `PostgreSQL documentation on resource consumption
+<https://www.postgresql.org/docs/current/runtime-config-resource.html#GUC-SHARED-BUFFERS>`_, the *shared_buffers*
+parameter should be set between 25% and 40% of total memory. It has a default value of 128 megabytes, so this should be
+set appropriately. For example, given a database with 2GB of memory, 512MB is a recommended value.
+
+CPS and NCMP Configuration
+==========================
+
+CPU and Memory Requirements
+---------------------------
+
+The following are minimum requirements for NCMP:
+
+* For 20,000 CM-handles: 2 CPUs and 2 GB RAM per instance, with 70% heap allocation.
+* For 50,000 CM-handles: 3 CPUs and 3 GB RAM per instance, with 70% heap allocation.
+
+JVM Memory Allocation
+^^^^^^^^^^^^^^^^^^^^^
+
+When running with 2 GB or more memory per instance, allocating 70% of the JVM memory to the heap ensures efficient
+memory management. It is not recommended to go above 70%.
+
+.. code-block:: yaml
+
+    JAVA_TOOL_OPTIONS: "-XX:InitialRAMPercentage=70.0 -XX:MaxRAMPercentage=70.0"
+
+
+Load balancer configuration
+===========================
+
+For optimal performance in CPS/NCMP, load balancers should be configured to use a least-requests policy, also known as
+least-connected. Use of round-robin load balancing can lead to instability.
+
 CPS OOM Charts
 ==============
 The CPS kubernetes chart is located in the `OOM repository <https://github.com/onap/oom/tree/master/kubernetes/cps>`_.
 CPS OOM Charts
 ==============
 The CPS kubernetes chart is located in the `OOM repository <https://github.com/onap/oom/tree/master/kubernetes/cps>`_.
@@ -104,7 +139,7 @@ Each cps component can be restarted independently by issuing the following comma
 Credentials Retrieval
 =====================
 
 Credentials Retrieval
 =====================
 
-Application and database credentials are kept in Kubernetes secrets. They are defined as external secrets in the
+When using OOM application and database credentials are kept in Kubernetes secrets. They are defined as external secrets in the
 values.yaml file to be used across different components as :
 
 .. container:: ulist
 values.yaml file to be used across different components as :
 
 .. container:: ulist
@@ -154,150 +189,36 @@ To get a listing of the cps-core Pods, run the following command:
 .. note::
     The CPS Service will have to be restarted each time a change is made to a configurable property.
 
 .. note::
     The CPS Service will have to be restarted each time a change is made to a configurable property.
 
-Additional CPS-Core Customizations
-==================================
-
-The following table lists some properties that can be specified as Helm chart
-values to configure the application to be deployed. This list is not exhaustive.
-
-Any spring supported property can be configured by providing in ``config.additional.<spring-supported-property-name>: value`` Example: config.additional.spring.datasource.hikari.maximumPoolSize: 30
-
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| Property                                  | Description                                                                                             | Default Value                 |
-+===========================================+=========================================================================================================+===============================+
-| config.appUserName                        | User name used by cps-core service to configure the authentication for REST API it exposes.             | ``cpsuser``                   |
-|                                           |                                                                                                         |                               |
-|                                           | This is the user name to be used by cps-core REST clients to authenticate themselves.                   |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.appUserPassword                    | Password used by cps-core service to configure the authentication for REST API it exposes.              | Not defined                   |
-|                                           |                                                                                                         |                               |
-|                                           | If not defined, the password is generated when deploying the application.                               |                               |
-|                                           |                                                                                                         |                               |
-|                                           | See also :ref:`cps_common_credentials_retrieval`.                                                       |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| postgres.config.pgUserName                | Internal user name used by cps-core to connect to its own database.                                     | ``cps``                       |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| postgres.config.pgUserPassword            | Internal password used by cps-core to connect to its own database.                                      | Not defined                   |
-|                                           |                                                                                                         |                               |
-|                                           | If not defined, the password is generated when deploying the application.                               |                               |
-|                                           |                                                                                                         |                               |
-|                                           | See also :ref:`cps_common_credentials_retrieval`.                                                       |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| postgres.config.pgDatabase                | Database name used by cps-core                                                                          | ``cpsdb``                     |
-|                                           |                                                                                                         |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| logging.level                             | Logging level set in cps-core                                                                           | info                          |
-|                                           |                                                                                                         |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.useStrimziKafka                    | If targeting a custom kafka cluster, i.e. useStrimziKafka: false, the                                   | true                          |
-|                                           | config.eventPublisher.spring.kafka values below must be set.                                            |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.eventPublisher.                    | Kafka hostname and port                                                                                 | ``<kafka-bootstrap>:9092``    |
-| spring.kafka.bootstrap-servers            |                                                                                                         |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.eventPublisher.                    | Kafka consumer client id                                                                                | ``cps-core``                  |
-| spring.kafka.consumer.client-id           |                                                                                                         |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.eventPublisher.                    | Kafka security protocol.                                                                                | ``SASL_PLAINTEXT``            |
-| spring.kafka.security.protocol            | Some possible values are:                                                                               |                               |
-|                                           |                                                                                                         |                               |
-|                                           | * ``PLAINTEXT``                                                                                         |                               |
-|                                           | * ``SASL_PLAINTEXT``, for authentication                                                                |                               |
-|                                           | * ``SASL_SSL``, for authentication and encryption                                                       |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.eventPublisher.                    | Kafka security SASL mechanism. Required for SASL_PLAINTEXT and SASL_SSL protocols.                      | Not defined                   |
-| spring.kafka.properties.                  | Some possible values are:                                                                               |                               |
-| sasl.mechanism                            |                                                                                                         |                               |
-|                                           | * ``PLAIN``, for PLAINTEXT                                                                              |                               |
-|                                           | * ``SCRAM-SHA-512``, for SSL                                                                            |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.eventPublisher.                    | Kafka security SASL JAAS configuration. Required for SASL_PLAINTEXT and SASL_SSL protocols.             | Not defined                   |
-| spring.kafka.properties.                  | Some possible values are:                                                                               |                               |
-| sasl.jaas.config                          |                                                                                                         |                               |
-|                                           | * ``org.apache.kafka.common.security.plain.PlainLoginModule required username="..." password="...";``,  |                               |
-|                                           |   for PLAINTEXT                                                                                         |                               |
-|                                           | * ``org.apache.kafka.common.security.scram.ScramLoginModule required username="..." password="...";``,  |                               |
-|                                           |   for SSL                                                                                               |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.eventPublisher.                    | Kafka security SASL SSL store type. Required for SASL_SSL protocol.                                     | Not defined                   |
-| spring.kafka.ssl.trust-store-type         | Some possible values are:                                                                               |                               |
-|                                           |                                                                                                         |                               |
-|                                           | * ``JKS``                                                                                               |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.eventPublisher.                    | Kafka security SASL SSL store file location. Required for SASL_SSL protocol.                            | Not defined                   |
-| spring.kafka.ssl.trust-store-location     |                                                                                                         |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.eventPublisher.                    | Kafka security SASL SSL store password. Required for SASL_SSL protocol.                                 | Not defined                   |
-| spring.kafka.ssl.trust-store-password     |                                                                                                         |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.eventPublisher.                    | Kafka security SASL SSL broker hostname identification verification. Required for SASL_SSL protocol.    | Not defined                   |
-| spring.kafka.properties.                  | Possible value is:                                                                                      |                               |
-| ssl.endpoint.identification.algorithm     |                                                                                                         |                               |
-|                                           | * ``""``, empty string to disable                                                                       |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.additional.                        | Core pool size in asynchronous execution of notification.                                               | ``2``                         |
-| notification.async.executor.              |                                                                                                         |                               |
-| core-pool-size                            |                                                                                                         |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.additional.                        | Max pool size in asynchronous execution of notification.                                                | ``1``                         |
-| notification.async.executor.              |                                                                                                         |                               |
-| max-pool-size                             |                                                                                                         |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.additional.                        | Queue Capacity in asynchronous execution of notification.                                               | ``500``                       |
-| notification.async.executor.              |                                                                                                         |                               |
-| queue-capacity                            |                                                                                                         |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.additional.                        | If the executor should wait for the tasks to be completed on shutdown                                   | ``true``                      |
-| notification.async.executor.              |                                                                                                         |                               |
-| wait-for-tasks-to-complete-on-shutdown    |                                                                                                         |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.additional.                        | Prefix to be added to the thread name in asynchronous execution of notifications.                       | ``Async-``                    |
-| notification.async.executor.              |                                                                                                         |                               |
-| thread-name-prefix                        |                                                                                                         |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.additional.                        | Specifies number of database connections between database and application.                              | ``10``                        |
-| spring.datasource.hikari.                 | This property controls the maximum size that the pool is allowed to reach,                              |                               |
-| maximumPoolSize                           | including both idle and in-use connections.                                                             |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-
-.. _additional-cps-ncmp-customizations:
-
-Additional CPS-NCMP Customizations
-==================================
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.dmiPluginUserName                  | User name used by cps-core to authenticate themselves for using ncmp-dmi-plugin service.                | ``dmiuser``                   |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.dmiPluginUserPassword              | Internal password used by cps-core to connect to ncmp-dmi-plugin service.                               | Not defined                   |
-|                                           |                                                                                                         |                               |
-|                                           | If not defined, the password is generated when deploying the application.                               |                               |
-|                                           |                                                                                                         |                               |
-|                                           | See also :ref:`cps_common_credentials_retrieval`.                                                       |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.ncmp.timers                        | Specifies the delay in milliseconds in which the module sync watch dog will wake again after finishing. | ``30000``                     |
-| .advised-modules-sync.sleep-time-ms       |                                                                                                         |                               |
-|                                           |                                                                                                         |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.ncmp.timers                        | Specifies the delay in milliseconds in which the retry mechanism watch dog                              |                               |
-| .locked-modules-sync.sleep-time-ms        | will wake again after finishing.                                                                        | ``300000``                    |
-|                                           |                                                                                                         |                               |
-|                                           |                                                                                                         |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.ncmp.timers                        | Specifies the delay in milliseconds in which the data sync watch dog will wake again after finishing.   | ``30000``                     |
-| .cm-handle-data-sync.sleep-time-ms        |                                                                                                         |                               |
-|                                           |                                                                                                         |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.additional.ncmp.dmi.httpclient     | Specifies the maximum time in seconds, to wait for establishing a connection for the HTTP Client.       | ``180``                       |
-| .connectionTimeoutInSeconds               |                                                                                                         |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.additional.ncmp.dmi.httpclient     | Specifies the maximum number of connections allowed per route in the HTTP client.                       | ``50``                        |
-| .maximumConnectionsPerRoute               |                                                                                                         |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.additional.ncmp.dmi.httpclient     | Specifies the maximum total number of connections that can be held by the HTTP client.                  | ``100``                       |
-| .maximumConnectionsTotal                  |                                                                                                         |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
-| config.additional.ncmp.dmi.httpclient     | Specifies the duration in seconds for the threshold, after which idle connections will be evicted       | ``5``                         |
-| .idleConnectionEvictionThresholdInSeconds | from the connection pool by the HTTP client.                                                            |                               |
-+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
+.. _configuration-properties:
+
+Configuration Properties
+========================
+The following tables list properties that can be configured in the deployment. This list is not exhaustive.
+
+.. csv-table:: 3PP Properties
+   :file: csv/3pp_properties.csv
+   :widths: 20, 50, 30
+   :header-rows: 1
+
+.. note::
+    - The default datasource is defined as ``jdbc:postgresql://${DB_HOST:localhost}:${DB_PORT:5432}/cpsdb``. So it can also be configured using environment variables to just set the hostname ``DB_HOST`` and port ``DB_PORT``.
+    - The kafka bootstrap-servers can also be overridden with the environment variable ``KAFKA_BOOTSTRAP_SERVER``.
+
+.. csv-table:: Common CPS-NCMP Custom Properties
+   :file: csv/common_custom_properties.csv
+   :widths: 20, 50, 30
+   :header-rows: 1
+
+.. csv-table:: NCMP Custom Properties
+   :file: csv/ncmp_custom_properties.csv
+   :widths: 20, 50, 30
+   :header-rows: 1
+
+.. note::
+    - [app]:  can be ``policy-executor`` or ``dmi``.
+    - [services]: ``all-services`` for 'policy-executor'.
+    - [services]: ``data-services`` and 'model-services' for 'dmi'.
+    - All ncmp.policy-executor properties can also be overridden using environment variables: ``POLICY_SERVICE_ENABLED``, ``POLICY_SERVICE_DEFAULT_DECISION``, ``POLICY_SERVICE_URL``, ``POLICY_SERVICE_PORT``
 
 CPS-Core Docker Installation
 ============================
 
 CPS-Core Docker Installation
 ============================
@@ -308,32 +229,31 @@ The latest instructions are covered in the `README <https://github.com/onap/cps/
 .. Below Label is used by documentation for other CPS components to link here, do not remove even if it gives a warning
 .. _cps_common_distributed_datastructures:
 
 .. Below Label is used by documentation for other CPS components to link here, do not remove even if it gives a warning
 .. _cps_common_distributed_datastructures:
 
-CPS-Core and NCMP Distributed Datastructures
-============================================
+NCMP Distributed Data Structures
+================================
 
 
-CPS-Core and NCMP both internally uses embedded distributed datastructure to replicate the state across various instances for low latency.
-These instances require some additional ports to be available. The default range starts from 5701 and based on the number of instances configured they are incremented sequentially.
+NCMP utilizes embedded distributed data structures to replicate state across various instances, ensuring low latency and high performance. Each JVM runs a Hazelcast instance to manage these data structures. To facilitate member visibility and cluster formation, an additional port (defaulting to 5701) must be available.
 
 Below are the list of distributed datastructures that we have.
 
 
 Below are the list of distributed datastructures that we have.
 
-+--------------+---------------------------------+----------------------------------------------------------+
-| Component    | Datastructure name              |                 Use                                      |
-+==============+=================================+==========================================================+
-| cps-core     | anchorDataCache                 | Used to resolve prefix for the container name.           |
-+--------------+---------------------------------+----------------------------------------------------------+
-| cps-ncmp     | moduleSyncStartedOnCmHandles    | Watchdog process to register cmHandles.                  |
-+--------------+---------------------------------+----------------------------------------------------------+
-| cps-ncmp     | dataSyncSemaphores              | Watchdog process to sync data from the nodes.            |
-+--------------+---------------------------------+----------------------------------------------------------+
-| cps-ncmp     | moduleSyncWorkQueue             | Queue used internally for workers to pick the task.      |
-+--------------+---------------------------------+----------------------------------------------------------+
-| cps-ncmp     | untrustworthyCmHandlesSet       | Stores untrustworthy cmHandles whose TrustLevel is NONE. |
-+--------------+---------------------------------+----------------------------------------------------------+
-| cps-ncmp     | trustLevelPerDmiPlugin          | Stores the TrustLevel for the dmi-plugins.               |
-+--------------+---------------------------------+----------------------------------------------------------+
-| cps-ncmp     | moduleSetTagCacheMapConfig      | Stores the Module Set Tags for cmHandles.                |
-+--------------+---------------------------------+----------------------------------------------------------+
-| cps-ncmp     | cmSubscriptionEventCache        | Stores and tracks CmSubscription requests.               |
-+--------------+---------------------------------+----------------------------------------------------------+
++--------------+------------------------------------+-----------------------------------------------------------+
+| Component    | Data Structure Name                |                 Use                                       |
++==============+====================================+===========================================================+
+| cps-ncmp     | moduleSyncStartedOnCmHandles       | Watchdog process to register CM Handles.                  |
++--------------+------------------------------------+-----------------------------------------------------------+
+| cps-ncmp     | dataSyncSemaphores                 | Watchdog process to sync data from the nodes.             |
++--------------+------------------------------------+-----------------------------------------------------------+
+| cps-ncmp     | moduleSyncWorkQueue                | Queue used internally for workers to pick the task.       |
++--------------+------------------------------------+-----------------------------------------------------------+
+| cps-ncmp     | trustLevelPerCmHandle              | Stores the trust level per CM Handle id                   |
++--------------+------------------------------------+-----------------------------------------------------------+
+| cps-ncmp     | trustLevelPerDmiPlugin             | Stores the trust level for the dmi-plugins.               |
++--------------+------------------------------------+-----------------------------------------------------------+
+| cps-ncmp     | cmNotificationSubscriptionCache    | Stores and tracks cm notification subscription requests.  |
++--------------+------------------------------------+-----------------------------------------------------------+
+| cps-ncmp     | cpsAndNcmpLock                     | Cps and NCMP distributed lock for various use cases.      |
++--------------+------------------------------------+-----------------------------------------------------------+
+| cps-ncmp     | cmHandleIdPerAlternateId           | Stores cm handle ids per alternate ids.                   |
++--------------+------------------------------------+-----------------------------------------------------------+
 
 Total number of caches : 8
 
 Total number of caches : 8
"cps"