From: ToineSiebelink Date: Tue, 15 Apr 2025 15:17:14 +0000 (+0100) Subject: Review RTD Configuration Properties X-Git-Tag: 3.6.2~3 X-Git-Url: https://gerrit.onap.org/r/gitweb?a=commitdiff_plain;h=d9376aa9f74576b030a77f20b6097bc3ed7e096d;p=cps.git Review RTD Configuration Properties - split into 3 tables - introduced CSV-file-defined tables for easier maintenance - removed unnecessary 3PP properties definitions - checked all custom properties still used and default values - corrected some default values (sometime just wrong casing) - updated application yaml spring.datasource.url so it has a default host value - removed code-formatting for property values (except for URLS to prevent failing link-checks) - added notes about using environment variables where applicable Issue-ID: CPS-2596 Change-Id: I6f413dd726088ca8a002da145bf4f6c52a1c7b2a Signed-off-by: ToineSiebelink --- diff --git a/cps-application/src/main/resources/application.yml b/cps-application/src/main/resources/application.yml index 9e9d57a441..c934b47cdd 100644 --- a/cps-application/src/main/resources/application.yml +++ b/cps-application/src/main/resources/application.yml @@ -2,7 +2,7 @@ # Copyright (C) 2021 Pantheon.tech # Modifications Copyright (C) 2021-2022 Bell Canada # Modifications Copyright (C) 2024 TechMahindra Ltd -# Modifications Copyright (C) 2021-2025 Nordix Foundation +# Modifications Copyright (C) 2021-2025 OpenInfra Foundation Europe. All rights reserved. # ================================================================================ # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -45,7 +45,7 @@ spring: hibernate.jdbc.batch_size: 100 datasource: - url: jdbc:postgresql://${DB_HOST}:${DB_PORT:5432}/cpsdb + url: jdbc:postgresql://${DB_HOST:localhost}:${DB_PORT:5432}/cpsdb username: ${DB_USERNAME} password: ${DB_PASSWORD} driverClassName: org.postgresql.Driver diff --git a/docs/cps-scheduled-processes.rst b/docs/cps-scheduled-processes.rst index 9af9a81c0c..83cba9f8bd 100644 --- a/docs/cps-scheduled-processes.rst +++ b/docs/cps-scheduled-processes.rst @@ -1,6 +1,6 @@ .. This work is licensed under a Creative Commons Attribution 4.0 International License. .. http://creativecommons.org/licenses/by/4.0 -.. Copyright (C) 2022-2025 Nordix Foundation +.. Copyright (C) 2022-2025 OpenInfra Foundation Europe. All rights reserved. .. DO NOT CHANGE THIS LABEL FOR RELEASE NOTES - EVEN THOUGH IT GIVES A WARNING @@ -20,7 +20,7 @@ The following section is a list of the current scheduled processes running withi Module Sync ----------- -The module sync is a user :ref:`configurable timed process`, +The module sync is a user :ref:`configurable timed process`, which is set to search for CM Handles within CPS with an *'ADVISED'* state. Once the CM Handle is processed by the module sync, the CM Handle state is then set to *'READY'*, if the process completes successfully. If for any reason the module sync fails, the CM Handle state will then be set to *'LOCKED'*, @@ -30,7 +30,7 @@ state are processed by the retry mechanism, by setting CM Handle state back to * Data Sync --------- -The data sync is a user :ref:`configurable timed process`, +The data sync is a user :ref:`configurable timed process`, which is set to search for CM Handles with a sync state of *'UNSYNCHRONIZED'*. Once the CM Handle(s) with a sync state of *'UNSYNCHRONIZED'* is processed by the data sync, the CM Handle sync state is then set to *'SYNCHRONIZED'*, if the process completes successfully. diff --git a/docs/csv/3pp_properties.csv b/docs/csv/3pp_properties.csv new file mode 100644 index 0000000000..822cb4681a --- /dev/null +++ b/docs/csv/3pp_properties.csv @@ -0,0 +1,9 @@ +Property,Description,Default Value +logging.level.org.onap.cps,Logging level set in cps & ncmp,INFO +spring.datasource.username,"Internal user name used by cps-core to connect to its own database.",cps +spring.datasource.password,"Internal password used by cps-core to connect to its own database. If not defined, the password is generated when deploying the application. See also :ref:`cps_common_credentials_retrieval`.",Not defined +spring.datasource.url,"URL to database name used by cps-core.", See note below +spring.datasource.hikari.maximumPoolSize,"Specifies number of database connections between database and application. This property controls the maximum size that the pool is allowed to reach, including both idle and in-use connections.",80 +spring.kafka.bootstrap-servers,"Kafka hostname and port",``localhost:9092`` +spring.kafka.consumer.client-id,"Kafka consumer client id",cps-core +spring.kafka.security.protocol,"Kafka security protocol. Some possible values are: PLAINTEXT, SASL_PLAINTEXT (for authentication), SASL_SSL (for authentication and encryption)",PLAINTEXT diff --git a/docs/csv/common_custom_properties.csv b/docs/csv/common_custom_properties.csv new file mode 100644 index 0000000000..3e6d3fe88c --- /dev/null +++ b/docs/csv/common_custom_properties.csv @@ -0,0 +1,7 @@ +Property,Description,Default Value +notification.async.executor.core-pool-size,"Core pool size in asynchronous execution of notification.",2 +notification.async.executor.max-pool-size,"Max pool size in asynchronous execution of notification.",10 +notification.async.executor.queue-capacity,"Queue Capacity in asynchronous execution of notification.",500 +notification.async.executor.wait-for-tasks-to-complete-on-shutdown,"If the executor should wait for the tasks to be completed on shutdown",true +notification.async.executor.thread-name-prefix,Prefix to be added to the thread name in asynchronous execution of notifications.",Async- +notification.async.executor.time-out-value-in-ms,"Maximum time allowed by the thread pool executor for execution of one of the threads in milliseconds.",60000 diff --git a/docs/csv/ncmp_custom_properties.csv b/docs/csv/ncmp_custom_properties.csv new file mode 100644 index 0000000000..341017a286 --- /dev/null +++ b/docs/csv/ncmp_custom_properties.csv @@ -0,0 +1,16 @@ +Property,"Description",Default Value +ncmp.timers.advised-modules-sync.sleep-time-ms,"Specifies the delay in milliseconds in which the module sync watch dog will wake again after finishing.",5000 +ncmp.timers.advised-modules-sync.initial-delay-ms,"Specifies the delay in milliseconds in which the module sync watch dog will wake up for the first time.",40000 +ncmp.timers.cm-handle-data-sync.sleep-time-ms,"Specifies the delay in milliseconds in which the data sync watch dog will wake again after finishing.",30000 +ncmp.timers.cm-handle-data-sync.initial-delay-ms,"Specifies the delay in milliseconds in which the data sync watch dog will wake up for the first time.",40000 +ncmp.[app].httpclient.[services].maximumInMemorySizeInMegabytes,"Maximum size (in MB) of the in-memory buffer for HTTP response data.",16 +ncmp.[app].httpclient.[services].maximumConnectionsTotal,"Maximum number of simultaneous connections allowed in the connection pool.",100 +ncmp.[app].httpclient.[services].pendingAcquireMaxCount,"Maximum number of pending requests when the connection pool is full.",50 +ncmp.[app].httpclient.[services].connectionTimeoutInSeconds,"Specifies the maximum time in seconds, to wait for establishing a connection for the HTTP client",30 +ncmp.[app].httpclient.[services].readTimeoutInSeconds,"Timeout (in seconds) for reading data from the server after the connection is established.",30 +ncmp.[app].httpclient.[services].writeTimeoutInSeconds,"Timeout (in seconds) for writing data to the server.",30 +ncmp.[app].httpclient.[services].responseTimeoutInSeconds,"Total timeout (in seconds) for receiving a complete response, including all processing stages.",60 +ncmp.policy-executor.enabled,"Enables or disables the policy-executor feature.",false +ncmp.policy-executor.defaultDecision,"The default (fallback) decision in case a problem with the external service occurs.",allow +ncmp.policy-executor.server.address,"The server address for the external policy executor service.",``http://policy-executor-stub`` +ncmp.policy-executor.server.port,"The port used for the external policy executor service.",8093 diff --git a/docs/deployment.rst b/docs/deployment.rst index 990fea1365..31673d6560 100644 --- a/docs/deployment.rst +++ b/docs/deployment.rst @@ -1,6 +1,6 @@ .. This work is licensed under a Creative Commons Attribution 4.0 International License. .. http://creativecommons.org/licenses/by/4.0 -.. Copyright (C) 2021-2025 Nordix Foundation +.. Copyright (C) 2021-2025 OpenInfra Foundation Europe. All rights reserved. .. Modifications Copyright (C) 2021 Bell Canada. .. DO NOT CHANGE THIS LABEL FOR RELEASE NOTES - EVEN THOUGH IT GIVES A WARNING @@ -139,7 +139,7 @@ Each cps component can be restarted independently by issuing the following comma Credentials Retrieval ===================== -Application and database credentials are kept in Kubernetes secrets. They are defined as external secrets in the +When using OOM application and database credentials are kept in Kubernetes secrets. They are defined as external secrets in the values.yaml file to be used across different components as : .. container:: ulist @@ -189,193 +189,36 @@ To get a listing of the cps-core Pods, run the following command: .. note:: The CPS Service will have to be restarted each time a change is made to a configurable property. -Additional CPS-Core Customizations -================================== - -The following table lists some properties that can be configured in the deployment. This list is not exhaustive. - -+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| Property | Description | Default Value | -+===========================================+=========================================================================================================+===============================+ -| appUserName | User name used by cps-core service to configure the authentication for REST API it exposes. | ``cpsuser`` | -| | This is the user name to be used by cps-core REST clients to authenticate themselves. | | -+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| appUserPassword | Password used by cps-core service to configure the authentication for REST API it exposes. | Not defined | -| | If not defined, the password is generated when deploying the application. | | -| | See also :ref:`cps_common_credentials_retrieval`. | | -+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| postgres.pgUserName | Internal user name used by cps-core to connect to its own database. | ``cps`` | -+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| postgres.pgUserPassword | Internal password used by cps-core to connect to its own database. | Not defined | -| | If not defined, the password is generated when deploying the application. | | -| | See also :ref:`cps_common_credentials_retrieval`. | | -+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| postgres.pgDatabase | Database name used by cps-core | ``cpsdb`` | -| | | | -+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| logging.level | Logging level set in cps-core | info | -| | | | -+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| useStrimziKafka | If targeting a custom kafka cluster, i.e. useStrimziKafka: false, the | true | -| | eventPublisher.spring.kafka values below must be set. | | -+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| eventPublisher. | Kafka hostname and port | ``:9092`` | -| spring.kafka.bootstrap-servers | | | -+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| eventPublisher. | Kafka consumer client id | ``cps-core`` | -| spring.kafka.consumer.client-id | | | -+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| eventPublisher. | Kafka security protocol. | ``SASL_PLAINTEXT`` | -| spring.kafka.security.protocol | Some possible values are: | | -| | * ``PLAINTEXT`` | | -| | * ``SASL_PLAINTEXT``, for authentication | | -| | * ``SASL_SSL``, for authentication and encryption | | -+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| eventPublisher. | Kafka security SASL mechanism. Required for SASL_PLAINTEXT and SASL_SSL protocols. | Not defined | -| spring.kafka.properties. | Some possible values are: | | -| sasl.mechanism | * ``PLAIN``, for PLAINTEXT | | -| | * ``SCRAM-SHA-512``, for SSL | | -+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| eventPublisher. | Kafka security SASL JAAS configuration. Required for SASL_PLAINTEXT and SASL_SSL protocols. | Not defined | -| spring.kafka.properties. | Some possible values are: | | -| sasl.jaas.config | * ``org.apache.kafka.common.security.plain.PlainLoginModule required username="..." password="...";``, | | -| | for PLAINTEXT | | -| | * ``org.apache.kafka.common.security.scram.ScramLoginModule required username="..." password="...";``, | | -| | for SSL | | -+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| eventPublisher. | Kafka security SASL SSL store type. Required for SASL_SSL protocol. | Not defined | -| spring.kafka.ssl.trust-store-type | Some possible values are: | | -| | * ``JKS`` | | -+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| eventPublisher. | Kafka security SASL SSL store file location. Required for SASL_SSL protocol. | Not defined | -| spring.kafka.ssl.trust-store-location | | | -+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| eventPublisher. | Kafka security SASL SSL store password. Required for SASL_SSL protocol. | Not defined | -| spring.kafka.ssl.trust-store-password | | | -+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| eventPublisher. | Kafka security SASL SSL broker hostname identification verification. Required for SASL_SSL protocol. | Not defined | -| spring.kafka.properties. | Possible value is: | | -| ssl.endpoint.identification.algorithm | | | -| | * ``""``, empty string to disable | | -+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| additional. | Core pool size in asynchronous execution of notification. | ``2`` | -| notification.async.executor. | | | -| core-pool-size | | | -+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| additional. | Max pool size in asynchronous execution of notification. | ``1`` | -| notification.async.executor. | | | -| max-pool-size | | | -+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| additional. | Queue Capacity in asynchronous execution of notification. | ``500`` | -| notification.async.executor. | | | -| queue-capacity | | | -+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| additional. | If the executor should wait for the tasks to be completed on shutdown | ``true`` | -| notification.async.executor. | | | -| wait-for-tasks-to-complete-on-shutdown | | | -+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| additional. | Prefix to be added to the thread name in asynchronous execution of notifications. | ``Async-`` | -| notification.async.executor. | | | -| thread-name-prefix | | | -+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| additional. | Maximum time allowed by the thread pool executor for execution of one of the threads in milliseconds. | ``60000`` | -| notification.async.executor. | | | -| time-out-value-in-ms | | | -+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| additional. | Specifies number of database connections between database and application. | ``10`` | -| spring.datasource.hikari. | This property controls the maximum size that the pool is allowed to reach, | | -| maximumPoolSize | including both idle and in-use connections. | | -+-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ - -.. _additional-cps-ncmp-customizations: - -Additional CPS-NCMP Customizations -================================== - -+-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ -| Property | Description | Default Value | -+=================================================+=======================================================================================+=================================+ -| dmiPluginUserName | User name used by cps-core to authenticate themselves for using ncmp-dmi-plugin | ``dmiuser`` | -| | service. | | -+-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ -| dmiPluginUserPassword | Internal password used by cps-core to connect to ncmp-dmi-plugin service. | Not defined | -| | If not defined, the password is generated when deploying the application. | | -| | See also :ref:`cps_common_credentials_retrieval`. | | -+-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ -| ncmp.timers | Specifies the delay in milliseconds in which the module sync watch dog will wake again| ``5000`` | -| .advised-modules-sync.sleep-time-ms | after finishing. | | -+-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ -| ncmp.timers | Specifies the delay in milliseconds in which the module sync watch dog will wake up | ``40000`` | -| .advised-modules-sync.initial-delay-ms | for the first time. | | -+-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ -| ncmp.timers | Specifies the delay in milliseconds in which the data sync watch dog will wake again | ``30000`` | -| .cm-handle-data-sync.sleep-time-ms | after finishing. | | -| | | | -+-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ -| ncmp.timers | Specifies the delay in milliseconds in which the data sync watch dog will wake up | ``40000`` | -| .cm-handle-data-sync.initial-delay-ms | for the first time. | | -| | | | -+-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ -| additional.ncmp | Maximum size (in MB) of the in-memory buffer for HTTP response data. | ``16`` | -| .[app] | | | -| .httpclient | | | -| .[services] | | | -| .maximumInMemorySizeInMegabytes | | | -+-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ -| additional.ncmp | Maximum number of simultaneous connections allowed in the connection pool. | ``100`` | -| .[app] | | | -| .httpclient | | | -| .[services] | | | -| .maximumConnectionsTotal | | | -+-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ -| additional.ncmp | Maximum number of pending requests when the connection pool is full. | ``50`` | -| .[app] | | | -| .httpclient | | | -| .[services] | | | -| .pendingAcquireMaxCount | | | -+-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ -| additional.ncmp | Specifies the maximum time in seconds, to wait for establishing a connection for the | ``30`` | -| .[app] | HTTP Client. | | -| .httpclient | | | -| .[services] | | | -| .connectionTimeoutInSeconds | | | -+-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ -| additional.ncmp | Timeout (in seconds) for reading data from the server after the connection is | ``30`` | -| .[app] | established. | | -| .httpclient | | | -| .[services] | | | -| .readTimeoutInSeconds | | | -+-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ -| additional.ncmp | Timeout (in seconds) for writing data to the server. | ``30`` | -| .[app] | | | -| .httpclient | | | -| .[services] | | | -| .writeTimeoutInSeconds | | | -+-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ -| additional.ncmp | Total timeout (in seconds) for receiving a complete response, including all processing| ``60`` | -| .[app] | stages. | | -| .httpclient | | | -| .[services] | | | -| .responseTimeoutInSeconds | | | -+-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ -| additional.ncmp.policy-executor | Enables or disables the policy-executor feature. | ``false`` | -| .enabled | | | -+-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ -| additional.ncmp.policy-executor | The default (fallback) decision in case a problem with the external service occurs. | ``allow`` | -| .defaultDecision | | | -+-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ -| additional.ncmp.policy-executor | The server address for the external policy executor service. | ``http://policy-executor-stub`` | -| .server.address | | | -+-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ -| additional.ncmp.policy-executor | The port used for the external policy executor service. | ``8093`` | -| .server.port | | | -+-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ +.. _configuration-properties: + +Configuration Properties +======================== +The following tables list properties that can be configured in the deployment. This list is not exhaustive. + +.. csv-table:: 3PP Properties + :file: csv/3pp_properties.csv + :widths: 20, 50, 30 + :header-rows: 1 .. note:: + - The default datasource is defined as ``jdbc:postgresql://${DB_HOST:localhost}:${DB_PORT:5432}/cpsdb``. So it can also be configured using environment variables to just set the hostname ``DB_HOST`` and port ``DB_PORT``. + - The kafka bootstrap-servers can also be overridden with the environment variable ``KAFKA_BOOTSTRAP_SERVER``. + +.. csv-table:: Common CPS-NCMP Custom Properties + :file: csv/common_custom_properties.csv + :widths: 20, 50, 30 + :header-rows: 1 - - [app] : can be 'policy-executor' or 'dmi'. - - [services] 'all-services' for 'policy-executor'. - - [services] 'data-services' and 'model-services' for 'dmi'. +.. csv-table:: NCMP Custom Properties + :file: csv/ncmp_custom_properties.csv + :widths: 20, 50, 30 + :header-rows: 1 + +.. note:: + - [app]: can be ``policy-executor`` or ``dmi``. + - [services]: ``all-services`` for 'policy-executor'. + - [services]: ``data-services`` and 'model-services' for 'dmi'. + - All ncmp.policy-executor properties can also be overridden using environment variables: ``POLICY_SERVICE_ENABLED``, ``POLICY_SERVICE_DEFAULT_DECISION``, ``POLICY_SERVICE_URL``, ``POLICY_SERVICE_PORT`` CPS-Core Docker Installation ============================ diff --git a/docs/policy-executor.rst b/docs/policy-executor.rst index 712b4fcf38..d7a0c7bf53 100644 --- a/docs/policy-executor.rst +++ b/docs/policy-executor.rst @@ -1,6 +1,6 @@ .. This work is licensed under a Creative Commons Attribution 4.0 International License. .. http://creativecommons.org/licenses/by/4.0 -.. Copyright (C) 2024-2025 Nordix Foundation +.. Copyright (C) 2024-2025 OpenInfra Foundation Europe. All rights reserved. .. DO NOT CHANGE THIS LABEL - EVEN THOUGH IT GIVES A WARNING .. _policy_executor: @@ -20,7 +20,7 @@ The details of the interface can be found in the ':ref:`policy_executor_consumed This feature is available on 'legacy data interface' for operation on a single cm handle: "/v1/ch/{cm-handle}/data/ds/{datastore-name}" and only applies to "ncmp-datastore:passthrough-running". -By default, the feature is not enabled. This is controlled by 'config.additional.ncmp.policy-executor.enabled' and other deployment parameters in the same group to enable it. See :ref:`additional-cps-ncmp-customizations` +By default, the feature is not enabled. This is controlled by 'config.additional.ncmp.policy-executor.enabled' and other deployment parameters in the same group to enable it. See :ref:`configuration-properties` .. DO NOT CHANGE THIS LABEL - EVEN THOUGH IT GIVES A WARNING .. _policy_executor_consumed_apis: