1 .. This work is a derivative of https://wiki.onap.org/display/DW/Running+Blueprints+Processor+Microservice+in+an+IDE
2 .. This work is licensed under a Creative Commons Attribution 4.0
3 .. International License. http://creativecommons.org/licenses/by/4.0
4 .. Copyright (C) 2020 Deutsche Telekom AG.
6 Running Blueprints Processor Microservice in an IDE
7 ====================================================
12 Run the blueprint processor locally in an IDE, while having the database running in a container.
13 This way, code changes can be conveniently tested and debugged.
18 Check out the code from Gerrit: https://gerrit.onap.org/r/#/admin/projects/ccsdk/cds
23 In the checked out directory, type
27 mvn clean install -Pq -Dadditionalparam=-Xdoclint:none
30 If an error ``invalid flag: --release`` appears when executing the maven install command, you need to upgrade Java version of your local
31 Maven installation. Use something like ``export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64``.
33 Wait for the maven install command to finish until you go further.
35 Spin up a Docker container with the database
36 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
38 The Blueprints Processor project uses a database to store information about the blueprints
39 and therefore it needs to be online before attempting to run it.
41 One way to create the database is by using the :file:`docker-compose.yaml` file.
42 This database will require a local directory to mount a volume, therefore before running docker-compose create following directory:
46 mkdir -p -m 755 /opt/app/cds/mysql/data
48 Navigate to the docker-compose file in the distribution module:
52 .. group-tab:: Frankfurt - Latest
56 cd ms/blueprintsprocessor/application/src/main/dc
58 .. group-tab:: El Alto - Dublin
62 ms/blueprintsprocessor/distribution/src/main/dc
64 And run docker-composer:
68 docker-compose up -d db
70 This should spin up a container of the MariaDB image in the background.
71 To check if it has worked, this command can be used:
75 docker-compose logs -f
77 The phrase ``mysqld: ready for connections`` indicates that the database was started correctly.
79 From now on, the Docker container will be available on the computer; if it ever gets stopped,
80 it can be started again by the command:
84 docker start <id of mariadb container>
86 Set permissions on the local file system
87 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
89 Blueprints processor uses the local file system for some operations and, therefore,
90 need some existing and accessible paths to run properly.
92 Execute the following commands to create the needed directories, and grant access to the current user to modify them:
96 mkdir -p -m 755 /opt/app/onap/blueprints/archive
97 mkdir -p -m 755 /opt/app/onap/blueprints/deploy
98 mkdir -p -m 755 /opt/app/onap/scripts
99 sudo chown -R $(id -u):$(id -g) /opt/app/onap/
101 Import the project into the IDE
102 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
106 .. tab:: IntelliJ IDEA
109 This is the recommended IDE for running CDS blueprint processor.
111 Go to *File | Open* and choose the :file:`pom.xml` file of the cds/ms/blueprintprocessor directory:
115 Import as a project. Sometimes it may be necessary to reimport Maven project, e.g. if some dependencies can't be found:
119 **Override some application properties:**
121 Next steps will create a run configuration profile overriding some application properties with custom values,
122 to reflect the local environment characteristics.
126 .. group-tab:: Frankfurt - Latest
128 Navigate to the main class of the Blueprints Processor, the BlueprintProcessorApplication class:
130 ``ms/blueprintsprocessor/application/src/main/kotlin/org/onap/ccsdk/cds/blueprintsprocessor/BlueprintProcessorApplication.kt``.
132 After dependencies are imported and indexes are set up you will see a green arrow
133 next to main function of BlueprintProcessorApplication class, indicating that the run configuration can now be
134 created. Right-click inside the class at any point to load the context menu and select create
135 a run configuration from context:
137 |imageCreateRunConfigKt|
139 **The following window will open:**
143 **Add the following in the field `VM Options`:**
146 :caption: **Custom values for properties**
148 -Dspring.profiles.active=dev
150 Optional: You can override any value from **application-dev.properties** file here. In this case use the following pattern:
154 -D<application-dev.properties key>=<application-dev.properties value>
156 .. group-tab:: El Alto
158 Navigate to the main class of the Blueprints Processor, the BlueprintProcessorApplication class:
160 ``ms/blueprintsprocessor/application/src/main/java/org/onap/ccsdk/cds/blueprintsprocessor/BlueprintProcessorApplication.java.``
162 After dependencies are imported and indexes are set up you will see a green arrow
163 next to main function of BlueprintProcessorApplication class, indicating that the run configuration can now be
164 created. Right-click inside the class at any point to load the context menu and select create
165 a run configuration from context:
167 |imageCreateRunConfigJava|
169 **The following window will open:**
173 **Add the following in the field `VM Options`:**
176 :caption: **Custom values for properties**
178 -Dspring.profiles.active=dev
180 Optional: You can override any value from **application-dev.properties** file here. In this case use the following pattern:
184 -D<application-dev.properties key>=<application-dev.properties value>
186 .. group-tab:: Dublin
188 Navigate to the main class of the Blueprints Processor, the BlueprintProcessorApplication class:
190 ``ms/blueprintsprocessor/application/src/main/java/org/onap/ccsdk/cds/blueprintsprocessor/BlueprintProcessorApplication.java``.
192 After dependencies are imported and indexes are set up you will see a green arrow
193 next to main function of BlueprintProcessorApplication class, indicating that the run configuration can now be
194 created. Right-click inside the class at any point to load the context menu and select create
195 a run configuration from context:
197 |imageCreateRunConfigJava|
199 **The following window will open:**
203 **Add the following in the field `VM Options`**
206 :caption: **Custom values for properties**
208 -DappName=ControllerBluePrints
209 -Dms_name=org.onap.ccsdk.apps.controllerblueprints
211 -Dspring.config.location=opt/app/onap/config/
212 -Dspring.datasource.url=jdbc:mysql://127.0.0.1:3306/sdnctl
213 -Dspring.datasource.username=sdnctl
214 -Dspring.datasource.password=sdnctl
215 -Dcontrollerblueprints.loadInitialData=true
216 -Dblueprintsprocessor.restclient.sdncodl.url=http://localhost:8282/
217 -Dblueprintsprocessor.db.primary.url=jdbc:mysql://localhost:3306/sdnctl
218 -Dblueprintsprocessor.db.primary.username=sdnctl
219 -Dblueprintsprocessor.db.primary.password=sdnctl
220 -Dblueprintsprocessor.db.primary.driverClassName=org.mariadb.jdbc.Driver
221 -Dblueprintsprocessor.db.primary.hibernateHbm2ddlAuto=update
222 -Dblueprintsprocessor.db.primary.hibernateDDLAuto=none
223 -Dblueprintsprocessor.db.primary.hibernateNamingStrategy=org.hibernate.cfg.ImprovedNamingStrategy
224 -Dblueprintsprocessor.db.primary.hibernateDialect=org.hibernate.dialect.MySQL5InnoDBDialect
225 -Dblueprints.processor.functions.python.executor.executionPath=./components/scripts/python/ccsdk_blueprints
226 -Dblueprints.processor.functions.python.executor.modulePaths=./components/scripts/python/ccsdk_blueprints,./components/scripts/python/ccsdk_netconf,./components/scripts/python/ccsdk_restconf
227 -Dblueprintsprocessor.restconfEnabled=true
228 -Dblueprintsprocessor.restclient.sdncodl.type=basic-auth
229 -Dblueprintsprocessor.restclient.sdncodl.url=http://localhost:8282/
230 -Dblueprintsprocessor.restclient.sdncodl.username=admin
231 -Dblueprintsprocessor.restclient.sdncodl.password=Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U
232 -Dblueprintsprocessor.grpcEnable=false
233 -Dblueprintsprocessor.grpcPort=9111
234 -Dblueprintsprocessor.blueprintDeployPath=/opt/app/onap/blueprints/deploy
235 -Dblueprintsprocessor.blueprintArchivePath=/opt/app/onap/blueprints/archive
236 -Dblueprintsprocessor.blueprintWorkingPath=/opt/app/onap/blueprints/work
237 -Dsecurity.user.password={bcrypt}$2a$10$duaUzVUVW0YPQCSIbGEkQOXwafZGwQ/b32/Ys4R1iwSSawFgz7QNu
238 -Dsecurity.user.name=ccsdkapps
239 -Dblueprintsprocessor.messageclient.self-service-api.kafkaEnable=false
240 -Dblueprintsprocessor.messageclient.self-service-api.topic=producer.t
241 -Dblueprintsprocessor.messageclient.self-service-api.type=kafka-basic-auth
242 -Dblueprintsprocessor.messageclient.self-service-api.bootstrapServers=127.0.0.1:9092
243 -Dblueprintsprocessor.messageclient.self-service-api.consumerTopic=receiver.t
244 -Dblueprintsprocessor.messageclient.self-service-api.groupId=receiver-id
245 -Dblueprintsprocessor.messageclient.self-service-api.clientId=default-client-id
246 -Dspring.profiles.active=dev
247 -Dblueprintsprocessor.httpPort=8080
251 **In the field 'Working Directory' browse to your application path** ``.../cds/ms/blueprintsprocessor/application``
252 **if path is not already specified correctly.**
254 Run configuration should now look something like this:
256 |imageRunConfigSetUp|
258 **Add/replace the following in Blueprint's application-dev.properties file.**
262 blueprintsprocessor.grpcclient.remote-python.type=token-auth
263 blueprintsprocessor.grpcclient.remote-python.host=localhost
264 blueprintsprocessor.grpcclient.remote-python.port=50051
265 blueprintsprocessor.grpcclient.remote-python.token=Basic Y2NzZGthcHBzOmNjc2RrYXBwcw==
267 blueprintprocessor.remoteScriptCommand.enabled=true
269 Take care that if a parameter already exist you need to change the value of the existing parameter to avoid duplicates.
272 **Run the application:**
274 Before running Blueprint Processor check that you use the correct Java version in IntelliJ.
275 Select either run or debug for the created Run Configuration to start the Blueprints Processor:
281 .. tab:: Visual Studio Code
285 .. group-tab:: Frankfurt - Latest
287 * **Step #1** - Make sure your installation of Visual Studio Code is up to date. This guide was writen using version 1.48
288 * **Step #2** - Install `Kotlin extension from the Visual Studio Code Marketplace <https://marketplace.visualstudio.com/items?itemName=fwcd.kotlin>`_
289 * **Step #3** - On the top menu click *Run | Open Configurations*
291 .. warning:: This should open the file called `launch.json` but in some cases you'll need to wait for the Kotlin Language Server to be installed before you can do anything.
292 Please watch the bottom bar in Visual Studio Code for messages about things getting installed.
294 * **Step #4** - add configuration shown below to your configurations list.
301 "name": "Blueprint Processor",
302 "projectRoot": "${workspaceFolder}/ms/blueprintsprocessor/application",
303 "mainClass": "-Dspring.profiles.active=dev org.onap.ccsdk.cds.blueprintsprocessor.BlueprintProcessorApplicationKt"
306 .. warning:: The `projectRoot` path assumes that you created your Workspace in the main CDS repository folder. If not - please change the path accordingly
308 .. note:: The `mainClass` contains a spring profile param before the full class name. This is done because `args` is not supported by Kotlin launch.json configuration.
309 If you have a cleaner idea how to solve this - please let us know.
311 **Add/replace the following in Blueprint's application-dev.properties file:**
315 blueprintsprocessor.grpcclient.remote-python.type=token-auth
316 blueprintsprocessor.grpcclient.remote-python.host=localhost
317 blueprintsprocessor.grpcclient.remote-python.port=50051
318 blueprintsprocessor.grpcclient.remote-python.token=Basic Y2NzZGthcHBzOmNjc2RrYXBwcw==
320 blueprintprocessor.remoteScriptCommand.enabled=true
322 **Currently the following entries need to be added in VSC too:**
326 logging.level.web=DEBUG
327 logging.level.org.springframework.web: DEBUG
329 #Encrypted username and password for health check service
330 endpoints.user.name=eHbVUbJAj4AG2522cSbrOQ==
331 endpoints.user.password=eHbVUbJAj4AG2522cSbrOQ==
333 #BaseUrls for health check blueprint processor services
334 blueprintprocessor.healthcheck.baseUrl=http://localhost:8080/
335 blueprintprocessor.healthcheck.mapping-service-name-with-service-link=[Execution service,/api/v1/execution-service/health-check],[Resources service,/api/v1/resources/health-check],[Template service,/api/v1/template/health-check]
337 #BaseUrls for health check Cds Listener services
338 cdslistener.healthcheck.baseUrl=http://cds-sdc-listener:8080/
339 cdslistener.healthcheck.mapping-service-name-with-service-link=[SDC Listener service,/api/v1/sdclistener/healthcheck]
342 management.endpoints.web.exposure.include=*
343 management.endpoint.health.show-details=always
344 management.info.git.mode=full
346 In VSC the properties are read from target folder, thats why the following maven command needs to be rerun:
350 mvn clean install -DskipTests=true -Dmaven.test.skip=true -Dmaven.javadoc.skip=true -Dadditionalparam=-Xdoclint:none
352 Click Run in Menu bar.
357 Testing the application
358 ~~~~~~~~~~~~~~~~~~~~~~~
360 There are two main features of the Blueprints Processor that can be of interest of a developer:
361 blueprint publish and blueprint process.
363 To upload custom blueprints, the endpoint ``api/v1/execution-service/publish`` is used.
365 To process, the endpoint is ``api/v1/execution-service/process``.
367 Postman is a software that can be used to send these request, and an example of
368 them is present on https://www.getpostman.com/collections/b99863b0cde7565a32fc.
370 A detailed description of the usage of different APIs of CDS will follow.
376 Imported packages or annotiations are not found, Run Config not available?
377 *****************************************************************************
379 1. Rebuild with ``maven install ...`` (see above)
380 2. Potentially change Maven home directory in Settings
381 3. Maven reimport in IDE
385 * Change Java Version to 11
388 .. image alignment inside tabs doesn't work
390 .. |imageRunConfigJava| image:: media/run_config_java.png
394 .. |imageRunConfigKt| image:: media/run_config_kt.png
398 .. |imageCreateRunConfigJava| image:: media/create_run_config_java.png
402 .. |imageCreateRunConfigKt| image:: media/create_run_config_kt.png
406 .. |imageImportProject| image:: media/import_project.png
410 .. |imageReimportMaven| image:: media/reimport_maven.png
414 .. |imageRunDebug| image:: media/run_debug.png
418 .. |imageBuildLogs| image:: media/build_logs.png
422 .. |imageLogsVSC| image:: media/vsc_logs.png
426 .. |imageRunConfigSetUp| image:: media/run_config_set_up.png