1 .. This work is licensed under a
2 Creative Commons Attribution 4.0 International License.
9 Integration is in charge of several types of tests:
11 - Use Cases: developed by use case teams, usually complex, demonstrating high value capabilities of ONAP. They may be partially automated and even
13 - CSIT Tests: functional tests created by the projects, partially hosted in CSIT repository
14 - Automatic Test Cases: these use cases are usually more simple and aim to validate that ONAP is working properly.
15 These tests have been developed to validate ONAP as a software solution.
16 In theory all the main functions shall be covered by such tests in order to have more robust CI/CD and then avoid regressions.
17 These tests are usually developed and maintained by the integration team.
19 We may also indicate that when the development of the test framework python-onapsdk
20 follows standard development quality rules and imposes the creation of
21 unit/functional/integration tests.
22 As an example python-onapsdk requires a unit test coverage of 98% before merging
23 a new feature, which is far above the project criteria in SonarCloud today.
28 The use cases of the last release are described in
29 :ref:`Verified Use cases <docs_usecases_release>`.
31 The history of the different use cases across the different releases can be
32 found in :ref:`Use Cases page <docs_usecases>`.
37 The CSIT tests are functional tests executed by the projects on mocked
38 environment to validate their components.
39 Historically it was hosted in a CSIT repository.
41 Integration team invited the projects to bring back such tests back to home
42 repository for 2 main reasons:
44 - integration cannot be a bottleneck: +2/merge from integration needed for each
46 - most of the tests are abandoned and not maintained when hosted in a third party
47 repository leading to CI/CD resource waste and misleading test reporting
49 In Guilin a PoC to help the project to re-insource their functional tests have
51 See `CSIT wiki page <https://wiki.onap.org/display/DW/Maximizing+Benefits+of+CSIT+in+ONAP+Development>`_
57 These tests are run daily/weekly on each new gate (new patchset in OOM, clamp
58 or SO). They can be in any language (bash, go, python,...), leveraging any test
59 framework (robotframework, MTS, python-onapsdk).
60 They are all embedded in `xtesting <https://pypi.org/project/xtesting/>`_ dockers.
63 Automatic tests are currently divided in 4 different categories:
65 - infrastructure-healthcheck: tests from OOM checking the ONAP namespace, certificates...
66 - healthcheck: basic tests on components
67 - smoke tests: end to end tests
70 A dashboard summarizing the status and providing the links to the test result
71 page or the logs is automatically created at the end of the execution of the
74 .. figure:: files/tests/test-dashboard.png
76 All the pages and artifacts are pushed to LF backend:
78 - Daily chaines: https://logs.onap.org/onap-integration/daily
79 - Weekly chains: https://logs.onap.org/onap-integration/weekly
80 - Gating chains: https://logs.onap.org/onap-integration/gating
83 Infrastructure Healthcheck Tests
84 ................................
86 .. csv-table:: Infrastructure Healthcheck Tests
87 :file: ./files/csv/tests-infrastructure-healthcheck.csv
92 See `Infrastructure Healthcheck README <https://git.onap.org/integration/xtesting/tree/infra-healthcheck/README.md>`_
93 to adapt then run infrastructure healthcheck tests on your own system.
95 Please note that the onap-k8s is run 2 times in CD chains. It is run just after
96 the installation (onap-k8s) and at the end of the test execution (onap-k8s-teardown)
97 in order to collect the logs of the different components during the test execution.
99 .. figure:: files/tests/test-onap-k8s.png
104 .. csv-table:: Healthcheck Tests
105 :file: ./files/csv/tests-healthcheck.csv
110 See `Healthcheck README <https://git.onap.org/integration/xtesting/tree/healthcheck/README.md>`_
111 to adapt then run healthcheck tests on your own system.
116 .. csv-table:: Smoke Tests
117 :file: ./files/csv/tests-smoke.csv
122 See `Python smoke test README <https://git.onap.org/integration/xtesting/tree/smoke-usecases-robot/README.md>`_
123 to adapt and run robot based smoke tests.
124 An html page is generated by the pythonsdk-test tests.
126 .. figure:: files/tests/test-basic-cnf.png
129 See `Robot smoke test README <https://git.onap.org/integration/xtesting/tree/smoke-usecases-pythonsdk/README.md>`_
130 to adapt and run python-onapsdk based smoke tests.
131 Standard Robot html pages are generated. See :ref:`Robot page <docs_robot>`.
136 .. csv-table:: Security Tests
137 :file: ./files/csv/tests-security.csv
142 See `Security test README <https://git.onap.org/integration/xtesting/tree/security/README.md>`_
143 to adapt then run the security tests on your own system.
145 Note for security tests, integration team follows `SECCOM recommendations and
146 apply waivers granted by SECCOM if needed through xfail lists <https://git.onap.org/integration/seccom/tree/>`_.
151 Ensuring the stability of ONAP is one of the missions of the Integration team.
152 CI chains and stability tests are performed to help stabilising the release.
153 See :ref:`Integration stability tests <integration-s3p>` for details.