.. This work is licensed under a
Creative Commons Attribution 4.0 International License.
-.. integration-tests:
+.. _integration-tests:
Tests
=====
.. important::
- Integration is in charged of several types of tests:
+ Integration is in charge of several types of tests:
- - Use Cases: developped by use case teams, usually complex, demonstrating high value capabilities ofr ONAP. They may be partially automated and even
+ - Use Cases: developed by use case teams, usually complex, demonstrating high value capabilities of ONAP. They may be partially automated and even
integrated in CD.
- CSIT Tests: functional tests created by the projects, partially hosted in CSIT repository
- - Automatic Test Cases: these use cases are usually more simple and aims to validate that ONAP is working properly.
+ - Automatic Test Cases: these use cases are usually more simple and aim to validate that ONAP is working properly.
These tests have been developed to validate ONAP as a software solution.
- In theory all the main function shall be covered by such tests in order to have more robust CI/CD and then avoid regressions.
+ In theory all the main functions shall be covered by such tests in order to have more robust CI/CD and then avoid regressions.
These tests are usually developed and maintained by the integration team.
-We may also indicate that when the development of the test framework python-onapsk
-follows standard developement quality rules and imposes the creation of
+We may also indicate that when the development of the test framework python-onapsdk
+follows standard development quality rules and imposes the creation of
unit/functional/integration tests.
As an example python-onapsdk requires a unit test coverage of 98% before merging
-a new feature, which is far above the project criteria in SonarCLoud today.
+a new feature, which is far above the project criteria in SonarCloud today.
Use Cases
---------
-The use cases are described in :ref:`Verified Use cases <docs_usecases>`.
+The use cases of the last release are described in
+:ref:`Verified Use cases <docs_usecases_release>`.
CSIT Tests
----------
The CSIT tests are functional tests executed by the projects on mocked
-environement to validate their components.
+environment to validate their components.
Historically it was hosted in a CSIT repository.
-Integration team invited the projects to bring back such tests bacdk to home
+Integration team invited the projects to bring back such tests back to home
repository for 2 main reasons:
- integration cannot be a bottleneck: +2/merge from integration needed for each
project
-- most of the tests are abandonned and not maintained when hosted in a third party
- repository leading to CI/CD time waste and misleading test reporting
+- most of the tests are abandoned and not maintained when hosted in a third party
+ repository leading to CI/CD resource waste and misleading test reporting
-In Guilin a PoC to help the project to re-insource their functional tests have
-been initiated.
-See `CSIT wiki page <https://wiki.onap.org/display/DW/Maximizing+Benefits+of+CSIT+in+ONAP+Development>`__
-for details.
-
-Automatic Tests
+Automated Tests
---------------
-These tests are run daily/weekly on each new gate (new patchset in OOM, clamp
+These tests are run daily/weekly on each new gate (new patchset in OOM, CLAMP
or SO). They can be in any language (bash, go, python,...), leveraging any test
framework (robotframework, MTS, python-onapsdk).
-They are all embedded in `xtesting <https://pypi.org/project/xtesting/>`__ dockers.
+They are all embedded in `xtesting <https://pypi.org/project/xtesting/>`_ dockers.
.. hint::
Automatic tests are currently divided in 4 different categories:
- - infrastructure-healthcheck: tests from OOM checking the ONAP namespace, certificates,..
+ - infrastructure-healthcheck: tests from OOM checking the ONAP namespace, certificates...
- healthcheck: basic tests on components
- smoke tests: end to end tests
- security tests
.. figure:: files/tests/test-dashboard.png
+ Test dashboard (Guilin version)
+
All the pages and artifacts are pushed to LF backend:
-- Daily chaines: https://logs.onap.org/onap-integration/daily
+- Daily chains: https://logs.onap.org/onap-integration/daily
- Weekly chains: https://logs.onap.org/onap-integration/weekly
-- Gating chains: https://logs.onap.org/onap-integration/gating
+- Gating chains: the result link is indicated in gerrit
+A video has been recorded to help launching some of the automated tests on ONAP Guilin.
+See `Running ONAP tests in Guilin Video <https://www.youtube.com/watch?v=ABvuJfyGDmw>`_
Infrastructure Healthcheck Tests
................................
:delim: ;
:header-rows: 1
-See `Infrastructure Healthcheck README <https://git.onap.org/integration/xtesting/tree/infra-healthcheck/README.md>`__
+See `Infrastructure Healthcheck README <https://git.onap.org/integration/xtesting/tree/infra-healthcheck/README.md>`_
to adapt then run infrastructure healthcheck tests on your own system.
Please note that the onap-k8s is run 2 times in CD chains. It is run just after
.. figure:: files/tests/test-onap-k8s.png
-Healhcheck Tests
-................
+Healthcheck Tests
+.................
.. csv-table:: Healthcheck Tests
:file: ./files/csv/tests-healthcheck.csv
:delim: ;
:header-rows: 1
-See `Healthcheck README <https://git.onap.org/integration/xtesting/tree/healthcheck/README.md>`__
+See `Healthcheck README <https://git.onap.org/integration/xtesting/tree/healthcheck/README.md>`_
to adapt then run healthcheck tests on your own system.
Smoke Tests
:delim: ;
:header-rows: 1
-See `Python smoke test README <https://git.onap.org/integration/xtesting/tree/smoke-usecases-robot/README.md>`__
-to adapt and run robot based smoke tests.
-An html page is generated by the pythonsdk-test tests.
-
-.. figure:: files/tests/test-basic-cnf.png
+There are 2 main families of smoke tests:
+* RobotFramework based tests, usually run from inside the cluster as a k8s job
+* Pythonsdk based tests. These tests (also known as onaptests) are consuming
+ several SDKs: the Openstack and Kubernetes SDK for the management of the cloud
+ resources and the python ONAP SDK for the interactions with ONAP
-See `Robot smoke test README <https://git.onap.org/integration/xtesting/tree/smoke-usecases-pythonsdk/README.md>`__
-to adapt and run pythonsdk based smoke tests.
+To launch the the robot based tests, please see
+`Robot smoke test README <https://git.onap.org/integration/xtesting/tree/smoke-usecases-robot/README.md>`_
Standard Robot html pages are generated. See :ref:`Robot page <docs_robot>`.
+To launch the pythonsdk based tests, please see
+`Python smoke test README <https://git.onap.org/integration/xtesting/tree/smoke-usecases-pythonsdk/README.md>`_
+
+.. note:
+ Please note that the OpenStack minimal version is pike.
+
+An html page is generated by the pythonsdk-test tests.
+
+.. figure:: files/tests/test-basic-cnf.png
+
Security Tests
...............
:delim: ;
:header-rows: 1
-See `Security test README <https://git.onap.org/integration/xtesting/tree/security/README.md>`__
+See `Security test README <https://git.onap.org/integration/xtesting/tree/security/README.md>`_
to adapt then run the security tests on your own system.
Note for security tests, integration team follows `SECCOM recommendations and
-apply waivers granted by SECCOM if needed through xfail lists <https://git.onap.org/integration/seccom/tree/>`__.
+apply waivers granted by SECCOM if needed through xfail lists <https://git.onap.org/integration/seccom/tree/>`_.
-Stability Testing
------------------
+Stability tests
+---------------
Ensuring the stability of ONAP is one of the missions of the Integration team.
CI chains and stability tests are performed to help stabilising the release.
Code Review / integration.git / blobdiff