netconf-pnp-simulator: convenient TLS and SSH configuration 84/105184/3
authorebo <eliezio.oliveira@est.tech>
Fri, 3 Apr 2020 14:24:15 +0000 (15:24 +0100)
committerBartek Grzybowski <b.grzybowski@partner.samsung.com>
Wed, 8 Apr 2020 08:52:09 +0000 (08:52 +0000)
- Simple SSH and TLS configuration. Instead of specific Netopeer2
  XML configuration files, the user only needs to provide:
  For SSH: id_XXX.pub
  For TLS: server_key.pem, server_cert.pem, and ca.pem

- SSH and TLS can be reconfigured at runtime by running
  /opt/bin/reconfigure-ssh.sh and /opt/bin/reconfigure-tls.sh respectively

- Improved log readability by using zlog (on C applications) and loguru
  for Python

See the updated documentation under ../docs for more information.

Issue-ID: INT-1516
Change-Id: I21052d2524f0610c6197875a544113cce1a02787
Signed-off-by: ebo <eliezio.oliveira@est.tech>
25 files changed:
test/mocks/netconf-pnp-simulator/docs/README.md [deleted file]
test/mocks/netconf-pnp-simulator/docs/README.rst [new file with mode: 0644]
test/mocks/netconf-pnp-simulator/engine/Dockerfile
test/mocks/netconf-pnp-simulator/engine/common.sh [new file with mode: 0644]
test/mocks/netconf-pnp-simulator/engine/config/ssh/load_auth_pubkey.xml [deleted file]
test/mocks/netconf-pnp-simulator/engine/config/tls/ca.pem [new file with mode: 0644]
test/mocks/netconf-pnp-simulator/engine/config/tls/load_server_certs.xml [deleted file]
test/mocks/netconf-pnp-simulator/engine/config/tls/netopeer2-client.sh [new file with mode: 0755]
test/mocks/netconf-pnp-simulator/engine/config/tls/server_cert.pem [new file with mode: 0644]
test/mocks/netconf-pnp-simulator/engine/config/tls/server_key.pem.pub [deleted file]
test/mocks/netconf-pnp-simulator/engine/configure-modules.sh [new file with mode: 0755]
test/mocks/netconf-pnp-simulator/engine/container-tag.yaml
test/mocks/netconf-pnp-simulator/engine/entrypoint.sh
test/mocks/netconf-pnp-simulator/engine/patches/Netopeer2/02-zlog.patch [new file with mode: 0644]
test/mocks/netconf-pnp-simulator/engine/patches/supervisor/01-std-log-format.patch [new file with mode: 0644]
test/mocks/netconf-pnp-simulator/engine/patches/sysrepo/02-zlog.patch [new file with mode: 0644]
test/mocks/netconf-pnp-simulator/engine/reconfigure-ssh.sh [new file with mode: 0755]
test/mocks/netconf-pnp-simulator/engine/reconfigure-tls.sh [new file with mode: 0755]
test/mocks/netconf-pnp-simulator/engine/supervisord.conf
test/mocks/netconf-pnp-simulator/engine/templates/load_auth_pubkey.xml [new file with mode: 0644]
test/mocks/netconf-pnp-simulator/engine/templates/load_server_certs.xml [new file with mode: 0644]
test/mocks/netconf-pnp-simulator/engine/templates/tls_listen.xml [moved from test/mocks/netconf-pnp-simulator/engine/config/tls/tls_listen.xml with 88% similarity]
test/mocks/netconf-pnp-simulator/engine/tests/test_ietf_interfaces.py [deleted file]
test/mocks/netconf-pnp-simulator/engine/tests/test_turing_machine.py [deleted file]
test/mocks/netconf-pnp-simulator/engine/zlog.conf [new file with mode: 0644]

diff --git a/test/mocks/netconf-pnp-simulator/docs/README.md b/test/mocks/netconf-pnp-simulator/docs/README.md
deleted file mode 100644 (file)
index 8aa2450..0000000
+++ /dev/null
@@ -1,63 +0,0 @@
-# NETCONF Plug-and-Play Simulator
-
-[![GitHub Tag][gh-tag-badge]]()
-[![Docker Automated Build][dockerhub-badge]][dockerhub]
-
-## Overview
-
-This project builds a modular engine that allows the creation of NETCONF-enabled devices simulators,
-either physical (PNF) and virtual (VNF).
-
-Basically it's a docker container running Sysrepo and Netopeer2 servers enhanced with a plugger script that, at
-start-time, performs the following actions:
-
-1. Configures TLS and SSH secure accesses to the Netopeer2 server;
-2. Installs multiple YANG models into sysrepo datastore;
-3. Launches the corresponding subscriber applications.
-
-The picture below unveils the architecture of this solution.
-
-![Architecture](images/Architecture.png)
-
-A YANG module contains the following files:
-
-| Filename | Purpose
-| -------- | -------
-|`model.yang` | The YANG model specified according to [RFC-6020][yang-rfc]. Alternatively, you can use your model's name as a basename for this file. Example: `mynetconf.yang`.
-|`data.json` or `data.xml` | An optional data file used to initialize your model.
-|`subscriber.py` | The Python 3 application that implements the behavioral aspects of the YANG model.
-|`requirements.txt` | [Optional] Additional Python packages specified in the [Requirements File Format][py-requirements].
-
-## Application
-
-The `subscriber` is free to implement any wanted passive or active behaviour:
-
-**Passive Behaviour**: The subscriber will receive an event for each modification externally applied to the YANG model.
-
-**Active Behaviour**: At any point in time the subscriber can proactively change its own YANG model.
-
-## Runtime Configuration
-
-### Customizing TLS and SSH accesses
-
-The distributed docker image comes with a sample configuration for TLS and SSH, that can be found at
-`/config/tls` and `/home/netconf/.ssh` directories respectively. The user can replace one or both configurations
-by mounting a custom directory under the respective TLS or SSH mounting point.
-
-### Python Virtual Environment Support
-
-Python programs usually use additional packages not included in the standard Python distribution,
-like the `requests` package, for example.
-We support this scenario by creating isolated Python environments for each custom-provided module whenever
-a `requirements.txt` file is present in the module directory.
-
-## Example Module
-
-The directory `examples/mynetconf` contains an example YANG model and its subscriber along with a
-Docker Compose configuration file to launch a basic simulator.
-
-[dockerhub]:                  https://hub.docker.com/r/blueonap/netconf-pnp-simulator/
-[dockerhub-badge]:            https://img.shields.io/docker/cloud/automated/blueonap/netconf-pnp-simulator
-[gh-tag-badge]:               https://img.shields.io/github/v/tag/blue-onap/netconf-pnp-simulator?label=Release
-[py-requirements]:            https://pip.pypa.io/en/stable/reference/pip_install/#requirements-file-format
-[yang-rfc]:                   https://tools.ietf.org/html/rfc6020
diff --git a/test/mocks/netconf-pnp-simulator/docs/README.rst b/test/mocks/netconf-pnp-simulator/docs/README.rst
new file mode 100644 (file)
index 0000000..4528279
--- /dev/null
@@ -0,0 +1,116 @@
+NETCONF Plug-and-Play Simulator
+===============================
+
+.. sectnum::
+
+.. _py-requirements: https://pip.pypa.io/en/stable/reference/pip_install/#requirements-file-format
+.. _yang-rfc: https://tools.ietf.org/html/rfc6020
+
+|ci-badge| |release-badge| |docker-badge|
+
+.. |ci-badge| image:: https://github.com/blue-onap/netconf-pnp-simulator/workflows/CI/badge.svg
+   :alt: CI
+.. |release-badge| image:: https://img.shields.io/github/v/tag/blue-onap/netconf-pnp-simulator?label=Release
+   :alt: GitHub tag
+.. |docker-badge| image:: https://img.shields.io/badge/docker%20registry-Quay.io-red
+   :target: https://quay.io/repository/blue-onap/netconf-pnp-simulator?tab=tags
+
+Overview
+--------
+
+This project builds a modular engine that allows the creation of NETCONF-enabled devices simulators,
+either physical (PNF), virtual (VNF), or cloud-native (CNF)
+
+Simply put, it's a docker container running Sysrepo and Netopeer2 servers enhanced with a plugger script that
+performs the following actions at start-time:
+
+1. Configures TLS and SSH secure accesses to the Netopeer2 server;
+2. Installs multiple YANG models into sysrepo datastore;
+3. Launches the corresponding subscriber applications.
+
+The picture below unveils the architecture of this solution.
+
+.. image:: images/Architecture.png
+   :width: 511px
+
+A YANG module contains the following files:
+
+.. list-table::
+   :widths: 10 50
+   :header-rows: 1
+
+   * - Filename
+     - Purpose
+   * - ``model.yang``
+     - The YANG model specified according to `RFC-6020 <yang-rfc_>`_ and named after the module's name, e.g., *mynetconf.yang*.
+   * - ``startup.json`` or ``startup.xml``
+     - An optional data file with the initial values of the model. Both JSON and XML formats are supported.
+   * - ``subscriber.py``
+     - The Python 3 application that implements the behavioral aspects of the YANG model.
+   * - ``requirements.txt``
+     - [Optional] Lists the additional Python packages required by the application, specified in the `Requirements File Format <py-requirements_>`_.
+
+Application
+-----------
+
+The ``subscriber.py`` application can implement any wanted passive or active behaviour:
+
+**Passive Behaviour**: The subscriber will receive an event for each modification externally applied to the YANG model.
+
+**Active Behaviour**: At any point in time the subscriber can proactively change its own YANG model.
+
+Runtime Configuration
+---------------------
+
+Customizing TLS and SSH accesses
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The distributed docker image comes with a sample configuration for TLS and SSH, that can be found at
+``/config/tls`` and ``/config/ssh`` directories respectively. The user can replace one or both configurations
+by mounting a custom directory under the respective TLS or SSH mounting point.
+
+TLS Configuration
+^^^^^^^^^^^^^^^^^
+
+You need to provide the following PEM files under ``/config/tls``:
+
+.. list-table::
+   :widths: 10 50
+   :header-rows: 1
+
+   * - File
+     - Contents
+   * - ``server_key.pem``
+     - The server's private key in plain (*not* protected by a passphrase).
+   * - ``server_cert.pem``
+     - The corresponding server's X.509v3 certificate.
+   * - ``ca.pem``
+     - The Certificate Authority (CA) certificate.
+
+.. TIP:: You can reload the configuration at runtime by running ``docker exec <CONTAINER NAME or ID> /opt/bin/reconfigure-tls.sh``
+
+SSH Configuration
+^^^^^^^^^^^^^^^^^
+
+For the SSH connection, you need to provide the public SSH key in one of these 3 files under ``/config/ssh``
+in order of preference:
+
+- ``id_ecdsa.pub``; or
+- ``id_dsa.pub``; or
+- ``id_rsa.pub``
+
+.. TIP:: You can reload the configuration at runtime by running ``docker exec <CONTAINER NAME or ID> /opt/bin/reconfigure-ssh.sh``
+
+Python Virtual Environment Support
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Python programs usually use additional packages not included in the standard Python distribution,
+like the ``requests`` package, for example.
+We support this scenario by creating isolated Python environments for each custom-provided module whenever
+a ``requirements.txt`` file is present in the module directory.
+
+Example Module
+--------------
+
+The directory ``examples/mynetconf`` contains an example YANG model and its subscriber along with a
+Docker Compose configuration file to launch a basic simulator.
index d4776a4..a5d17c5 100644 (file)
@@ -17,8 +17,9 @@
 # SPDX-License-Identifier: Apache-2.0
 # ============LICENSE_END=========================================================
 
-FROM python:3.7.6-alpine3.11 as build
+FROM python:3.7.7-alpine3.11 as build
 
+ARG zlog_version=1.2.14
 ARG libyang_version=v1.0-r5
 ARG sysrepo_version=v0.7.9
 ARG libnetconf2_version=v0.12-r2
@@ -77,6 +78,13 @@ RUN set -eux \
       && make \
       && make install
 
+# zlog
+RUN set -eux \
+      && git clone --branch $zlog_version --depth 1 https://github.com/HardySimpson/zlog \
+      && cd zlog/src \
+      && make PREFIX=/opt \
+      && make install PREFIX=/opt
+
 # sysrepo
 COPY patches/sysrepo/ ./patches/sysrepo/
 RUN set -eux \
@@ -89,6 +97,7 @@ RUN set -eux \
          -DCMAKE_INSTALL_PREFIX:PATH=/opt \
          -DGEN_PYTHON_VERSION=3 \
          -DPYTHON_MODULE_PATH:PATH=/opt/lib/python3.7/site-packages \
+         -DBUILD_EXAMPLES=0 \
          .. \
       && make -j2 \
       && make install
@@ -132,18 +141,21 @@ RUN set -eux \
       && make -j2 \
       && make install
 
-FROM python:3.7.6-alpine3.11
+FROM python:3.7.7-alpine3.11
 LABEL authors="eliezio.oliveira@est.tech"
 
 RUN set -eux \
-      && pip install supervisor \
+      && pip install loguru supervisor virtualenv \
       && apk update \
       && apk upgrade -a \
       && apk add \
+         coreutils \
          libcurl \
          libev \
+         openssl \
          pcre \
          protobuf-c \
+         xmlstarlet \
       # v0.9.3 has somes bugs as warned in libnetconf2/CMakeLists.txt:237
       && apk add --repository http://dl-cdn.alpinelinux.org/alpine/v3.10/main libssh==0.8.8-r0 \
       && rm -rf /var/cache/apk/*
@@ -153,14 +165,19 @@ COPY --from=build /opt/ /opt/
 ENV LD_LIBRARY_PATH=/opt/lib:/opt/lib64
 ENV PYTHONPATH=/opt/lib/python3.7/site-packages
 
+COPY patches/supervisor/ /usr/src/patches/supervisor/
+
+RUN set -eux \
+      && cd /usr/local/lib/python3.7/site-packages \
+      && for p in /usr/src/patches/supervisor/*.patch; do patch -p1 -i $p; done
+
 COPY config/ /config
 VOLUME /config
+COPY templates/ /templates
 
 # finish setup and add netconf user
 RUN adduser --system --disabled-password --gecos 'Netconf User' netconf
 
-ENV HOME=/home/netconf
-VOLUME $HOME/.local/share/virtualenvs
 # This is NOT a robust health check but it does help tox-docker to detect when
 # it can start the tests.
 HEALTHCHECK --interval=1s --start-period=2s --retries=10 CMD test -f /run/netopeer2-server.pid
@@ -170,6 +187,12 @@ EXPOSE 830
 COPY supervisord.conf /etc/supervisord.conf
 RUN mkdir /etc/supervisord.d
 
-COPY entrypoint.sh /opt/bin/
+COPY zlog.conf /opt/etc/
+
+# Sensible defaults for loguru configuration
+ENV LOGURU_FORMAT="<green>{time:YYYY-DD-MM HH:mm:ss.SSS}</green> {level: <5} [mynetconf] <lvl>{message}</lvl>"
+ENV LOGURU_COLORIZE=True
+
+COPY entrypoint.sh common.sh configure-*.sh reconfigure-*.sh /opt/bin/
 
 CMD /opt/bin/entrypoint.sh
diff --git a/test/mocks/netconf-pnp-simulator/engine/common.sh b/test/mocks/netconf-pnp-simulator/engine/common.sh
new file mode 100644 (file)
index 0000000..6e938e7
--- /dev/null
@@ -0,0 +1,121 @@
+#!/bin/ash
+# shellcheck disable=SC2086
+
+# ============LICENSE_START=======================================================
+#  Copyright (C) 2020 Nordix Foundation.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+# ============LICENSE_END=========================================================
+
+set -o errexit
+set -o pipefail
+set -o nounset
+[ "${SHELL_XTRACE:-false}" = "true" ] && set -o xtrace
+
+export PATH=/opt/bin:/usr/local/bin:/usr/bin:/bin
+
+CONFIG=/config
+TEMPLATES=/templates
+
+PROC_NAME=${0##*/}
+PROC_NAME=${PROC_NAME%.sh}
+
+function now_ms() {
+    # Requires coreutils package
+    date +"%Y-%m-%d %H:%M:%S.%3N"
+}
+
+function log() {
+    local level=$1
+    shift
+    local message="$*"
+    >&2 printf "%s %-5s [%s] %s\n" "$(now_ms)" $level $PROC_NAME "$message"
+}
+
+find_file() {
+  local dir=$1
+  shift
+  for app in "$@"; do
+    if [ -f $dir/$app ]; then
+      echo -n $dir/$app
+      break
+    fi
+  done
+}
+
+
+# Extracts the body of a PEM file by removing the dashed header and footer
+pem_body() {
+    grep -Fv -- ----- "$1"
+}
+
+
+# ------------------------------------
+# SSH Common Definitions and Functions
+# ------------------------------------
+
+SSH_CONFIG=$CONFIG/ssh
+
+configure_ssh() {
+    local datastore=$1
+    local operation=$2
+    local dir=$3
+
+    log INFO Configure SSH ingress service
+    ssh_pubkey=$(find_file $SSH_CONFIG id_ecdsa.pub id_dsa.pub id_rsa.pub)
+    test -n "$ssh_pubkey"
+    name=${ssh_pubkey##*/}
+    name=${name%%.pub}
+    set -- $(cat $ssh_pubkey)
+    xmlstarlet ed --pf --omit-decl \
+        --update '//_:name[text()="netconf"]/following-sibling::_:authorized-key/_:name' --value "$name" \
+        --update '//_:name[text()="netconf"]/following-sibling::_:authorized-key/_:algorithm' --value "$1" \
+        --update '//_:name[text()="netconf"]/following-sibling::_:authorized-key/_:key-data' --value "$2" \
+        $dir/load_auth_pubkey.xml | \
+    sysrepocfg --datastore=$datastore --permanent --format=xml ietf-system --${operation}=-
+}
+
+
+# ------------------------------------
+# SSL Common Definitions and Functions
+# ------------------------------------
+
+TLS_CONFIG=$CONFIG/tls
+KEY_PATH=/opt/etc/keystored/keys
+
+configure_tls() {
+    local datastore=$1
+    local operation=$2
+    local dir=$3
+
+    log INFO Update server private key
+    cp $TLS_CONFIG/server_key.pem $KEY_PATH
+
+    log INFO Load CA and server certificates
+    ca_cert=$(pem_body $TLS_CONFIG/ca.pem)
+    server_cert=$(pem_body $TLS_CONFIG/server_cert.pem)
+    xmlstarlet ed --pf --omit-decl \
+        --update '//_:name[text()="server_cert"]/following-sibling::_:certificate' --value "$server_cert" \
+        --update '//_:name[text()="ca"]/following-sibling::_:certificate' --value "$ca_cert" \
+        $dir/load_server_certs.xml | \
+    sysrepocfg --datastore=$datastore --permanent --format=xml ietf-keystore --${operation}=-
+
+    log INFO Configure TLS ingress service
+    ca_fingerprint=$(openssl x509 -noout -fingerprint -in $TLS_CONFIG/ca.pem | cut -d= -f2)
+    xmlstarlet ed --pf --omit-decl \
+        --update '//_:name[text()="netconf"]/preceding-sibling::_:fingerprint' --value "02:$ca_fingerprint" \
+        $dir/tls_listen.xml | \
+    sysrepocfg --datastore=$datastore --permanent --format=xml ietf-netconf-server --${operation}=-
+}
diff --git a/test/mocks/netconf-pnp-simulator/engine/config/ssh/load_auth_pubkey.xml b/test/mocks/netconf-pnp-simulator/engine/config/ssh/load_auth_pubkey.xml
deleted file mode 100644 (file)
index 4f35c2f..0000000
+++ /dev/null
@@ -1,12 +0,0 @@
-<system xmlns="urn:ietf:params:xml:ns:yang:ietf-system">
-  <authentication>
-    <user>
-      <name>netconf</name>
-      <authorized-key>
-        <name>id_rsa</name>
-        <algorithm>ssh-rsa</algorithm>
-        <key-data>AAAAB3NzaC1yc2EAAAADAQABAAABAQD4pCY/jetSrsN3ToQwSIopEwDEFps7l327brjqp8a0vXmGuNztvnNDuQyGD5lKLDeK+dGSz+aHdCnD/10rIVSVxnw/TCyGWGHsYHpXqK0ZdiJ6HXX8FqGylTJZWTiSLSDrUwk8Mq8uIk3Sdy5E9yGgKcmA5GInBQuqMhZbzt1KhLhyp67+dIJ+D3b/JzSyPRHt9XMBpGTYMEuhjBM2aH5C9pltrmRq2NIF/cST1eidhTV2wMSqGm9jwDG7CwxAeYvan1cazZIrIfY7a/rD3sbxSPlzH92nUhw8m0qneKjDWO+kzCJVlWQ/q9c6hg69N2tBctYel3WTFw1usbbG/ZCF</key-data>
-      </authorized-key>
-    </user>
-  </authentication>
-</system>
diff --git a/test/mocks/netconf-pnp-simulator/engine/config/tls/ca.pem b/test/mocks/netconf-pnp-simulator/engine/config/tls/ca.pem
new file mode 100644 (file)
index 0000000..62593ab
--- /dev/null
@@ -0,0 +1,24 @@
+-----BEGIN CERTIFICATE-----
+MIID7TCCAtWgAwIBAgIJAMtE1NGAR5KoMA0GCSqGSIb3DQEBBQUAMIGMMQswCQYD
+VQQGEwJDWjEWMBQGA1UECAwNU291dGggTW9yYXZpYTENMAsGA1UEBwwEQnJubzEP
+MA0GA1UECgwGQ0VTTkVUMQwwCgYDVQQLDANUTUMxEzARBgNVBAMMCmV4YW1wbGUg
+Q0ExIjAgBgkqhkiG9w0BCQEWE2V4YW1wbGVjYUBsb2NhbGhvc3QwHhcNMTQwNzI0
+MTQxOTAyWhcNMjQwNzIxMTQxOTAyWjCBjDELMAkGA1UEBhMCQ1oxFjAUBgNVBAgM
+DVNvdXRoIE1vcmF2aWExDTALBgNVBAcMBEJybm8xDzANBgNVBAoMBkNFU05FVDEM
+MAoGA1UECwwDVE1DMRMwEQYDVQQDDApleGFtcGxlIENBMSIwIAYJKoZIhvcNAQkB
+FhNleGFtcGxlY2FAbG9jYWxob3N0MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB
+CgKCAQEArD3TDHPAMT2Z84orK4lMlarbgooIUCcRZyLe+QM+8KY8Hn+mGaxPEOTS
+L3ywszqefB/Utm2hPKLHX684iRC14ID9WDGHxPjvoPArhgFhfV+qnPfxKTgxZC12
+uOj4u1V9y+SkTCocFbRfXVBGpojrBuDHXkDMDEWNvr8/52YCv7bGaiBwUHolcLCU
+bmtKILCG0RNJyTaJpXQdAeq5Z1SJotpbfYFFtAXB32hVoLug1dzl2tjG9sb1wq3Q
+aDExcbC5w6P65qOkNoyym9ne6QlQagCqVDyFn3vcqkRaTjvZmxauCeUxXgJoXkyW
+cm0lM1KMHdoTArmchw2Dz0yHHSyDAQIDAQABo1AwTjAdBgNVHQ4EFgQUc1YQIqjZ
+sHVwlea0AB4N+ilNI2gwHwYDVR0jBBgwFoAUc1YQIqjZsHVwlea0AB4N+ilNI2gw
+DAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQUFAAOCAQEAI/1KH60qnw9Xs2RGfi0/
+IKf5EynXt4bQX8EIyVKwSkYKe04zZxYfLIl/Q2HOPYoFmm3daj5ddr0ZS1i4p4fT
+UhstjsYWvXs3W/HhVmFUslakkn3PrswhP77fCk6eEJLxdfyJ1C7Uudq2m1isZbKi
+h+XF0mG1LxJaDMocSz4eAya7M5brwjy8DoOmA1TnLQFCVcpn+sCr7VC4wE/JqxyV
+hBCk/MuGqqM3B1j90bGFZ112ZOecyE0EDSr6IbiRBtmeNbEwOFjKXhNLYdxpBZ9D
+8A/368OckZkCrVLGuJNxK9UwCVTe8IhotHUqU9EqFDmxdV8oIdU/OzUwwNPA/Bd/
+9g==
+-----END CERTIFICATE-----
diff --git a/test/mocks/netconf-pnp-simulator/engine/config/tls/load_server_certs.xml b/test/mocks/netconf-pnp-simulator/engine/config/tls/load_server_certs.xml
deleted file mode 100644 (file)
index 8872a8e..0000000
+++ /dev/null
@@ -1,62 +0,0 @@
-<keystore xmlns="urn:ietf:params:xml:ns:yang:ietf-keystore">
-  <private-keys>
-    <private-key>
-      <name>server_key</name>
-      <certificate-chains>
-        <certificate-chain>
-          <name>server_cert</name>
-          <certificate>MIIECTCCAvGgAwIBAgIBCDANBgkqhkiG9w0BAQsFADCBjDELMAkGA1UEBhMCQ1ox
-FjAUBgNVBAgMDVNvdXRoIE1vcmF2aWExDTALBgNVBAcMBEJybm8xDzANBgNVBAoM
-BkNFU05FVDEMMAoGA1UECwwDVE1DMRMwEQYDVQQDDApleGFtcGxlIENBMSIwIAYJ
-KoZIhvcNAQkBFhNleGFtcGxlY2FAbG9jYWxob3N0MB4XDTE1MDczMDA3MjU1MFoX
-DTM1MDcyNTA3MjU1MFowgYUxCzAJBgNVBAYTAkNaMRYwFAYDVQQIDA1Tb3V0aCBN
-b3JhdmlhMQ8wDQYDVQQKDAZDRVNORVQxDDAKBgNVBAsMA1RNQzEXMBUGA1UEAwwO
-ZXhhbXBsZSBzZXJ2ZXIxJjAkBgkqhkiG9w0BCQEWF2V4YW1wbGVzZXJ2ZXJAbG9j
-YWxob3N0MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAsdI1TBjzX1Pg
-QXFuPCw5/kQwU7qkrhirMcFAXhI8EoXepPa9fKAVuMjHW32P6nNzDpnhFe0YGdNl
-oIEN3hJJ87cVOqj4o7zZMbq3zVG2L8As7MTA8tYXm2fSC/0rIxxRRemcGUXM0q+4
-LEACjZj2pOKonaivF5VbhgNjPCO1Jj/TamUc0aViE577C9L9EiObGM+bGbabWk/K
-WKLsvxUc+sKZXaJ7psTVgpggJAkUszlmwOQgFiMSR53E9/CAkQYhzGVCmH44Vs6H
-zs3RZjOTbce4wr4ongiA5LbPeSNSCFjy9loKpaE1rtOjkNBVdiNPCQTmLuODXUTK
-gkeL+9v/OwIDAQABo3sweTAJBgNVHRMEAjAAMCwGCWCGSAGG+EIBDQQfFh1PcGVu
-U1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQU83qEtQDFzDvLoaII
-vqiU6k7j1uswHwYDVR0jBBgwFoAUc1YQIqjZsHVwlea0AB4N+ilNI2gwDQYJKoZI
-hvcNAQELBQADggEBAJ+QOLi4gPWGofMkLTqSsbv5xRvTw0xa/sJnEeiejtygAu3o
-McAsyevSH9EYVPCANxzISPzd9SFaO56HxWgcxLn9vi8ZNvo2wIp9zucNu285ced1
-K/2nDZfBmvBxXnj/n7spwqOyuoIc8sR7P7YyI806Qsfhk3ybNZE5UHJFZKDRQKvR
-J1t4nk9saeo87kIuNEDfYNdwYZzRfXoGJ5qIJQK+uJJv9noaIhfFowDW/G14Ji5p
-Vh/YtvnOPh7aBjOj8jmzk8MqzK+TZgT7GWu48Nd/NaV8g/DNg9hlN047LaNsJly3
-NX3+VBlpMnA4rKwl1OnmYSirIVh9RJqNwqe6k/k=</certificate>
-        </certificate-chain>
-      </certificate-chains>
-    </private-key>
-  </private-keys>
-  <trusted-certificates>
-    <name>trusted_ca_list</name>
-    <trusted-certificate>
-      <name>ca</name>
-      <certificate>MIID7TCCAtWgAwIBAgIJAMtE1NGAR5KoMA0GCSqGSIb3DQEBBQUAMIGMMQswCQYD
-VQQGEwJDWjEWMBQGA1UECAwNU291dGggTW9yYXZpYTENMAsGA1UEBwwEQnJubzEP
-MA0GA1UECgwGQ0VTTkVUMQwwCgYDVQQLDANUTUMxEzARBgNVBAMMCmV4YW1wbGUg
-Q0ExIjAgBgkqhkiG9w0BCQEWE2V4YW1wbGVjYUBsb2NhbGhvc3QwHhcNMTQwNzI0
-MTQxOTAyWhcNMjQwNzIxMTQxOTAyWjCBjDELMAkGA1UEBhMCQ1oxFjAUBgNVBAgM
-DVNvdXRoIE1vcmF2aWExDTALBgNVBAcMBEJybm8xDzANBgNVBAoMBkNFU05FVDEM
-MAoGA1UECwwDVE1DMRMwEQYDVQQDDApleGFtcGxlIENBMSIwIAYJKoZIhvcNAQkB
-FhNleGFtcGxlY2FAbG9jYWxob3N0MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB
-CgKCAQEArD3TDHPAMT2Z84orK4lMlarbgooIUCcRZyLe+QM+8KY8Hn+mGaxPEOTS
-L3ywszqefB/Utm2hPKLHX684iRC14ID9WDGHxPjvoPArhgFhfV+qnPfxKTgxZC12
-uOj4u1V9y+SkTCocFbRfXVBGpojrBuDHXkDMDEWNvr8/52YCv7bGaiBwUHolcLCU
-bmtKILCG0RNJyTaJpXQdAeq5Z1SJotpbfYFFtAXB32hVoLug1dzl2tjG9sb1wq3Q
-aDExcbC5w6P65qOkNoyym9ne6QlQagCqVDyFn3vcqkRaTjvZmxauCeUxXgJoXkyW
-cm0lM1KMHdoTArmchw2Dz0yHHSyDAQIDAQABo1AwTjAdBgNVHQ4EFgQUc1YQIqjZ
-sHVwlea0AB4N+ilNI2gwHwYDVR0jBBgwFoAUc1YQIqjZsHVwlea0AB4N+ilNI2gw
-DAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQUFAAOCAQEAI/1KH60qnw9Xs2RGfi0/
-IKf5EynXt4bQX8EIyVKwSkYKe04zZxYfLIl/Q2HOPYoFmm3daj5ddr0ZS1i4p4fT
-UhstjsYWvXs3W/HhVmFUslakkn3PrswhP77fCk6eEJLxdfyJ1C7Uudq2m1isZbKi
-h+XF0mG1LxJaDMocSz4eAya7M5brwjy8DoOmA1TnLQFCVcpn+sCr7VC4wE/JqxyV
-hBCk/MuGqqM3B1j90bGFZ112ZOecyE0EDSr6IbiRBtmeNbEwOFjKXhNLYdxpBZ9D
-8A/368OckZkCrVLGuJNxK9UwCVTe8IhotHUqU9EqFDmxdV8oIdU/OzUwwNPA/Bd/
-9g==</certificate>
-    </trusted-certificate>
-  </trusted-certificates>
-</keystore>
diff --git a/test/mocks/netconf-pnp-simulator/engine/config/tls/netopeer2-client.sh b/test/mocks/netconf-pnp-simulator/engine/config/tls/netopeer2-client.sh
new file mode 100755 (executable)
index 0000000..535f3fe
--- /dev/null
@@ -0,0 +1,95 @@
+#!/bin/bash
+
+# ============LICENSE_START=======================================================
+#  Copyright (C) 2020 Nordix Foundation.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+# ============LICENSE_END=========================================================
+
+# Performs a smoke-test of the NETCONF-Pnp-Simulator by establishing a TLS
+# connection and sending a dummy NETCONF Hello Message.
+
+set -euxo pipefail
+
+SERVER_HOST=localhost
+SERVER_PORT=6513
+
+SCRIPT_PATH=$(dirname $(realpath -s $0))
+
+CLIENT_CERT="
+-----BEGIN CERTIFICATE-----
+MIIECTCCAvGgAwIBAgIBBzANBgkqhkiG9w0BAQsFADCBjDELMAkGA1UEBhMCQ1ox
+FjAUBgNVBAgMDVNvdXRoIE1vcmF2aWExDTALBgNVBAcMBEJybm8xDzANBgNVBAoM
+BkNFU05FVDEMMAoGA1UECwwDVE1DMRMwEQYDVQQDDApleGFtcGxlIENBMSIwIAYJ
+KoZIhvcNAQkBFhNleGFtcGxlY2FAbG9jYWxob3N0MB4XDTE1MDczMDA3MjcxOFoX
+DTM1MDcyNTA3MjcxOFowgYUxCzAJBgNVBAYTAkNaMRYwFAYDVQQIDA1Tb3V0aCBN
+b3JhdmlhMQ8wDQYDVQQKDAZDRVNORVQxDDAKBgNVBAsMA1RNQzEXMBUGA1UEAwwO
+ZXhhbXBsZSBjbGllbnQxJjAkBgkqhkiG9w0BCQEWF2V4YW1wbGVjbGllbnRAbG9j
+YWxob3N0MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAueCQaNQWoNmF
+K6LKu1p8U8ZWdWg/PvDdLsJyzfzl/Qw4UA68SfFNaY06zZl8QB9W02nr5kWeeMY0
+VA3adrPgOlvfx3oWlFbkETnMaN4OT3WTQ0Wt6jAWZDzVfopwpJPAzRPxACDftIqF
+GagYcF32hZlVNqqnVdbXh0S0EViweqp/dbG4VDUHSNVbglc+u4UbEzNIFXMdEFsJ
+ZpkynOmSiTsIATqIhb+2srkVgLwhfkC2qkuHQwAHdubuB07ObM2z01UhyEdDvEYG
+HwtYAGDBL2TAcsI0oGeVkRyuOkV0QY0UN7UEFI1yTYw+xZ42HgFx3uGwApCImxhb
+j69GBYWFqwIDAQABo3sweTAJBgNVHRMEAjAAMCwGCWCGSAGG+EIBDQQfFh1PcGVu
+U1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUXGpLeLnh2cSDARAV
+A7KrBxGYpo8wHwYDVR0jBBgwFoAUc1YQIqjZsHVwlea0AB4N+ilNI2gwDQYJKoZI
+hvcNAQELBQADggEBAJPV3RTXFRtNyOU4rjPpYeBAIAFp2aqGc4t2J1c7oPp/1n+l
+ZvjnwtlJpZHxMM783e2ryDQ6dkvXDf8kpwKlg3U3mkJ3xKkDdWrM4QwghXdCN519
+aa9qmu0zdFL+jUAaWlQ5tsceOrvbusCcbMqiFGk/QfpHqPv52SVWbYyUx7IX7DE+
+UjgsLHycfV/tlcx4ZE6soTzl9VdgSL/zmzG3rjsr58J80rXckLgBhvijgBlIAJvW
+fC7D0vaouvBInSFXymdPVoUDZ30cdGLf+hI/i/TfsEMOinLrXVdkSGNo6FXAHKSv
+XeB9oFKSzhQ7OPyRyqvEPycUSw/qD6FVr80oDDc=
+-----END CERTIFICATE-----
+"
+
+CLIENT_KEY="
+-----BEGIN RSA PRIVATE KEY-----
+MIIEpAIBAAKCAQEAueCQaNQWoNmFK6LKu1p8U8ZWdWg/PvDdLsJyzfzl/Qw4UA68
+SfFNaY06zZl8QB9W02nr5kWeeMY0VA3adrPgOlvfx3oWlFbkETnMaN4OT3WTQ0Wt
+6jAWZDzVfopwpJPAzRPxACDftIqFGagYcF32hZlVNqqnVdbXh0S0EViweqp/dbG4
+VDUHSNVbglc+u4UbEzNIFXMdEFsJZpkynOmSiTsIATqIhb+2srkVgLwhfkC2qkuH
+QwAHdubuB07ObM2z01UhyEdDvEYGHwtYAGDBL2TAcsI0oGeVkRyuOkV0QY0UN7UE
+FI1yTYw+xZ42HgFx3uGwApCImxhbj69GBYWFqwIDAQABAoIBAQCZN9kR8DGu6V7y
+t0Ax68asL8O5B/OKaHWKQ9LqpVrXmikZJOxkbzoGldow/CIFoU+q+Zbwu9aDa65a
+0wiP7Hoa4Py3q5XNNUrOQDyU/OYC7cI0I83WS0lJ2zOJGYj8wKae5Z81IeQFKGHK
+4lsy1OGPAvPRGh7RjUUgRavA2MCwe07rWRuDb/OJFe4Oh56UMEjwMiNBtMNtncog
+j1vr/qgRJdf9tf0zlJmLvUJ9+HSFFV9I/97LJyFhb95gAfHkjdVroLVgT3Cho+4P
+WtZaKCIGD0OwfOG2nLV4leXvRUk62/LMlB8NI9+JF7Xm+HCKbaWHNWC7mvWSLV58
+Zl4AbUWRAoGBANyJ6SFHFRHSPDY026SsdMzXR0eUxBAK7G70oSBKKhY+O1j0ocLE
+jI2krHJBhHbLlnvJVyMUaCUOTS5m0uDw9hgSsAqeSL3hL38kxVZw+KNG9Ouno1Fl
+KnE/xXHlPQyeGs/P8nAMzHZxQtEsQdQayJEhK2XXHTsy7Q3MxDisfVJ1AoGBANfD
+34gB+OMx6pwj7zk3qWbYXSX8xjCZMR0ciko+h4xeMP2N8B0oyoqC+v1ABMAtJ3wG
+sGZd0hV9gwM7OUM3SEwkn6oeg1GemWLcn4rlSmTnZc4aeVwrEWlnSNFX3s4g9l4u
+k8Ugu4MVJYqH8HuDQ5Ggl6/QAwPzMSEdCW0O+jOfAoGAIBRbegC5+t6m7Yegz4Ja
+dxV1g98K6f58x+MDsQu4tYWV4mmrQgaPH2dtwizvlMwmdpkh+LNWNtWuumowkJHc
+akIFo3XExQIFg6wYnGtQb4e5xrGa2xMpKlIJaXjb+YLiCYqJDG2ALFZrTrvuU2kV
+9a5qfqTc1qigvNolTM0iaaUCgYApmrZWhnLUdEKV2wP813PNxfioI4afxlpHD8LG
+sCn48gymR6E+Lihn7vuwq5B+8fYEH1ISWxLwW+RQUjIneNhy/jjfV8TgjyFqg7or
+0Sy4KjpiNI6kLBXOakELRNNMkeSPopGR2E7v5rr3bGD9oAD+aqX1G7oJH/KgPPYd
+Vl7+ZwKBgQDcHyWYrimjyUgKaQD2GmoO9wdcJYQ59ke9K+OuGlp4ti5arsi7N1tP
+B4f09aeELM2ASIuk8Q/Mx0jQFnm8lzRFXdewgvdPoZW/7VufM9O7dGPOc41cm2Dh
+yrTcXx/VmUBb+/fnXVEgCv7gylp/wtdTGHQBQJHR81jFBz0lnLj+gg==
+-----END RSA PRIVATE KEY-----
+"
+
+DUMMY_HELLO='<hello xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"/>]]>]]>'
+
+(echo -n "$DUMMY_HELLO"; sleep 1) | \
+    openssl s_client -connect $SERVER_HOST:$SERVER_PORT \
+    -state \
+    -CAfile $SCRIPT_PATH/ca.pem \
+    -cert <(echo "$CLIENT_CERT") \
+    -key <(echo "$CLIENT_KEY")
diff --git a/test/mocks/netconf-pnp-simulator/engine/config/tls/server_cert.pem b/test/mocks/netconf-pnp-simulator/engine/config/tls/server_cert.pem
new file mode 100644 (file)
index 0000000..c0e03a3
--- /dev/null
@@ -0,0 +1,24 @@
+-----BEGIN CERTIFICATE-----
+MIIECTCCAvGgAwIBAgIBCDANBgkqhkiG9w0BAQsFADCBjDELMAkGA1UEBhMCQ1ox
+FjAUBgNVBAgMDVNvdXRoIE1vcmF2aWExDTALBgNVBAcMBEJybm8xDzANBgNVBAoM
+BkNFU05FVDEMMAoGA1UECwwDVE1DMRMwEQYDVQQDDApleGFtcGxlIENBMSIwIAYJ
+KoZIhvcNAQkBFhNleGFtcGxlY2FAbG9jYWxob3N0MB4XDTE1MDczMDA3MjU1MFoX
+DTM1MDcyNTA3MjU1MFowgYUxCzAJBgNVBAYTAkNaMRYwFAYDVQQIDA1Tb3V0aCBN
+b3JhdmlhMQ8wDQYDVQQKDAZDRVNORVQxDDAKBgNVBAsMA1RNQzEXMBUGA1UEAwwO
+ZXhhbXBsZSBzZXJ2ZXIxJjAkBgkqhkiG9w0BCQEWF2V4YW1wbGVzZXJ2ZXJAbG9j
+YWxob3N0MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAsdI1TBjzX1Pg
+QXFuPCw5/kQwU7qkrhirMcFAXhI8EoXepPa9fKAVuMjHW32P6nNzDpnhFe0YGdNl
+oIEN3hJJ87cVOqj4o7zZMbq3zVG2L8As7MTA8tYXm2fSC/0rIxxRRemcGUXM0q+4
+LEACjZj2pOKonaivF5VbhgNjPCO1Jj/TamUc0aViE577C9L9EiObGM+bGbabWk/K
+WKLsvxUc+sKZXaJ7psTVgpggJAkUszlmwOQgFiMSR53E9/CAkQYhzGVCmH44Vs6H
+zs3RZjOTbce4wr4ongiA5LbPeSNSCFjy9loKpaE1rtOjkNBVdiNPCQTmLuODXUTK
+gkeL+9v/OwIDAQABo3sweTAJBgNVHRMEAjAAMCwGCWCGSAGG+EIBDQQfFh1PcGVu
+U1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQU83qEtQDFzDvLoaII
+vqiU6k7j1uswHwYDVR0jBBgwFoAUc1YQIqjZsHVwlea0AB4N+ilNI2gwDQYJKoZI
+hvcNAQELBQADggEBAJ+QOLi4gPWGofMkLTqSsbv5xRvTw0xa/sJnEeiejtygAu3o
+McAsyevSH9EYVPCANxzISPzd9SFaO56HxWgcxLn9vi8ZNvo2wIp9zucNu285ced1
+K/2nDZfBmvBxXnj/n7spwqOyuoIc8sR7P7YyI806Qsfhk3ybNZE5UHJFZKDRQKvR
+J1t4nk9saeo87kIuNEDfYNdwYZzRfXoGJ5qIJQK+uJJv9noaIhfFowDW/G14Ji5p
+Vh/YtvnOPh7aBjOj8jmzk8MqzK+TZgT7GWu48Nd/NaV8g/DNg9hlN047LaNsJly3
+NX3+VBlpMnA4rKwl1OnmYSirIVh9RJqNwqe6k/k=
+-----END CERTIFICATE-----
diff --git a/test/mocks/netconf-pnp-simulator/engine/config/tls/server_key.pem.pub b/test/mocks/netconf-pnp-simulator/engine/config/tls/server_key.pem.pub
deleted file mode 100644 (file)
index 9ccec4a..0000000
+++ /dev/null
@@ -1,9 +0,0 @@
------BEGIN PUBLIC KEY-----
-MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAsdI1TBjzX1PgQXFuPCw5
-/kQwU7qkrhirMcFAXhI8EoXepPa9fKAVuMjHW32P6nNzDpnhFe0YGdNloIEN3hJJ
-87cVOqj4o7zZMbq3zVG2L8As7MTA8tYXm2fSC/0rIxxRRemcGUXM0q+4LEACjZj2
-pOKonaivF5VbhgNjPCO1Jj/TamUc0aViE577C9L9EiObGM+bGbabWk/KWKLsvxUc
-+sKZXaJ7psTVgpggJAkUszlmwOQgFiMSR53E9/CAkQYhzGVCmH44Vs6Hzs3RZjOT
-bce4wr4ongiA5LbPeSNSCFjy9loKpaE1rtOjkNBVdiNPCQTmLuODXUTKgkeL+9v/
-OwIDAQAB
------END PUBLIC KEY-----
diff --git a/test/mocks/netconf-pnp-simulator/engine/configure-modules.sh b/test/mocks/netconf-pnp-simulator/engine/configure-modules.sh
new file mode 100755 (executable)
index 0000000..2010b50
--- /dev/null
@@ -0,0 +1,95 @@
+#!/bin/ash
+# shellcheck disable=SC2086
+
+# ============LICENSE_START=======================================================
+#  Copyright (C) 2020 Nordix Foundation.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+# ============LICENSE_END=========================================================
+
+set -eu
+
+HERE=${0%/*}
+source $HERE/common.sh
+
+MODELS_CONFIG=$CONFIG/modules
+BASE_VIRTUALENVS=$HOME/.local/share/virtualenvs
+
+install_and_configure_yang_model()
+{
+    local dir=$1
+    local model=$2
+
+    log INFO Importing Yang model \"$model\"
+    yang=$(find_file $dir $model.yang model.yang)
+    sysrepoctl --install --yang=$yang
+    data=$(find_file $dir startup.json startup.xml data.json data.xml)
+    if [ -n "$data" ]; then
+      log INFO Initialing Yang model \"$model\"
+      sysrepocfg --datastore=startup --import=$data $model
+    fi
+}
+
+configure_subscriber_execution()
+{
+  local dir=$1
+  local model=$2
+  local app=$3
+
+  APP_PATH=$PATH
+  if [ -r "$dir/requirements.txt" ]; then
+    env_dir=$(create_python_venv $dir $model)
+    APP_PATH=$env_dir/bin:$APP_PATH
+  fi
+  log INFO Preparing launching of module \"$model\" application
+  cat > /etc/supervisord.d/$model.conf <<EOF
+[program:subs-$model]
+command=$app $model
+stdout_logfile=/dev/stdout
+stdout_logfile_maxbytes=0
+redirect_stderr=true
+autorestart=true
+environment=PATH=$APP_PATH,PYTHONUNBUFFERED="1"
+EOF
+}
+
+create_python_venv()
+{
+  local dir=$1
+  local model=$2
+
+  log INFO Creating virtual environment for module $model
+  mkdir -p $BASE_VIRTUALENVS
+  env_dir=$BASE_VIRTUALENVS/$model
+  (
+    virtualenv --system-site-packages $env_dir
+    cd $env_dir
+    # shellcheck disable=SC1091
+    . ./bin/activate
+    pip install --requirement "$dir"/requirements.txt
+  ) 1>&2
+  echo $env_dir
+}
+
+for dir in "$MODELS_CONFIG"/*; do
+  if [ -d $dir ]; then
+    model=${dir##*/}
+    install_and_configure_yang_model $dir $model
+    app="$dir/subscriber.py"
+    if [ -x "$app" ]; then
+      configure_subscriber_execution $dir $model $app
+    fi
+  fi
+done
index 6636080..378f33b 100755 (executable)
@@ -1,4 +1,4 @@
-#!/bin/sh
+#!/bin/ash
 # shellcheck disable=SC2086
 
 #-
 # SPDX-License-Identifier: Apache-2.0
 # ============LICENSE_END=========================================================
 
-set -o errexit
-set -o nounset
-set -o pipefail
-set -o xtrace
+set -eu
 
-export PATH=/opt/bin:/usr/local/bin:/usr/bin:/bin
+HERE=${0%/*}
+source $HERE/common.sh
 
-CONFIG=/config
-SSH_CONFIG=$CONFIG/ssh
-TLS_CONFIG=$CONFIG/tls
-MODELS_CONFIG=$CONFIG/modules
-KEY_PATH=/opt/etc/keystored/keys
-BASE_VIRTUALENVS=$HOME/.local/share/virtualenvs
+configure_ssh startup merge $TEMPLATES
+configure_tls startup merge $TEMPLATES
 
-find_file() {
-  local dir=$1
-  shift
-  for prog in "$@"; do
-    if [ -f $dir/$prog ]; then
-      echo -n $dir/$prog
-      break
-    fi
-  done
-}
-
-find_executable() {
-  local dir=$1
-  shift
-  for prog in "$@"; do
-    if [ -x $dir/$prog ]; then
-      echo -n $dir/$prog
-      break
-    fi
-  done
-}
-
-configure_ssh()
-{
-  sysrepocfg --datastore=startup --format=xml ietf-system --import=$SSH_CONFIG/load_auth_pubkey.xml
-}
-
-configure_tls()
-{
-  cp $TLS_CONFIG/server_key.pem $KEY_PATH
-  cp $TLS_CONFIG/server_key.pem.pub $KEY_PATH
-  sysrepocfg --datastore=startup --format=xml ietf-keystore --merge=$TLS_CONFIG/load_server_certs.xml
-  sysrepocfg --datastore=startup --format=xml ietf-netconf-server --merge=$TLS_CONFIG/tls_listen.xml
-}
-
-configure_modules()
-{
-  for dir in "$MODELS_CONFIG"/*; do
-    if [ -d $dir ]; then
-      model=${dir##*/}
-      install_and_configure_yang_model $dir $model
-      prog=$(find_executable $dir subscriber.py)
-      if [ -n "$prog" ]; then
-        configure_subscriber_execution $dir $model $prog
-      fi
-    fi
-  done
-}
-
-install_and_configure_yang_model()
-{
-    local dir=$1
-    local model=$2
-
-    yang=$(find_file $dir $model.yang model.yang)
-    sysrepoctl --install --yang=$yang
-    data=$(find_file $dir startup.json startup.xml data.json data.xml)
-    if [ -n "$data" ]; then
-      sysrepocfg --datastore=startup --import=$data $model
-    fi
-}
-
-configure_subscriber_execution()
-{
-  local dir=$1
-  local model=$2
-  local prog=$3
-
-  PROG_PATH=$PATH
-  if [ -r "$dir/requirements.txt" ]; then
-    env_dir=$(create_python_venv $dir)
-    PROG_PATH=$env_dir/bin:$PROG_PATH
-  fi
-  cat > /etc/supervisord.d/$model.conf <<EOF
-[program:subs-$model]
-command=$prog $model
-redirect_stderr=true
-autorestart=true
-environment=PATH=$PROG_PATH,PYTHONUNBUFFERED="1"
-EOF
-}
-
-create_python_venv()
-{
-  local dir=$1
-
-  mkdir -p $BASE_VIRTUALENVS
-  env_dir=$BASE_VIRTUALENVS/$model
-  (
-    python3 -m venv --system-site-packages $env_dir
-    cd $env_dir
-    . ./bin/activate
-    pip install --upgrade pip
-    pip install -r "$dir"/requirements.txt
-  ) 1>&2
-  echo $env_dir
-}
-
-configure_ssh
-configure_tls
-configure_modules
+$HERE/configure-modules.sh
 
 exec /usr/local/bin/supervisord -c /etc/supervisord.conf
diff --git a/test/mocks/netconf-pnp-simulator/engine/patches/Netopeer2/02-zlog.patch b/test/mocks/netconf-pnp-simulator/engine/patches/Netopeer2/02-zlog.patch
new file mode 100644 (file)
index 0000000..804b652
--- /dev/null
@@ -0,0 +1,105 @@
+diff --git a/server/CMakeLists.txt b/server/CMakeLists.txt
+index f0c82c1..99c6a3d 100755
+--- a/server/CMakeLists.txt
++++ b/server/CMakeLists.txt
+@@ -130,6 +130,13 @@ add_library(serverobj OBJECT ${srcs})
+ # netopeer2-server target
+ add_executable(netopeer2-server $<TARGET_OBJECTS:serverobj> main.c)
+
++# dependencies - zlog
++find_library(ZLOG zlog)
++if(NOT ZLOG)
++    message(FATAL_ERROR "Unable to find zlog library.")
++endif()
++target_link_libraries(netopeer2-server ${ZLOG})
++
+ # dependencies - pthread
+ set(CMAKE_THREAD_PREFER_PTHREAD TRUE)
+ find_package(Threads REQUIRED)
+diff --git a/server/log.c b/server/log.c
+index e660635..6b8117b 100644
+--- a/server/log.c
++++ b/server/log.c
+@@ -27,6 +27,8 @@
+ #include <nc_server.h>
+ #include <sysrepo.h>
+
++#include <zlog.h>
++
+ volatile uint8_t np2_verbose_level;
+ uint8_t np2_libssh_verbose_level;
+ uint8_t np2_sr_verbose_level;
+@@ -102,44 +104,24 @@ np2_err_location(void)
+ static void
+ np2log(int priority, const char *fmt, ...)
+ {
+-    char *format;
+     va_list ap;
+
+     va_start(ap, fmt);
+-    vsyslog(priority, fmt, ap);
+-    va_end(ap);
+-
+-    if (np2_stderr_log) {
+-        format = malloc(11 + strlen(fmt) + 2);
+-        if (!format) {
+-            fprintf(stderr, "ERROR: Memory allocation failed (%s:%d)", __FILE__, __LINE__);
+-            return;
+-        }
+-
+-        switch (priority) {
+-        case LOG_ERR:
+-            sprintf(format, "[ERR]: %s\n", fmt);
++    switch (priority) {
++        case LOG_INFO:
++            vdzlog_info(fmt, ap);
+             break;
+         case LOG_WARNING:
+-            sprintf(format, "[WRN]: %s\n", fmt);
+-            break;
+-        case LOG_INFO:
+-            sprintf(format, "[INF]: %s\n", fmt);
++            vdzlog_warn(fmt, ap);
+             break;
+         case LOG_DEBUG:
+-            sprintf(format, "[DBG]: %s\n", fmt);
++            vdzlog_debug(fmt, ap);
+             break;
+         default:
+-            sprintf(format, "[UNKNOWN]: %s\n", fmt);
++            vdzlog_error(fmt, ap);
+             break;
+-        }
+-
+-        va_start(ap, fmt);
+-        vfprintf(stderr, format, ap);
+-        va_end(ap);
+-
+-        free(format);
+     }
++    va_end(ap);
+ }
+
+ /**
+diff --git a/server/main.c b/server/main.c
+index 601e8a8..9d28931 100644
+--- a/server/main.c
++++ b/server/main.c
+@@ -39,6 +39,8 @@
+ #include <nc_server.h>
+ #include <sysrepo.h>
+
++#include <zlog.h>
++
+ #include "common.h"
+ #include "operations.h"
+ #include "netconf_monitoring.h"
+@@ -1545,6 +1547,8 @@ main(int argc, char *argv[])
+     openlog("netopeer2-server", LOG_PID, LOG_DAEMON);
+     np2_stderr_log = 1;
+
++    dzlog_init("/opt/etc/zlog.conf", "netopeer2-server");
++
+     /* process command line options */
+     while ((c = getopt(argc, argv, OPTSTRING)) != -1) {
+         switch (c) {
diff --git a/test/mocks/netconf-pnp-simulator/engine/patches/supervisor/01-std-log-format.patch b/test/mocks/netconf-pnp-simulator/engine/patches/supervisor/01-std-log-format.patch
new file mode 100644 (file)
index 0000000..528a374
--- /dev/null
@@ -0,0 +1,26 @@
+diff --git a/supervisor/loggers.py b/supervisor/loggers.py
+index 84d47ae..d23db3c 100644
+--- a/supervisor/loggers.py
++++ b/supervisor/loggers.py
+@@ -287,7 +287,7 @@ class LogRecord:
+             now = time.time()
+             msecs = (now - long(now)) * 1000
+             part1 = time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(now))
+-            asctime = '%s,%03d' % (part1, msecs)
++            asctime = '%s.%03d' % (part1, msecs)
+             levelname = LOG_LEVELS_BY_NUM[self.level]
+             msg = as_string(self.msg)
+             if self.kw:
+diff --git a/supervisor/options.py b/supervisor/options.py
+index 4e98340..fc19300 100644
+--- a/supervisor/options.py
++++ b/supervisor/options.py
+@@ -1463,7 +1463,7 @@ class ServerOptions(Options):
+
+     def make_logger(self):
+         # must be called after realize() and after supervisor does setuid()
+-        format = '%(asctime)s %(levelname)s %(message)s\n'
++        format = '%(asctime)s %(levelname)-5s [supervisor] %(message)s\n'
+         self.logger = loggers.getLogger(self.loglevel)
+         if self.nodaemon:
+             loggers.handle_stdout(self.logger, format)
diff --git a/test/mocks/netconf-pnp-simulator/engine/patches/sysrepo/02-zlog.patch b/test/mocks/netconf-pnp-simulator/engine/patches/sysrepo/02-zlog.patch
new file mode 100644 (file)
index 0000000..0223563
--- /dev/null
@@ -0,0 +1,172 @@
+diff --git a/CMakeLists.txt b/CMakeLists.txt
+index 14c8467..5af087e 100644
+--- a/CMakeLists.txt
++++ b/CMakeLists.txt
+@@ -105,6 +105,11 @@ configure_file(${PROJECT_SOURCE_DIR}/inc/sysrepo/values.h.in ${PROJECT_BINARY_DI
+ configure_file(${PROJECT_SOURCE_DIR}/inc/sysrepo/xpath.h ${PROJECT_BINARY_DIR}/inc/sysrepo/xpath.h COPYONLY)
+
+ # find required libraries
++find_library(ZLOG zlog)
++if(NOT ZLOG)
++    message(FATAL_ERROR "zlog must be installed.")
++endif()
++
+ find_package(EV REQUIRED)
+ include_directories(${EV_INCLUDE_DIR})
+
+diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
+index 342ad9d..d026a81 100644
+--- a/src/CMakeLists.txt
++++ b/src/CMakeLists.txt
+@@ -85,9 +85,9 @@ add_dependencies(SR_SRC COMMON)
+ add_dependencies(SR_ENGINE COMMON)
+
+ if(USE_AVL_LIB)
+-    set(LINK_LIBRARIES pthread ${AVL_LIBRARIES} ${EV_LIBRARIES} ${PROTOBUF-C_LIBRARIES} ${YANG_LIBRARIES})
++    set(LINK_LIBRARIES pthread ${AVL_LIBRARIES} ${EV_LIBRARIES} ${PROTOBUF-C_LIBRARIES} ${YANG_LIBRARIES} ${ZLOG})
+ else(USE_AVL_LIB)
+-    set(LINK_LIBRARIES pthread ${REDBLACK_LIBRARIES} ${EV_LIBRARIES} ${PROTOBUF-C_LIBRARIES} ${YANG_LIBRARIES})
++    set(LINK_LIBRARIES pthread ${REDBLACK_LIBRARIES} ${EV_LIBRARIES} ${PROTOBUF-C_LIBRARIES} ${YANG_LIBRARIES} ${ZLOG})
+ endif(USE_AVL_LIB)
+
+ #handle rt library that doesn't exist on OS X
+diff --git a/src/common/sr_logger.c b/src/common/sr_logger.c
+index 8dd6f31..ea94044 100644
+--- a/src/common/sr_logger.c
++++ b/src/common/sr_logger.c
+@@ -29,6 +29,8 @@
+ #include <stdarg.h>
+ #include <pthread.h>
+
++#include <zlog.h>
++
+ #include "sr_common.h"
+ #include "sr_logger.h"
+
+@@ -76,6 +78,7 @@ void
+ sr_logger_init(const char *app_name)
+ {
+ #if SR_LOGGING_ENABLED
++    dzlog_init("/opt/etc/zlog.conf", app_name);
+     if (NULL != sr_syslog_identifier) {
+         /* if some syslog identifier was already set, release it as we are going to set new one */
+         free((char*)sr_syslog_identifier);
+diff --git a/src/common/sr_logger.h b/src/common/sr_logger.h
+index 37c3487..c95a68d 100644
+--- a/src/common/sr_logger.h
++++ b/src/common/sr_logger.h
+@@ -31,6 +31,8 @@
+ #include <syslog.h>
+ #include <pthread.h>
+
++#include <zlog.h>
++
+ #include "sr_constants.h"
+
+ /**
+@@ -156,37 +158,31 @@ extern __thread char strerror_buf [SR_MAX_STRERROR_LEN]; /**< thread local buffe
+ /**
+  * Internal output macro
+  */
+-#define SR_LOG__INTERNAL(LL, MSG, ...) \
+-    do { \
+-        if (sr_ll_stderr >= LL) \
+-            SR_LOG__STDERR(LL, MSG, __VA_ARGS__) \
+-        if (sr_ll_syslog >= LL) \
+-            SR_LOG__SYSLOG(LL, MSG, __VA_ARGS__) \
+-        if (NULL != sr_log_callback) \
+-            SR_LOG__CALLBACK(LL, MSG, __VA_ARGS__) \
+-    } while(0)
+-
+ #if SR_LOGGING_ENABLED
+
+ /** Prints an error message (with format specifiers). */
+-#define SR_LOG_ERR(MSG, ...) SR_LOG__INTERNAL(SR_LL_ERR, MSG, __VA_ARGS__)
++#define SR_LOG_ERR(MSG, ...) dzlog(__FILE__, sizeof(__FILE__)-1, __func__, sizeof(__func__)-1, \
++        __LINE__, ZLOG_LEVEL_ERROR, MSG, __VA_ARGS__)
+ /** Prints an error message. */
+-#define SR_LOG_ERR_MSG(MSG) SR_LOG__INTERNAL(SR_LL_ERR, MSG "%s", "")
++#define SR_LOG_ERR_MSG(MSG) SR_LOG_ERR(MSG "%s", "")
+
+ /** Prints a warning message (with format specifiers). */
+-#define SR_LOG_WRN(MSG, ...) SR_LOG__INTERNAL(SR_LL_WRN, MSG, __VA_ARGS__)
++#define SR_LOG_WRN(MSG, ...) dzlog(__FILE__, sizeof(__FILE__)-1, __func__, sizeof(__func__)-1, \
++        __LINE__, ZLOG_LEVEL_WARN, MSG, __VA_ARGS__)
+ /** Prints a warning message. */
+-#define SR_LOG_WRN_MSG(MSG) SR_LOG__INTERNAL(SR_LL_WRN, MSG "%s", "")
++#define SR_LOG_WRN_MSG(MSG) SR_LOG_WRN(MSG "%s", "")
+
+ /** Prints an informational message (with format specifiers). */
+-#define SR_LOG_INF(MSG, ...) SR_LOG__INTERNAL(SR_LL_INF, MSG, __VA_ARGS__)
++#define SR_LOG_INF(MSG, ...) dzlog(__FILE__, sizeof(__FILE__)-1, __func__, sizeof(__func__)-1, \
++        __LINE__, ZLOG_LEVEL_INFO, MSG, __VA_ARGS__)
+ /** Prints an informational message. */
+-#define SR_LOG_INF_MSG(MSG) SR_LOG__INTERNAL(SR_LL_INF, MSG "%s", "")
++#define SR_LOG_INF_MSG(MSG) SR_LOG_INF(MSG "%s", "")
+
+ /** Prints a development debug message (with format specifiers). */
+-#define SR_LOG_DBG(MSG, ...) SR_LOG__INTERNAL(SR_LL_DBG, MSG, __VA_ARGS__)
++#define SR_LOG_DBG(MSG, ...) dzlog(__FILE__, sizeof(__FILE__)-1, __func__, sizeof(__func__)-1, \
++        __LINE__, ZLOG_LEVEL_DEBUG, MSG, __VA_ARGS__)
+ /** Prints a development debug message. */
+-#define SR_LOG_DBG_MSG(MSG) SR_LOG__INTERNAL(SR_LL_DBG, MSG "%s", "")
++#define SR_LOG_DBG_MSG(MSG) SR_LOG_DBG(MSG "%s", "")
+
+ #else
+ #define SR_LOG_ERR(...)
+diff --git a/src/executables/sysrepocfg.c b/src/executables/sysrepocfg.c
+index 0000951..f48ed5e 100644
+--- a/src/executables/sysrepocfg.c
++++ b/src/executables/sysrepocfg.c
+@@ -2000,6 +2000,9 @@ main(int argc, char* argv[])
+         }
+     }
+
++    /* init logger */
++    sr_logger_init("sysrepocfg");
++
+     /* set log levels */
+     sr_log_stderr(SR_LL_ERR);
+     sr_log_syslog(SR_LL_NONE);
+diff --git a/src/executables/sysrepoctl.c b/src/executables/sysrepoctl.c
+index 3b02e7d..60ffd7e 100644
+--- a/src/executables/sysrepoctl.c
++++ b/src/executables/sysrepoctl.c
+@@ -1311,6 +1311,9 @@ main(int argc, char* argv[])
+         search_dir_count = 1;
+     }
+
++    /* init logger */
++    sr_logger_init("sysrepoctl");
++
+     /* set log levels */
+     sr_log_stderr(SR_LL_ERR);
+     sr_log_syslog(SR_LL_NONE);
+diff --git a/src/clientlib/client_library.c b/src/clientlib/client_library.c
+index c3da2e5..b3beab7 100644
+--- a/src/clientlib/client_library.c
++++ b/src/clientlib/client_library.c
+@@ -377,6 +377,11 @@ sr_connect(const char *app_name, const sr_conn_options_t opts, sr_conn_ctx_t **c
+
+     CHECK_NULL_ARG2(app_name, conn_ctx_p);
+
++    if (0 == connections_cnt) {
++        /* this is the first connection - initialize logging */
++        sr_logger_init(app_name);
++    }
++
+     SR_LOG_DBG_MSG("Connecting to Sysrepo Engine.");
+
+     /* create the connection */
+@@ -385,11 +390,6 @@ sr_connect(const char *app_name, const sr_conn_options_t opts, sr_conn_ctx_t **c
+
+     pthread_mutex_lock(&global_lock);
+
+-    if (0 == connections_cnt) {
+-        /* this is the first connection - initialize logging */
+-        sr_logger_init(app_name);
+-    }
+-
+     /* attempt to connect to sysrepo daemon socket */
+     rc = cl_socket_connect(connection, SR_DAEMON_SOCKET);
+     if (SR_ERR_OK != rc) {
diff --git a/test/mocks/netconf-pnp-simulator/engine/reconfigure-ssh.sh b/test/mocks/netconf-pnp-simulator/engine/reconfigure-ssh.sh
new file mode 100755 (executable)
index 0000000..2634dc1
--- /dev/null
@@ -0,0 +1,37 @@
+#!/bin/ash
+# shellcheck disable=SC2086
+
+# ============LICENSE_START=======================================================
+#  Copyright (C) 2020 Nordix Foundation.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+# ============LICENSE_END=========================================================
+
+set -eu
+
+HERE=${0%/*}
+source $HERE/common.sh
+
+SSH_CONFIG=$CONFIG/ssh
+
+WORKDIR=$(mktemp -d)
+trap "rm -rf $WORKDIR" EXIT
+
+sysrepocfg --format=xml --export=$WORKDIR/load_auth_pubkey.xml ietf-system
+configure_ssh running import $WORKDIR
+
+pid=$(cat /var/run/netopeer2-server.pid)
+log INFO Restart Netopeer2 pid=$pid
+kill $pid
diff --git a/test/mocks/netconf-pnp-simulator/engine/reconfigure-tls.sh b/test/mocks/netconf-pnp-simulator/engine/reconfigure-tls.sh
new file mode 100755 (executable)
index 0000000..6c97064
--- /dev/null
@@ -0,0 +1,36 @@
+#!/bin/ash
+# shellcheck disable=SC2086
+
+# ============LICENSE_START=======================================================
+#  Copyright (C) 2020 Nordix Foundation.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+# ============LICENSE_END=========================================================
+
+set -eu
+
+HERE=${0%/*}
+source $HERE/common.sh
+
+WORKDIR=$(mktemp -d)
+trap "rm -rf $WORKDIR" EXIT
+
+sysrepocfg --format=xml --export=$WORKDIR/load_server_certs.xml ietf-keystore
+sysrepocfg --format=xml --export=$WORKDIR/tls_listen.xml ietf-netconf-server
+configure_tls running import $WORKDIR
+
+pid=$(cat /var/run/netopeer2-server.pid)
+log INFO Restart Netopeer2 pid=$pid
+kill $pid
index 9e6fd42..980ac36 100644 (file)
 # ============LICENSE_END=========================================================
 
 [supervisord]
+user=root
 nodaemon=true
 logfile=/dev/null
 logfile_maxbytes=0
-loglevel=debug
+loglevel=info
 
 [program:sysrepod]
 command=/opt/bin/sysrepod -d -l3
 autorestart=true
+stdout_logfile=/dev/stdout
+stdout_logfile_maxbytes=0
 redirect_stderr=true
 priority=1
 
 [program:sysrepo-plugind]
 command=/opt/bin/sysrepo-plugind -d -l3
 autorestart=true
+stdout_logfile=/dev/stdout
+stdout_logfile_maxbytes=0
 redirect_stderr=true
 priority=2
 
 [program:netopeer2-server]
 command=/opt/bin/netopeer2-server -d -v3
 autorestart=true
+stdout_logfile=/dev/stdout
+stdout_logfile_maxbytes=0
 redirect_stderr=true
 priority=3
 
diff --git a/test/mocks/netconf-pnp-simulator/engine/templates/load_auth_pubkey.xml b/test/mocks/netconf-pnp-simulator/engine/templates/load_auth_pubkey.xml
new file mode 100644 (file)
index 0000000..93b662f
--- /dev/null
@@ -0,0 +1,12 @@
+<system xmlns="urn:ietf:params:xml:ns:yang:ietf-system">
+  <authentication>
+    <user>
+      <name>netconf</name>
+      <authorized-key>
+        <name></name>
+        <algorithm></algorithm>
+        <key-data></key-data>
+      </authorized-key>
+    </user>
+  </authentication>
+</system>
diff --git a/test/mocks/netconf-pnp-simulator/engine/templates/load_server_certs.xml b/test/mocks/netconf-pnp-simulator/engine/templates/load_server_certs.xml
new file mode 100644 (file)
index 0000000..ef02ded
--- /dev/null
@@ -0,0 +1,20 @@
+<keystore xmlns="urn:ietf:params:xml:ns:yang:ietf-keystore">
+  <private-keys>
+    <private-key>
+      <name>server_key</name>
+      <certificate-chains>
+        <certificate-chain>
+          <name>server_cert</name>
+          <certificate></certificate>
+        </certificate-chain>
+      </certificate-chains>
+    </private-key>
+  </private-keys>
+  <trusted-certificates>
+    <name>trusted_ca_list</name>
+    <trusted-certificate>
+      <name>ca</name>
+      <certificate></certificate>
+    </trusted-certificate>
+  </trusted-certificates>
+</keystore>
@@ -15,7 +15,7 @@
           <cert-maps>
             <cert-to-name>
               <id>1</id>
-              <fingerprint>02:E9:38:1F:F6:8B:62:DE:0A:0B:C5:03:81:A8:03:49:A0:00:7F:8B:F3</fingerprint>
+              <fingerprint></fingerprint>
               <map-type xmlns:x509c2n="urn:ietf:params:xml:ns:yang:ietf-x509-cert-to-name">x509c2n:specified</map-type>
               <name>netconf</name>
             </cert-to-name>
diff --git a/test/mocks/netconf-pnp-simulator/engine/tests/test_ietf_interfaces.py b/test/mocks/netconf-pnp-simulator/engine/tests/test_ietf_interfaces.py
deleted file mode 100644 (file)
index 87733ac..0000000
+++ /dev/null
@@ -1,93 +0,0 @@
-import unittest
-import nctest
-import os
-
-class TestIETFInterfaces(nctest.NCTestCase):
-    """ Tests basic NETCONF operations on the turing-machine YANG module. """
-
-    def __init__(self, *args, **kwargs):
-        super(TestIETFInterfaces, self).__init__(*args, **kwargs)
-        self.ns = {"nc": "urn:ietf:params:xml:ns:netconf:base:1.0", "if": "urn:ietf:params:xml:ns:yang:ietf-interfaces"}
-
-    def test_edit_config(self):
-        config_xml = """<nc:config xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0">
-            <interfaces xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces">
-                <interface nc:operation="{}">
-                    <name>TestInterface</name>
-                    <description>Interface under test</description>
-                    <type xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type">ianaift:ethernetCsmacd</type>
-                    <ipv4 xmlns="urn:ietf:params:xml:ns:yang:ietf-ip">
-                        <mtu>1500</mtu>
-                        <address>
-                            <ip>192.168.2.100</ip>
-                            <prefix-length>24</prefix-length>
-                        </address>
-                    </ipv4>
-                    <ipv6 xmlns="urn:ietf:params:xml:ns:yang:ietf-ip">
-                        <address>
-                            <ip>2001:db8::10</ip>
-                            <prefix-length>32</prefix-length>
-                        </address>
-                    </ipv6>
-                </interface>
-            </interfaces>
-        </nc:config>"""
-
-        filter_xml = """<nc:filter xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0">
-            <interfaces xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces" />
-            </nc:filter>"""
-
-        with_default_report_all = """report-all"""
-
-        # get from running - should be empty
-        reply = self.nc.get_config(source="running", filter=filter_xml)
-        self.check_reply_data(reply)
-        deltas = reply.data.xpath("/nc:rpc-reply/nc:data/if:interfaces/if:interface[if:name='TestInterface']", namespaces=self.ns)
-        self.assertEqual(len(deltas), 0)
-
-        # set data - candidate
-        reply = self.nc.edit_config(target='candidate', config=config_xml.format("merge"))
-        self.check_reply_ok(reply)
-
-        # get from candidate
-        reply = self.nc.get_config(source="candidate", filter=filter_xml)
-        self.check_reply_data(reply)
-        interfaces = reply.data.xpath("/nc:rpc-reply/nc:data/if:interfaces/if:interface[if:name='TestInterface']", namespaces=self.ns)
-        self.assertEqual(len(interfaces), 1)
-
-        # default leaf should NOT be present
-        enabled = reply.data.xpath("/nc:rpc-reply/nc:data/if:interfaces/if:interface[if:name='TestInterface']/enabled", namespaces=self.ns)
-        self.assertEqual(len(enabled), 0)
-
-        # get from candidate with with defaults = 'report-all'
-        reply = self.nc.get_config(source="candidate", filter=filter_xml, with_defaults=with_default_report_all)
-        self.check_reply_data(reply)
-        interfaces = reply.data.xpath("/nc:rpc-reply/nc:data/if:interfaces/if:interface[if:name='TestInterface']", namespaces=self.ns)
-        self.assertEqual(len(interfaces), 1)
-
-        # default leaf should be present
-        enabled = reply.data.xpath("/nc:rpc-reply/nc:data/if:interfaces/if:interface[if:name='TestInterface']/enabled", namespaces=self.ns)
-        self.assertEqual(len(enabled), 0) # TODO: change to 1 once this is implemented
-
-        # get from running - should be empty
-        reply = self.nc.get_config(source="running", filter=filter_xml)
-        self.check_reply_data(reply)
-        deltas = reply.data.xpath("/nc:rpc-reply/nc:data/if:interfaces/if:interface[if:name='TestInterface']", namespaces=self.ns)
-        self.assertEqual(len(deltas), 0)
-
-        # commit - should fail, not enabled in running
-        reply = self.nc.commit()
-        self.check_reply_err(reply)
-
-        # delete from candidate
-        reply = self.nc.edit_config(target='candidate', config=config_xml.format("delete"))
-        self.check_reply_ok(reply)
-
-        # get from candidate - should be empty
-        reply = self.nc.get_config(source="candidate", filter=filter_xml)
-        self.check_reply_data(reply)
-        deltas = reply.data.xpath("/nc:rpc-reply/nc:data/if:interfaces/if:interface[if:name='TestInterface']", namespaces=self.ns)
-        self.assertEqual(len(deltas), 0)
-
-if __name__ == '__main__':
-    unittest.main()
diff --git a/test/mocks/netconf-pnp-simulator/engine/tests/test_turing_machine.py b/test/mocks/netconf-pnp-simulator/engine/tests/test_turing_machine.py
deleted file mode 100644 (file)
index 63a0c2d..0000000
+++ /dev/null
@@ -1,124 +0,0 @@
-import unittest
-import nctest
-import os
-
-class TestTuringMachine(nctest.NCTestCase):
-    """ Tests basic NETCONF operations on the turing-machine YANG module. """
-
-    def __init__(self, *args, **kwargs):
-        super(TestTuringMachine, self).__init__(*args, **kwargs)
-        self.ns = {"nc": "urn:ietf:params:xml:ns:netconf:base:1.0", "tm": "http://example.net/turing-machine"}
-
-    def check_deltas_in_data(self, data):
-        deltas = data.xpath("/nc:rpc-reply/nc:data/tm:turing-machine/tm:transition-function/*", namespaces=self.ns)
-        self.assertNotEqual(len(deltas), 0)
-        for d in deltas:
-            self.assertTrue(d.tag.endswith("delta"))
-
-    def check_labels_only_in_data(self, data):
-        children = data.xpath("/nc:rpc-reply/nc:data/*", namespaces=self.ns)
-        self.assertNotEqual(len(children), 0)
-        for child in children:
-            self.assertTrue(child.tag.endswith("turing-machine"))
-        children = data.xpath("/nc:rpc-reply/nc:data/tm:turing-machine/*", namespaces=self.ns)
-        self.assertNotEqual(len(children), 0)
-        for child in children:
-            self.assertTrue(child.tag.endswith("transition-function"))
-        children = data.xpath("/nc:rpc-reply/nc:data/tm:turing-machine/tm:transition-function/*", namespaces=self.ns)
-        self.assertNotEqual(len(children), 0)
-        for child in children:
-            self.assertTrue(child.tag.endswith("delta"))
-        children = data.xpath("/nc:rpc-reply/nc:data/tm:turing-machine/tm:transition-function/tm:delta/*", namespaces=self.ns)
-        self.assertNotEqual(len(children), 0)
-        for child in children:
-            self.assertTrue(child.tag.endswith("label"))
-
-    def test_get(self):
-        reply = self.nc.get()
-        self.check_reply_data(reply)
-        self.check_deltas_in_data(reply.data)
-
-    def test_get_config_startup(self):
-        reply = self.nc.get_config(source="startup")
-        self.check_reply_data(reply)
-        self.check_deltas_in_data(reply.data)
-
-    def test_get_config_running(self):
-        reply = self.nc.get_config(source="running")
-        self.check_reply_data(reply)
-        self.check_deltas_in_data(reply.data)
-
-    def test_get_subtree_filter(self):
-        filter_xml = """<nc:filter xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0">
-            <turing-machine xmlns="http://example.net/turing-machine">
-                <transition-function>
-                    <delta>
-                        <label />
-                    </delta>
-                </transition-function>
-            </turing-machine>
-            </nc:filter>"""
-        reply = self.nc.get_config(source="running", filter=filter_xml)
-        self.check_reply_data(reply)
-        self.check_deltas_in_data(reply.data)
-        self.check_labels_only_in_data(reply.data)
-
-    def test_get_xpath_filter(self):
-        # https://github.com/ncclient/ncclient/issues/166
-        filter_xml = """<nc:filter type="xpath" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"
-            xmlns:tm="http://example.net/turing-machine"
-            select="/tm:turing-machine/transition-function/delta/label" />
-            """
-        reply = self.nc.get(filter=filter_xml)
-        self.check_reply_data(reply)
-        self.check_deltas_in_data(reply.data)
-        self.check_labels_only_in_data(reply.data)
-
-    @unittest.skipIf(os.environ.get("DOCKER_IMG_TAG") == "latest", "bug in Netopeer2 replace operation")
-    def test_edit_config(self):
-        config_xml = """<nc:config xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0">
-            <turing-machine xmlns="http://example.net/turing-machine">
-                <transition-function>
-                    <delta nc:operation="{}">
-                        <label>test-transition-rule</label>
-                        <input>
-                            <symbol>{}</symbol>
-                            <state>{}</state>
-                        </input>
-                    </delta>
-                </transition-function>
-            </turing-machine></nc:config>"""
-        # merge
-        reply = self.nc.edit_config(target='running', config=config_xml.format("merge", 9, 99))
-        self.check_reply_ok(reply)
-        # get
-        reply = self.nc.get_config(source="running")
-        self.check_reply_data(reply)
-        deltas = reply.data.xpath("/nc:rpc-reply/nc:data/tm:turing-machine/tm:transition-function/tm:delta[tm:label='test-transition-rule']", namespaces=self.ns)
-        self.assertEqual(len(deltas), 1)
-        # create already existing - expect error
-        reply = self.nc.edit_config(target='running', config=config_xml.format("create", 9, 99))
-        self.check_reply_err(reply)
-        # replace
-        reply = self.nc.edit_config(target='running', config=config_xml.format("replace", 9, 88))
-        self.check_reply_ok(reply)
-        # get
-        reply = self.nc.get_config(source="running")
-        self.check_reply_data(reply)
-        states = reply.data.xpath("/nc:rpc-reply/nc:data/tm:turing-machine/tm:transition-function/tm:delta[tm:label='test-transition-rule']/tm:input/tm:state", namespaces=self.ns)
-        self.assertEqual(len(states), 1)
-        self.assertEqual(states[0].text, "88")
-        # delete
-        reply = self.nc.edit_config(target='running', config=config_xml.format("delete", 9, 88))
-        self.check_reply_ok(reply)
-        # delete non-existing - expect error
-        reply = self.nc.edit_config(target='running', config=config_xml.format("delete", 9, 88))
-        self.check_reply_err(reply)
-        # get - should be empty
-        reply = self.nc.get_config(source="running")
-        self.check_reply_data(reply)
-        deltas = reply.data.xpath("/nc:rpc-reply/nc:data/tm:turing-machine/tm:transition-function/tm:delta[tm:label='test-transition-rule']", namespaces=self.ns)
-        self.assertEqual(len(deltas), 0)
-
-if __name__ == '__main__':
-    unittest.main()
diff --git a/test/mocks/netconf-pnp-simulator/engine/zlog.conf b/test/mocks/netconf-pnp-simulator/engine/zlog.conf
new file mode 100644 (file)
index 0000000..0a1c72b
--- /dev/null
@@ -0,0 +1,7 @@
+[formats]
+
+common = "%d(%F %T).%ms %-5V [%c] %m%n"
+
+[rules]
+
+*.INFO >stderr; common