Change lab jobs to pull logs from lab CI systems

It turns out that the "External Job" job type cannot
integrate with the Jenkins Robot plugin.

So, instead of that, this defines the lab jobs to retrieve
the logs (including Robot logs) from the lab CI systems.
These logs can then be post-processed by the Jenkins Robot
plugin as well as get archived to LF Nexus by LF Jenkins,
without the lab CI systems needing to upload logs directly.

These lab jobs will be remotely triggered via REST API calls by
the lab CI systems as the lab CI jobs complete.

Change-Id: I3b793088c459ebb1219aee80fd43cf669a62128a
Issue-ID: INT-278
Signed-off-by: Gary Wu <gary.i.wu@huawei.com>

diff --git a/jjb/lab/include-raw-lab-retrieve-logs.sh b/jjb/lab/include-raw-lab-retrieve-logs.sh
new file mode 100644 (file)
index 0000000..6335e49
--- /dev/null
+++ b/jjb/lab/include-raw-lab-retrieve-logs.sh
@@ -0,0 +1,9 @@
+#!/bin/bash
+set -xe -o pipefail
+
+echo Job triggered by $SRC_BUILD_URL
+echo Retriving logs from $LOG_DIR_URL
+
+rm -rf archives
+wget -r -nv -nd --no-parent -l 1 --reject="index.html*" -P archives "$LOG_DIR_URL"
+cat archives/console.log

diff --git a/jjb/lab/lab-templates.yaml b/jjb/lab/lab-templates.yaml
new file mode 100644 (file)
index 0000000..6d37da5
--- /dev/null
+++ b/jjb/lab/lab-templates.yaml
@@ -0,0 +1,36 @@
+---
+- job-template:
+    name: 'lab-{lab-name}-{env-name}-{job-name}'
+    disabled: false
+    project-type: freestyle
+    build-node: ubuntu1604-basebuild-2c-1g
+    auth-token: lab
+
+    properties:
+      - lf-infra-properties:
+          project: 'integration'
+          build-days-to-keep: '{build-days-to-keep}'
+
+    parameters:
+      - string:
+          name: LOG_DIR_URL
+          description: 'URL for the log directory produced by the external job'
+      - string:
+          name: SRC_BUILD_URL
+          description: 'URL for the external Jenkins build triggering this build'
+
+    wrappers:
+      - lf-infra-wrappers:
+          build-timeout: '{build-timeout}'
+          jenkins-ssh-credential: '{jenkins-ssh-credential}'
+
+    builders:
+      - shell:
+          !include-raw:
+          - include-raw-lab-retrieve-logs.sh
+
+    publishers:
+      - integration-robot:
+          unstable-if: 0.0
+          pass-if: 100.0
+      - lf-infra-publish