reduce ccsdk-ansible-server image footprint 78/77578/1
authorSylvain Desbureaux <sylvain.desbureaux@orange.com>
Wed, 30 Jan 2019 13:36:14 +0000 (14:36 +0100)
committerSylvain Desbureaux <sylvain.desbureaux@orange.com>
Wed, 30 Jan 2019 13:39:24 +0000 (14:39 +0100)
Project "CIA" scope is to provide smaller and multi architecture docker
images.

This proposition aims to do that for ccsdk-ansible-server.

The current size of this proposal is 139mb compared to the 618Mb image
proposed before (and before actual ansible installation).

Principles used are the following:

1. a "basic" `docker build .` should provide a working image;
2. use the smallest (and multi arch) base image possibe;
3. install necessary stuff only;
4. leverage python best practices for installation;
5. remove installation stuff occuring at run level and move them to
   build level.

* Dockerfile is now at the root part;
* ansible configuration file `ansible.cfg` is created and put in
  `configuration` folder;
* ansible configuration file `ansible.cfg` is pushed during the
  build part with a reproductible process;
* `python` files are on `ansible-server" folder so copying during build
  works out of the box;
* yml files are put directly on `ansible-server/Playbooks` so copying
during build works out the box;

* Dockerfile base image is now alpine:3.8;
* `timeout` executable being now from busybox, change in the
  command-line call has been made (`--signal` is now `-s` and `-t` is
  set before setting the timeout value).

* Only install python and python-pip as alpine package and remove build
  dependencies package after use.

* put all python requirements in `requirements.txt` package and launch
  pip installation command with this file.

* call directly python main script (`RestServer.py`) as entrypoint and
  do the python installation stuff at build time.

- [x] change pom.xml so building parts works
- [ ] consider changing destination folder (`/opt/ansible-server/`) to
      the ancient one (`/opt/onap/ccsdk/`) if this is problematic

Change-Id: I5c66f29ca40d9224aef3d230593735b8e0aa1f3b
Issue-ID: INT-816
Signed-off-by: Sylvain Desbureaux <sylvain.desbureaux@orange.com>
16 files changed:
ansible-server/pom.xml
ansible-server/src/main/.dockerignore [new file with mode: 0644]
ansible-server/src/main/Dockerfile [new file with mode: 0644]
ansible-server/src/main/ansible-server/AnsibleModule.py [moved from ansible-server/src/main/scripts/AnsibleModule.py with 93% similarity]
ansible-server/src/main/ansible-server/AnsibleSql.py [new file with mode: 0755]
ansible-server/src/main/ansible-server/BuildHostFile.py [moved from ansible-server/src/main/scripts/BuildHostFile.py with 100% similarity]
ansible-server/src/main/ansible-server/LoadAnsibleMySql.py [new file with mode: 0755]
ansible-server/src/main/ansible-server/Playbooks/Ansible_inventory [moved from ansible-server/src/main/yml/Ansible_inventory with 99% similarity]
ansible-server/src/main/ansible-server/README [moved from ansible-server/src/main/scripts/README with 100% similarity]
ansible-server/src/main/ansible-server/RestServer.py [new file with mode: 0644]
ansible-server/src/main/ansible-server/RestServer_config [moved from ansible-server/src/main/scripts/RestServer_config with 97% similarity]
ansible-server/src/main/ansible-server/UsersRestServer.py [moved from ansible-server/src/main/scripts/UsersRestServer.py with 100% similarity]
ansible-server/src/main/ansible-server/requirements.txt [new file with mode: 0644]
ansible-server/src/main/configuration/ansible.cfg [new file with mode: 0644]
ansible-server/src/main/scripts/RestServer.py [deleted file]
ansible-server/src/main/scripts/startAnsibleServer.sh [deleted file]

index 29b5e63..cccc7d0 100644 (file)
@@ -1,11 +1,12 @@
 <?xml version="1.0" encoding="UTF-8"?>
 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
 
-    <parent>
+   <parent>
         <groupId>org.onap.ccsdk.parent</groupId>
         <artifactId>odlparent-lite</artifactId>
         <version>1.2.1-SNAPSHOT</version>
     </parent>
+
     <modelVersion>4.0.0</modelVersion>
     <packaging>pom</packaging>
     <groupId>org.onap.ccsdk.distribution</groupId>
@@ -18,7 +19,7 @@
     <properties>
         <image.name>onap/ccsdk-ansible-server-image</image.name>
         <ccsdk.project.version>${project.version}</ccsdk.project.version>
-       <ccsdk.distribution.version>${project.version}</ccsdk.distribution.version>
+             <ccsdk.distribution.version>${project.version}</ccsdk.distribution.version>
         <docker.buildArg.https_proxy>${https_proxy}</docker.buildArg.https_proxy>
         <maven.build.timestamp.format>yyyyMMdd'T'HHmmss'Z'</maven.build.timestamp.format>
     </properties>
                             <outputDirectory>${basedir}/target/docker-stage</outputDirectory>
                             <resources>
                                 <resource>
-                                    <directory>src/main/docker</directory>
-                                    <includes>
-                                        <include>Dockerfile</include>
-                                    </includes>
-                                    <filtering>true</filtering>
-                                </resource>
-                            </resources>
-                        </configuration>
-                    </execution>
-
-                    <execution>
-                        <id>copy-scripts</id>
-                        <goals>
-                            <goal>copy-resources</goal>
-                        </goals><!-- here the phase you need -->
-                        <phase>validate</phase>
-                        <configuration>
-                            <outputDirectory>${basedir}/target/docker-stage/opt/onap/ccsdk</outputDirectory>
-                            <resources>
-                                <resource>
-                                    <directory>src/main/scripts</directory>
-                                    <includes>
-                                        <include>*</include>
-                                    </includes>
-                                    <filtering>false</filtering>
-                                </resource>
-                            </resources>
-                        </configuration>
-                    </execution>
-
-                    <execution>
-                        <id>copy-yml</id>
-                        <goals>
-                            <goal>copy-resources</goal>
-                        </goals><!-- here the phase you need -->
-                        <phase>validate</phase>
-                        <configuration>
-                            <outputDirectory>${basedir}/target/docker-stage/opt/onap/ccsdk/Playbooks</outputDirectory>
-                            <resources>
-                                <resource>
-                                    <directory>src/main/yml</directory>
+                                    <directory>src/main</directory>
                                     <includes>
-                                        <include>*</include>
+                                        <include>**/*</include>
                                     </includes>
                                     <filtering>false</filtering>
                                 </resource>
                         <configuration>
                             <executable>/usr/bin/find</executable>
                             <arguments>
-                                <argument>${basedir}/target/docker-stage/opt/onap/ccsdk</argument>
+                                <argument>${basedir}/target/docker-stage/ansible-server</argument>
                                 <argument>-name</argument>
                                 <argument>*.py</argument>
                                 <argument>-exec</argument>
                             </arguments>
                         </configuration>
                     </execution>
-                       <execution>
-                                               <id>change shell permissions</id>
-                                               <phase>process-sources</phase>
-                                               <goals>
-                                                       <goal>exec</goal>
-                                               </goals>
-                                               <configuration>
-                                                       <executable>/usr/bin/find</executable>
-                                                       <arguments>
-                                                               <argument>${basedir}/target/docker-stage/opt/onap/ccsdk</argument>
-                                                               <argument>-name</argument>
-                                                               <argument>*.sh</argument>
-                                                               <argument>-exec</argument>
-                                                               <argument>chmod</argument>
-                                                               <argument>+x</argument>
-                                                               <argument>{}</argument>
-                                                               <argument>;</argument>
-                                                       </arguments>
-                                               </configuration>
-                                       </execution>
                 </executions>
             </plugin>
         </plugins>
diff --git a/ansible-server/src/main/.dockerignore b/ansible-server/src/main/.dockerignore
new file mode 100644 (file)
index 0000000..0d20b64
--- /dev/null
@@ -0,0 +1 @@
+*.pyc
diff --git a/ansible-server/src/main/Dockerfile b/ansible-server/src/main/Dockerfile
new file mode 100644 (file)
index 0000000..0555e7d
--- /dev/null
@@ -0,0 +1,27 @@
+FROM alpine:3.8
+
+LABEL maintainer="SDN-C Team (sdnc@lists.openecomp.org)"
+ARG PIP_TAG=18.0
+
+WORKDIR /opt/
+
+COPY ansible-server/requirements.txt ansible-server/requirements.txt
+
+RUN apk add --no-cache py2-pip \
+                       python2 &&\
+    apk add --no-cache --virtual .build-deps build-base \
+                                             libffi-dev \
+                                             openssl-dev \
+                                             python2-dev &&\
+    pip install --no-cache-dir --upgrade pip==$PIP_TAG && \
+    pip install --no-cache-dir -r ansible-server/requirements.txt &&\
+    apk del .build-deps
+
+COPY ansible-server ansible-server
+COPY configuration/ansible.cfg /etc/ansible/ansible.cfg
+
+WORKDIR /opt/ansible-server
+
+EXPOSE 8000
+
+ENTRYPOINT ["python2", "RestServer.py"]
@@ -18,7 +18,7 @@
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
-* 
+*
 * ECOMP is a trademark and service mark of AT&T Intellectual Property.
 * ============LICENSE_END=========================================================
 */
@@ -41,7 +41,7 @@ def ansibleSysCall (inventory_path, playbook_path, nodelist, mandatory,
     log = []
 
     str_parameters = ''
-                
+
     if not envparameters == '':
         for key in envparameters:
             if str_parameters == '':
@@ -50,13 +50,13 @@ def ansibleSysCall (inventory_path, playbook_path, nodelist, mandatory,
                 #str_parameters += ' '  + str(key) + '=\'' + str(envparameters[key])  + '\''
                 str_parameters += ', '  + str(key) + '=\'' + str(envparameters[key])  + '\''
         str_parameters += '"'
-                        
+
     if len(str_parameters) > 0:
-        cmd = 'export HOME=/root; env; cd ' + playbookdir + ';' +'timeout --signal=KILL ' + str(timeout) + \
+        cmd = 'export HOME=/root; env; cd ' + playbookdir + ';' +'timeout -s KILL -t ' + str(timeout) + \
               ' ansible-playbook -v --timeout ' + str(timeout) + ' --extra-vars ' + str_parameters + ' -i ' + \
               inventory_path + ' ' + playbook_path + ' | tee log.file'
     else:
-        cmd = 'export HOME=/root; env; cd ' + playbookdir + ';' +'timeout --signal=KILL ' + str(timeout) + \
+        cmd = 'export HOME=/root; env; cd ' + playbookdir + ';' +'timeout -s KILL -t ' + str(timeout) + \
               ' ansible-playbook -v --timeout ' + str(timeout) + ' -i ' + inventory_path + ' ' + playbook_path +' | tee log.file'
 
     cherrypy.log("CMD: " + cmd)
@@ -64,30 +64,30 @@ def ansibleSysCall (inventory_path, playbook_path, nodelist, mandatory,
     cherrypy.log("PlayBook Start: " + playbookdir )
     p = subprocess.Popen(cmd, shell=True,
                          stdout=subprocess.PIPE,
-                         stderr=subprocess.STDOUT) 
+                         stderr=subprocess.STDOUT)
     #PAP
     #p.wait()
     (stdout_value, err) = p.communicate()
-    
+
     stdout_value_cleanup = ''
     for line in stdout_value:
         stdout_value_cleanup += line.replace('  ', ' ')
-    stdout_value = stdout_value_cleanup.splitlines() 
+    stdout_value = stdout_value_cleanup.splitlines()
 
     ParseFlag = False
     retval = {}
     returncode = p.returncode
 
     if returncode == 137:
-        
+
         cherrypy.log("   ansible-playbook system call timed out")
         # ansible-playbook system call timed out
         for line in stdout_value: # p.stdout.readlines():
             log.append (line)
-            
-            
+
+
     else:
-            
+
         for line in stdout_value: # p.stdout.readlines():
             print line # line,
             if ParseFlag and len(line.strip())>0:
@@ -103,7 +103,7 @@ def ansibleSysCall (inventory_path, playbook_path, nodelist, mandatory,
             if "Killed" in line: # check for timeout
                 cherrypy.log(" Playbook Killed(timeout)")
                 returncode = 137
-        
+
     # retval['p'] = p.wait()
 
     #cherrypy.log("*** <" + playbookdir + "> [" + str(log) + "] ***")
@@ -125,7 +125,7 @@ if __name__ == '__main__':
 
 
     d = Manager().dict()
-    
+
     p = Process(nodelist=ansible_call, args=('ansible_module_config', playbook_file, nodelist,d, ))
     p.start()
 
diff --git a/ansible-server/src/main/ansible-server/AnsibleSql.py b/ansible-server/src/main/ansible-server/AnsibleSql.py
new file mode 100755 (executable)
index 0000000..ab58a96
--- /dev/null
@@ -0,0 +1,322 @@
+'''
+/*-
+* ============LICENSE_START=======================================================
+* ONAP : APPC
+* ================================================================================
+* Copyright (C) 2017 AT&T Intellectual Property.  All rights reserved.
+* ================================================================================
+* Copyright (C) 2017 Amdocs
+* =============================================================================
+* Licensed under the Apache License, Version 2.0 (the "License");
+* you may not use this file except in compliance with the License.
+* You may obtain a copy of the License at
+*
+*      http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+* 
+* ECOMP is a trademark and service mark of AT&T Intellectual Property.
+* ============LICENSE_END=========================================================
+*/
+'''
+
+import pymysql, sys
+from os import listdir
+from os.path import isfile, join
+
+class mySql():
+    
+    def __init__(self, myhost, myuser, mypasswd, mydb):
+        self.con = True
+        self.error = ''
+        self.db = None
+        try:
+            self.db = pymysql.connect(host=myhost,
+                                      user=myuser,
+                                      passwd=mypasswd,
+                                      db=mydb)
+            self.cur = self.db.cursor()
+        except Exception as e:
+            self.error = e[1]
+            self.con = False
+
+    def Query (self, myquery, val = None):
+        results = None
+        try:
+            if val:
+                self.cur.execute(myquery, val)
+            else:
+                self.cur.execute(myquery)
+            self.db.commit()
+            results = self.cur.fetchall()
+        except Exception, e:
+            results = repr(e)
+        return results
+    
+    def Close (self):
+        if self.db:
+            self.db.close()
+
+def loadPlaybook (sqlintf, value, version, ext = '.yml'):
+
+    errorCode = 0
+    diag = ''
+    
+    # Test if primary key already defined
+    query = "SELECT name FROM playbook WHERE name='" + value +"'"
+    results = sqlintf.Query (query)
+    if len(results) > 0:
+        pass
+    else:
+        query = "INSERT INTO playbook (name) VALUES ('" + value + "')"
+        results = sqlintf.Query (query)
+        if len(results) > 0:
+            errorCode = 1
+            diag = results
+
+    # Load playbook
+    file = open(playbook_path + value + ext, 'r')
+    load_file = file.read()
+        
+    if not errorCode:
+        sql = "UPDATE playbook SET value=%s, version=%s, type=%s WHERE name=%s"
+
+        results = sqlintf.Query(sql, (load_file, version, ext, value))
+
+        if len (results) > 0:
+            # Error loading playbook
+            errorCode = 1
+            diag = results
+
+    return errorCode, diag
+
+def loadCredentials (sqlintf, hostgroup, hostname, cred):
+    errorCode = 0
+    diag = ''
+    
+    # Load credentials
+
+    query = "SELECT hostname,hostgroup FROM inventory WHERE hostname='" + hostname +"'"
+    results = sqlintf.Query (query)
+
+    if hostname in str (results):
+
+        results_hostgroups = results[0][1]
+        
+        if hostgroup in results_hostgroups.split(','):
+            query = "UPDATE inventory SET hostname='" + hostname + "',credentials='" +\
+                    cred +\
+                    "' WHERE hostname='" + hostname + "'"
+        else:
+
+            results_hostgroups = results_hostgroups + ',' + hostgroup
+            
+            query = "UPDATE inventory SET hostname='" + hostname + "',credentials='" +\
+                    cred + "',hostgroup='" + results_hostgroups + \
+                    "' WHERE hostname='" + hostname + "'"
+                        
+        results = sqlintf.Query (query)
+        
+    else:
+        
+        query = "INSERT INTO inventory (hostgroup, hostname, credentials) VALUES ('" + \
+                hostgroup + "','" + hostname + "','" + cred + "')"
+        results = sqlintf.Query (query)
+
+    if len (results) > 0:
+        # Error loading playbook
+        errorCode = 1
+        diag = results
+
+    return errorCode, diag
+    
+
+def readPlaybook (sqlintf, value, version=None):
+
+    errorCode = 0
+    diag = ''
+
+    print "***> in AnsibleSql.readPlaybook"
+    
+    if not version:
+        query = "SELECT MAX(version) FROM playbook WHERE name like'" + value + "%'"
+        print "   Query:", query
+        results = sqlintf.Query (query)
+        version = results[0][0]
+
+    print "   Provided playbook name:", value 
+    print "   Used version:", version
+
+    results = []
+    if version:
+        query = "SELECT value,type FROM playbook WHERE name='" + value + "@" + version + "'"
+        results = sqlintf.Query (query)
+
+        print "Query:", query
+        print "Results:", results
+    
+    if len(results) == 0:
+        errorCode = 1
+    else:
+        if len(results[0]) == 0:
+            errorCode = 1
+            diag = results[0]
+        else:
+            diag = results[0]
+
+    return value, version, errorCode, diag
+
+def readCredentials (sqlintf, tag):
+    errorCode = []
+    diag = []
+
+    print "***> in AnsibleSql.readCredential"
+    
+    # Load credentials
+
+    for rec in tag:
+
+        # Try hostgroup
+        query = "SELECT hostgroup, hostname, credentials FROM inventory WHERE hostgroup LIKE '%" + \
+                rec +"%'"
+        query_results = sqlintf.Query (query)
+
+        results = ()
+        for q in query_results:
+            if rec in q[0].split(','):
+                l = list(q)
+                l[0] = rec
+                q = tuple(l)
+                results = (q,) + results
+
+        if len(results) == 0:
+            # Try hostname
+            query = "SELECT hostgroup, hostname, credentials FROM inventory WHERE hostname='" + \
+                    rec +"'"
+            results = sqlintf.Query (query)
+
+        print "   Query:", query
+        print "   Results:", len(results), results
+
+        if len(results) == 0:
+            errorCode = 1
+            hostgroup = rec
+            hostname = rec
+            credentials = 'ansible_connection=ssh ansible_ssh_user=na ansible_ssh_private_key_file=na\n'
+            diag.append([hostgroup, hostname, credentials])
+        else:
+            errorCode = 0
+            for i in range(len (results)):
+                for h in results[i][0].split(','):
+                    hostgroup = h
+                    hostname = results[i][1]
+                    credentials = results[i][2]
+                    diag.append([hostgroup, hostname, credentials])
+
+    return errorCode, diag
+    
+
+if __name__ == '__main__':
+
+    ################################################################
+    # Change below
+    ################################################################
+    host="localhost"                    # your host, usually localhost
+    user="mysql_user_id"                # your username
+    passwd="password_4_mysql_user_id"   # your password
+    db="ansible"                        # name of the data base
+
+    playbook_path = "/home/ubuntu/RestServerOpenSource/"
+    inventory = "/home/ubuntu/RestServerOpenSource/Ansible_inventory"
+    ################################################################
+
+    onlyfiles = [f for f in listdir(playbook_path)
+                 if isfile(join(playbook_path, f))]
+
+    sqlintf = mySql (host, user, passwd, db)
+
+    # Load playbooks
+
+    print "Loading playbooks"
+    for file in onlyfiles:
+        if "yml" in file:
+
+            name = file.split (".yml")[0]
+            print "  Loading:", name
+            version = name.split("@")[1]
+            errorCode, diag = loadPlaybook (sqlintf, name, version, '.yml')
+            if errorCode:
+                print "  Results: Failed - ", diag
+            else:
+                print "  Results: Success"
+
+    print "\nLoading inventory"
+    
+    # Load inventory
+
+    hostgroup = None
+    inv = {}
+    file = open(inventory, 'r')
+
+    for line in file:
+
+        if '[' in line and ']' in line:
+            hostgroup = line.strip().replace('[','').replace(']','')
+            inv[hostgroup] = {}
+        elif hostgroup and len(line.strip())>0:
+            host = line.strip().split(" ")[0]
+            credentials = line.replace(host,"")
+            inv[hostgroup][host] = credentials
+                               
+    file.close()
+
+    for hostgroup in inv:
+        print "  Loading:", hostgroup
+        hostfqdn = ''
+        cred = ''
+        for hostname in inv[hostgroup]:
+            cred = inv[hostgroup][hostname]
+            errorCode, diag = loadCredentials (sqlintf, hostgroup, hostname, cred)
+            if errorCode:
+                print "  Results: Failed - ", diag
+            else:
+                print "  Results: Success"
+                
+    print "\nReading playbook"
+    
+    # Read playbook
+
+    if not sqlintf.con:
+        print "Cannot connect to MySql:", sqlintf.error
+        sys.exit()
+        
+    name = "ansible_sleep"
+    print "Reading playbook:", name
+    value, version, errorCode, diag = readPlaybook (sqlintf, name)
+    if errorCode:
+        print "Results: Failed - ", diag
+    else:
+        print "Results: Success"
+        print value
+        print version
+        print diag
+
+    print "\nReading inventory"
+
+    # Read inventory
+
+    tag = ["your_inventory_test_group_name"]
+    print "Reading inventory tag:", tag
+    errorCode, diag = readCredentials (sqlintf, tag)
+    if errorCode:
+        print "Results: Failed - ", diag
+    else:
+        print "Results: Success"
+        print diag
+                
+    sqlintf.Close()
+
diff --git a/ansible-server/src/main/ansible-server/LoadAnsibleMySql.py b/ansible-server/src/main/ansible-server/LoadAnsibleMySql.py
new file mode 100755 (executable)
index 0000000..0a1c78a
--- /dev/null
@@ -0,0 +1,207 @@
+'''
+/*-
+* ============LICENSE_START=======================================================
+* ONAP : APPC
+* ================================================================================
+* Copyright (C) 2017 AT&T Intellectual Property.  All rights reserved.
+* ================================================================================
+* Copyright (C) 2017 Amdocs
+* =============================================================================
+* Licensed under the Apache License, Version 2.0 (the "License");
+* you may not use this file except in compliance with the License.
+* You may obtain a copy of the License at
+*
+*      http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+* 
+* ECOMP is a trademark and service mark of AT&T Intellectual Property.
+* ============LICENSE_END=========================================================
+*/
+'''
+
+#!/usr/bin/python
+import pymysql
+from os import listdir
+from os.path import isfile, join
+
+class mySql():
+    
+    def __init__(self, myhost, myuser, mypasswd, mydb):
+        self.db = pymysql.connect(host=myhost,
+                                  user=myuser,
+                                  passwd=mypasswd,
+                                  db=mydb)
+        self.cur = self.db.cursor()
+
+    def Query (self, myquery, val = None):
+        results = None
+        error = None
+        try:
+            if val:
+                self.cur.execute(myquery, val)
+            else:
+                self.cur.execute(myquery)
+            self.db.commit()
+            results = self.cur.fetchall()
+        except Exception, e:
+            error = str (e)
+        return results, error
+
+    def Close (self):
+        self.db.close()
+
+def loadPlaybook (value, version, ext = '.yml'):
+
+    errorCode = 0
+    diag = ''
+    
+    # Test if primary key already defined
+    query = "SELECT name FROM playbook WHERE name='" + value +"'"
+    results, error = sqlintf.Query (query)
+    if results:
+        # print "Primary key already defined: Updating playbook"
+        pass
+    else:
+        # print "Primary key not defined: Insert new playbook"
+        query = "INSERT INTO playbook (name) VALUES ('" + value + "')"
+        results, error = sqlintf.Query (query)
+        if error:
+            errorCode = 1
+            diag = error
+
+    # Load playbook
+    file = open(playbook_path + value + ext, 'r')
+    load_file = file.read()
+        
+    # Load playbook
+
+    if not errorCode:
+        sql = "UPDATE playbook SET value=%s, version=%s, type=%s WHERE name=%s"
+
+        results, error = sqlintf.Query(sql, (load_file, version, ext, value))
+
+        if error:
+            # Error loading playbook
+            errorCode = 1
+            diag = error
+            
+    return errorCode, diag
+
+def loadCredentials (hostgroup, hostname, cred):
+    errorCode = 0
+    diag = ''
+    
+    # Load credentials
+
+    query = "SELECT hostname,hostgroup FROM inventory WHERE hostname='" + hostname +"'"
+    results = sqlintf.Query (query)
+
+    print '==>', results
+    
+    if hostname in str(results):
+
+        results_hostgroups = results[0][0][1]
+
+        # print "Record already defined: Updating inventory"
+        if hostgroup in results_hostgroups.split(','):
+            query = "UPDATE inventory SET hostname='" + hostname + "',credentials='" +\
+                    cred +\
+                    "' WHERE hostname='" + hostname + "'"
+        else:
+            
+            results_hostgroups = results_hostgroups + ',' + hostgroup
+
+            query = "UPDATE inventory SET hostname='" + hostname + "',credentials='" +\
+                    cred + "',hostgroup='" + results_hostgroups + \
+                    "' WHERE hostname='" + hostname + "'"
+
+        results, error = sqlintf.Query (query)
+        
+    else:
+        
+        query = "INSERT INTO inventory (hostgroup, hostname, credentials) VALUES ('" + \
+                hostgroup + "','" + hostname + "','" + cred + "')"
+        results, error = sqlintf.Query (query)
+
+    if error:
+        # Error loading credentials
+        errorCode = 1
+        diag = results
+
+    return errorCode, diag
+    
+
+if __name__ == '__main__':
+
+    ################################################################
+    # Change below
+    ################################################################
+    host="localhost"                    # your host, usually localhost
+    user="mysql_user_id"                # your username
+    passwd="password_4_mysql_user_id"   # your password
+    db="ansible"                        # name of the data base
+
+    playbook_path = "/home/ubuntu/RestServerOpenSource/"
+    inventory = "/home/ubuntu/RestServerOpenSource/Ansible_inventory"
+    ################################################################
+    
+    onlyfiles = [f for f in listdir(playbook_path)
+                 if isfile(join(playbook_path, f))]
+
+    sqlintf = mySql (host, user, passwd, db)
+
+    # Load playbooks
+    print "Loading playbooks"
+    for file in onlyfiles:
+        if "yml" in file:
+            name = file.split (".yml")[0]
+            print "  Loading:", name
+            version = name.split("@")[1]
+            errorCode, diag = loadPlaybook (name, version)
+            if errorCode:
+                print "  Results: Failed - ", diag
+            else:
+                print "  Results: Success"
+        if "tar.gz" in file:
+            name = file.split (".tar.gz")[0]
+            print "  Loading:", name
+            version = name.split("@")[1]
+            errorCode, diag = loadPlaybook (name, version, ".tar.gz")
+
+    print "\nLoading inventory"
+    
+    # Load inventory
+    hostgroup = None
+    inv = {}
+    file = open(inventory, 'r')
+
+    for line in file:
+
+        if '[' in line and ']' in line:
+            hostgroup = line.strip().replace('[','').replace(']','')
+            inv[hostgroup] = {}
+        elif hostgroup and len(line.strip())>0:
+            host = line.strip().split(" ")[0]
+            credentials = line.replace(host,"")
+            inv[hostgroup][host] = credentials
+                               
+    file.close()
+
+    for hostgroup in inv:
+        print "  Loading:", hostgroup
+        hostfqdn = ''
+        cred = ''
+        for hostname in inv[hostgroup]:
+            cred = inv[hostgroup][hostname]
+            errorCode, diag = loadCredentials (hostgroup, hostname, cred)
+            if errorCode:
+                print "  Results: Failed - ", diag
+            else:
+                print "  Results: Success"
+                
+    sqlintf.Close()
 # * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # * See the License for the specific language governing permissions and
 # * limitations under the License.
-# * 
+# *
 # * ECOMP is a trademark and service mark of AT&T Intellectual Property.
 # * ============LICENSE_END=========================================================
 # */
 
 [host]
 localhost   ansible_connection=local
-
diff --git a/ansible-server/src/main/ansible-server/RestServer.py b/ansible-server/src/main/ansible-server/RestServer.py
new file mode 100644 (file)
index 0000000..23484a5
--- /dev/null
@@ -0,0 +1,949 @@
+'''
+#!/usr/bin/python
+/*-
+* ============LICENSE_START=======================================================
+* ONAP : APPC
+* ================================================================================
+* Copyright (C) 2017 AT&T Intellectual Property.  All rights reserved.
+* ================================================================================
+* Copyright (C) 2017 Amdocs
+* =============================================================================
+* Licensed under the Apache License, Version 2.0 (the "License");
+* you may not use this file except in compliance with the License.
+* You may obtain a copy of the License at
+*
+*      http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*
+* ECOMP is a trademark and service mark of AT&T Intellectual Property.
+* ============LICENSE_END=========================================================
+*/
+'''
+
+import time, datetime, json, os, sys, subprocess, re
+import uuid
+import tarfile
+import shutil
+
+import requests
+
+import cherrypy
+from cherrypy.lib.httputil import parse_query_string
+from cherrypy.lib import auth_basic
+
+from multiprocessing import Process, Manager
+
+from AnsibleModule import ansibleSysCall
+
+import AnsibleSql
+from AnsibleSql import readPlaybook, readCredentials
+
+from os import listdir
+from os.path import isfile, join
+
+TestRecord = Manager().dict()
+ActiveProcess = {}
+
+def sys_call (cmd):
+    p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
+    output = p.stdout.readlines()
+    retval = p.wait()
+    if len (output) > 0:
+        for i in range(len(output)):
+            output[i] = output[i].strip()
+    return retval, output
+
+def callback (Id, Result, Output, Log, returncode):
+
+    print "***> in RestServer.callback"
+
+    if Id in TestRecord:
+        time_now = datetime.datetime.utcnow()
+        delta_time = (time_now - TestRecord[Id]['Time']).total_seconds()
+        Result['PlaybookName'] = TestRecord[Id]['PlaybookName']
+        Result['Version'] = TestRecord[Id]['Version']
+        if returncode == 137:
+            Result['StatusCode'] = 500
+            Result['StatusMessage'] = "TERMINATED"
+        else:
+            Result['StatusCode'] = 200
+            Result['StatusMessage'] = "FINISHED"
+
+        # Need to update the whole data structure for key=Id otherwise Manager is not updated
+        TestRecord[Id] = {'PlaybookName': TestRecord[Id]['PlaybookName'],
+                          'LCM': TestRecord[Id]['LCM'],
+                          'Version': TestRecord[Id]['Version'],
+                          'NodeList': TestRecord[Id]['NodeList'],
+                          'HostGroupList': TestRecord[Id]['HostGroupList'],
+                          'HostNameList': TestRecord[Id]['HostNameList'],
+                          'Time': TestRecord[Id]['Time'],
+                          'Timeout': TestRecord[Id]['Timeout'],
+                          'Duration': str(delta_time),
+                          'EnvParameters': TestRecord[Id]['EnvParameters'],
+                          'LocalParameters': TestRecord[Id]['LocalParameters'],
+                          'FileParameters': TestRecord[Id]['FileParameters'],
+                          'CallBack': TestRecord[Id]['CallBack'],
+                          'Result': Result,
+                          'Log': Log,
+                          'Output': Output,
+                          'Path': TestRecord[Id]['Path'],
+                          'Mandatory': TestRecord[Id]['Path']}
+
+        if not TestRecord[Id]['CallBack'] == None:
+
+            # Posting results to callback server
+
+            data = {"StatusCode": 200,
+                    "StatusMessage": "FINISHED",
+                    "PlaybookName": TestRecord[Id]["PlaybookName"],
+                    "Version": TestRecord[Id]["Version"],
+                    "Duration": TestRecord[Id]["Duration"],
+                    "Results": TestRecord[Id]['Result']['Results']}
+
+            if not TestRecord[Id]['Output']['Output'] == {}:
+                for key in data["Results"]:
+                    if key in TestRecord[Id]['Output']['Output']:
+                        data["Results"][key]["Output"] = TestRecord[Id]['Output']['Output'][key]
+
+            print "     Posting to", TestRecord[Id]['CallBack']
+
+            s = requests.Session()
+            r = s.post(TestRecord[Id]['CallBack'], data = json.dumps(data),
+                       headers = {'content-type': 'application/json'})
+            print  "     Response", r.status_code, r.text
+
+def RunAnsible_Playbook (callback, Id, Inventory, Playbook, NodeList, TestRecord,
+                         Path, ArchiveFlag):
+
+    print "***> in RestServer.RunAnsible_Playbook"
+
+    # Run test in playbook for given target
+    Result = ''
+
+    retval, log, returncode = ansibleSysCall (Inventory, Playbook, NodeList,
+                                              TestRecord[Id]['Mandatory'],
+                                              TestRecord[Id]['EnvParameters'],
+                                              TestRecord[Id]['LocalParameters'],
+                                              TestRecord[Id]['LCM'],
+                                              TestRecord[Id]['Timeout'])
+
+
+    print "   returncode:", returncode
+    print "   retval:    ", retval
+    print "   log:       ", log
+
+    Log = ''.join(log)
+    Output = {'Output': {}}
+
+    onlyfiles = [f for f in listdir(Path)
+                 if isfile(join(Path, f))]
+
+    for file in onlyfiles:
+        if "results.txt" in file:
+            f = open(Path + "/" + file, "r")
+            key = file.split("_")[0]
+            Output['Output'][key] = f.read()
+            f.close()
+
+    Result = {'Results': {}}
+    if 'could not be found' in Log:
+        Result['Results'] = {"StatusCode": 101,
+                             "StatusMessage": "PLAYBOOK NOT FOUND"}
+    if returncode == 137:
+        Result['Results'] = {"StatusCode": 500,
+                             "StatusMessage": "TERMINATED"}
+
+    elif TestRecord[Id]['NodeList'] == []:
+
+        host_index = None
+
+        if 'TargetNode' in TestRecord[Id]['EnvParameters']:
+            targetlist = TestRecord[Id]['EnvParameters']['TargetNode'].split(' ')
+        else:
+            targetlist = ["localhost"]
+
+        for key in retval:
+            for i in range (len(targetlist)):
+                if key in targetlist[i]:
+                    host_index = i
+
+            if int(retval[key][0]) > 0 and int(retval[key][2]) == 0 and \
+                   int(retval[key][3]) == 0:
+
+                if host_index:
+                    Result['Results'][targetlist[host_index]] = \
+                             {"GroupName": 'na', "StatusCode": 200, \
+                              "StatusMessage": "SUCCESS"}
+                else:
+                    Result['Results'][key] = \
+                             {"GroupName": 'na', "StatusCode": 200, \
+                              "StatusMessage": "SUCCESS"}
+            elif int(retval[key][2]) > 0:
+                if host_index:
+                    Result['Results'][targetlist[host_index]] = \
+                       {"GroupName": 'na', "StatusCode": 400, \
+                        "StatusMessage": "NOT REACHABLE"}
+                else:
+                    Result['Results'][key] = \
+                       {"GroupName": 'na', "StatusCode": 400, \
+                        "StatusMessage": "NOT REACHABLE"}
+            elif int(retval[key][3]) > 0:
+                if host_index:
+                    Result['Results'][targetlist[host_index]] = \
+                       {"GroupName": 'na', "StatusCode": 400, \
+                        "StatusMessage": "FAILURE"}
+                else:
+                    Result['Results'][key] = \
+                       {"GroupName": 'na', "StatusCode": 400, \
+                        "StatusMessage": "FAILURE"}
+    else:
+
+        for key in retval:
+
+            if len(TestRecord[Id]['HostNameList']) > 0:
+
+                host_index = []
+                for i in range (len(TestRecord[Id]['HostNameList'])):
+                    if key in TestRecord[Id]['HostNameList'][i]:
+                        host_index.append(i)
+
+                if int(retval[key][0]) > 0 and int(retval[key][2]) == 0 and \
+                       int(retval[key][3]) == 0:
+
+                    if len(host_index) > 0:
+                        Result['Results'][TestRecord[Id]['HostNameList'][host_index[0]]] = \
+                          {"GroupName": TestRecord[Id]['HostGroupList'][host_index[0]],
+                           "StatusCode": 200, "StatusMessage": "SUCCESS"}
+
+                        for i in range (1, len(host_index)):
+                            Result['Results'][TestRecord[Id]['HostNameList'][host_index[i]]]["GroupName"]+=\
+                             "," + TestRecord[Id]['HostGroupList'][host_index[i]]
+                    else:
+                       Result['Results'][key] = \
+                          {"GroupName": key,
+                           "StatusCode": 200, "StatusMessage": "SUCCESS"}
+
+                elif int(retval[key][2]) > 0:
+
+                    if len(host_index) > 0:
+                        Result['Results'][TestRecord[Id]['HostNameList'][host_index[0]]] = \
+                          {"GroupName": TestRecord[Id]['HostGroupList'][host_index[0]],
+                           "StatusCode": 400, "StatusMessage": "NOT REACHABLE"}
+
+                        for i in range (1, len(host_index)):
+                            Result['Results'][TestRecord[Id]['HostNameList'][host_index[i]]]["GroupName"]+=\
+                             "," + TestRecord[Id]['HostGroupList'][host_index[i]]
+                    else:
+                       Result['Results'][key] = \
+                          {"GroupName": key,
+                           "StatusCode": 200, "StatusMessage": "NOT REACHABLE"}
+
+                elif int(retval[key][3]) > 0:
+
+                    if len(host_index) > 0:
+                        Result['Results'][TestRecord[Id]['HostNameList'][host_index[0]]] = \
+                          {"GroupName": TestRecord[Id]['HostGroupList'][host_index[0]],
+                           "StatusCode": 400, "StatusMessage": "FAILURE"}
+
+                        for i in range (1, len(host_index)):
+                            Result['Results'][TestRecord[Id]['HostNameList'][host_index[i]]]["GroupName"]+=\
+                             "," + TestRecord[Id]['HostGroupList'][host_index[i]]
+                    else:
+                       Result['Results'][key] = \
+                          {"GroupName": key,
+                           "StatusCode": 200, "StatusMessage": "FAILURE"}
+            else:
+                host_index = None
+                for i in range (len(TestRecord[Id]['NodeList'])):
+                    if key in TestRecord[Id]['NodeList'][i]:
+                        host_index = i
+
+                if int(retval[key][0]) > 0 and int(retval[key][2]) == 0 and \
+                       int(retval[key][3]) == 0:
+                    Result['Results'][TestRecord[Id]['NodeList'][host_index]] = \
+                             {"GroupName": 'na', "StatusCode": 200, \
+                             "StatusMessage": "SUCCESS"}
+                elif int(retval[key][2]) > 0:
+                    Result['Results'][TestRecord[Id]['NodeList'][host_index]] = \
+                       {"GroupName": 'na', "StatusCode": 400, "StatusMessage": "NOT REACHABLE"}
+                elif int(retval[key][3]) > 0:
+                    Result['Results'][TestRecord[Id]['NodeList'][host_index]] = \
+                       {"GroupName": 'na', "StatusCode": 400, "StatusMessage": "FAILURE"}
+
+    callback (Id, Result, Output, Log, returncode)
+
+class TestManager (object):
+
+    @cherrypy.expose
+    @cherrypy.tools.json_out()
+    @cherrypy.tools.json_in()
+    @cherrypy.tools.allow(methods=['POST', 'GET', 'DELETE'])
+
+    def Dispatch(self, **kwargs):
+
+        # Let cherrypy error handler deal with malformed requests
+        # No need for explicit error handler, we use default ones
+
+        time_now = datetime.datetime.utcnow()
+
+        # Erase old test results (2x timeout)
+        if TestRecord:
+            for key in TestRecord.copy():
+                delta_time = (time_now - TestRecord[key]['Time']).seconds
+                if delta_time > 2*TestRecord[key]['Timeout']:
+                    print "Deleted history for test", key
+                    if os.path.exists(TestRecord[key]['Path']):
+                        shutil.rmtree (TestRecord[key]['Path'])
+                    del TestRecord[key]
+
+        print "***> in RestServer.Dispatch:", cherrypy.request.method
+
+        HomeDir = os.path.dirname(os.path.realpath("~/"))
+
+        if 'POST' in cherrypy.request.method:
+
+            input_json = cherrypy.request.json
+            print "   Payload:      ", input_json
+
+            if 'Id' in input_json and 'PlaybookName' in input_json:
+
+                if True:
+
+                    if not input_json['Id'] in TestRecord:
+
+                        Id = input_json['Id']
+                        PlaybookName = input_json['PlaybookName']
+
+                        version = None
+                        if 'Version' in input_json:
+                            version = input_json['Version']
+
+                        AnsibleInvFail = True
+                        AnsiblePlaybookFail = True
+
+                        MySqlConFail = True
+                        MySqlCause = ''
+
+                        LocalNodeList = None
+
+                        str_uuid = str (uuid.uuid4())
+
+                        LCM = PlaybookName.split(".")[0].split('_')[-1]
+                        PlaybookDir = HomeDir + "/" + ansible_temp + "/" + \
+                                      PlaybookName.split(".")[0] + "_" + str_uuid
+                        AnsibleInv = LCM + "_" + "inventory"
+                        ArchiveFlag = False
+
+                        print "   LCM:          ", LCM
+                        print "   PlaybookDir:  ", ansible_temp + PlaybookDir.split(ansible_temp)[1]
+                        print "   AnsibleInv:   ", AnsibleInv
+                        print "   ansible_temp: ", ansible_temp
+
+                        if not os.path.exists(HomeDir + "/" + ansible_temp):
+                            os.makedirs(HomeDir + "/" + ansible_temp)
+
+                        os.mkdir(PlaybookDir)
+
+                        # Process inventory file for target
+
+                        hostgrouplist = []
+                        hostnamelist = []
+
+                        NodeList = []
+                        if 'NodeList' in input_json:
+                            NodeList = input_json['NodeList']
+
+                        print "   NodeList:     ", NodeList
+
+                        if NodeList == []:
+                                # By default set to local host
+                                AnsibleInvFail = False
+
+                                LocalNodeList = "host"
+                                LocalCredentials = "localhost   ansible_connection=local"
+                                f = open(PlaybookDir + "/" + AnsibleInv, "w")
+                                f.write("[" + LocalNodeList + "]\n")
+                                f.write(LocalCredentials)
+                                f.close()
+
+                        else:
+
+                                if from_files:
+
+                                    # Get credentials from file
+
+                                    data_inventory_orig = {}
+                                    data_inventory_target = {}
+                                    curr_group = None
+
+                                    print "***>", ansible_path + "/" + ansible_inv
+                                    f = open(ansible_path + "/" + ansible_inv, "r")
+                                    for line in f:
+                                        line = line.rstrip()
+
+                                        if len(line)> 0:
+                                            if '#' not in line:
+                                                if "[" in line and "]" in line:
+                                                    data_inventory_orig[line] = []
+                                                    curr_group = line
+                                                else:
+                                                    data_inventory_orig[curr_group].append(line)
+                                    f.close()
+
+                                    for node in NodeList:
+                                        Fail = True
+                                        if "[" + node + "]" in data_inventory_orig:
+                                            if not "[" + node + "]" in data_inventory_target:
+
+                                                print "RESET", "[" + node + "]"
+                                                data_inventory_target["[" + node + "]"] = []
+                                            else:
+                                                print "OK", "[" + node + "]"
+                                            Fail = False
+                                            for cred in data_inventory_orig["[" + node + "]"]:
+                                                data_inventory_target["[" + node + "]"].append(cred)
+
+                                        else:
+                                            for key in data_inventory_orig:
+                                                if node in " ".join(data_inventory_orig[key]):
+                                                    if not key in data_inventory_target:
+                                                        data_inventory_target[key] = []
+                                                    for cred in data_inventory_orig[key]:
+                                                        if node in cred:
+                                                            data_inventory_target[key].append(cred)
+                                                            Fail = False
+
+                                        if Fail:
+                                            data_inventory_target["["+node+"]"] = \
+                                                   [node + " ansible_connection=ssh ansible_ssh_user=na ansible_ssh_private_key_file=na"]
+
+                                    AnsibleInvFail = False
+
+                                    f = open(PlaybookDir + "/" + AnsibleInv, "w")
+                                    for key in data_inventory_target:
+                                        f.write(key + "\n")
+                                        for rec in data_inventory_target[key]:
+                                            hostgrouplist.append(key.replace("[", '').replace("]", ''))
+                                            hostnamelist.append(rec.split(' ')[0])
+                                            f.write(rec + "\n")
+                                    f.close()
+
+                                else:
+
+                                    # Get credentials from mySQL
+
+                                    sqlintf = AnsibleSql.mySql (host, user, passwd,
+                                                                db)
+                                    if sqlintf.con:
+                                        MySqlConFail = False
+                                        errorCode, diag = readCredentials (sqlintf,
+                                                                           NodeList)
+
+                                        print errorCode, diag
+                                        if len (diag) > 0:
+                                            f = open(PlaybookDir + "/" + AnsibleInv,
+                                                     "w")
+                                            AnsibleInvFail = False
+                                            # [hostgroup, hostname, credentials]
+                                            for i in range(len(diag)):
+                                                f.write('[' +  diag[i][0] + ']' + "\n")
+                                                f.write(diag[i][1]+ " " +  diag[i][2] + "\n\n")
+                                                hostgrouplist.append(diag[i][0])
+                                                hostnamelist.append(diag[i][1])
+                                            f.close()
+                                        else:
+                                            MySqlConFailCause = sqlintf.error
+                                    sqlintf.Close()
+
+                        timeout = timeout_seconds
+                        if 'Timeout' in input_json:
+                            timeout = int (input_json['Timeout'])
+
+                        EnvParam = {}
+                        if 'EnvParameters' in input_json:
+                            EnvParam = input_json['EnvParameters']
+
+                        LocalParam = {}
+                        if 'LocalParameters' in input_json:
+                            LocalParam = input_json['LocalParameters']
+
+                        FileParam = {}
+                        if 'FileParameters' in input_json:
+                            FileParam = input_json['FileParameters']
+
+                        callback_flag = None
+                        if 'CallBack' in input_json:
+                            callback_flag = input_json['CallBack']
+
+                        TestRecord[Id] = {'PlaybookName': PlaybookName,
+                                          'LCM': LCM,
+                                          'Version': version,
+                                          'NodeList': NodeList,
+                                          'HostGroupList': hostgrouplist,
+                                          'HostNameList': hostnamelist,
+                                          'Time': time_now,
+                                          'Duration': timeout,
+                                          'Timeout': timeout,
+                                          'EnvParameters': EnvParam,
+                                          'LocalParameters': LocalParam,
+                                          'FileParameters': FileParam,
+                                          'CallBack': callback_flag,
+                                          'Result': {"StatusCode": 100,
+                                                     "StatusMessage": 'PENDING',
+                                                     "ExpectedDuration": str(timeout) + "sec"},
+                                          'Log': '',
+                                          'Output': {},
+                                          'Path': PlaybookDir,
+                                          'Mandatory': None}
+
+                        # Write files
+
+                        if not TestRecord[Id]['FileParameters'] == {}:
+                            for key in TestRecord[Id]['FileParameters']:
+                                filename = key
+                                filecontent = TestRecord[Id]['FileParameters'][key]
+                                f = open(PlaybookDir + "/" + filename, "w")
+                                f.write(filecontent)
+                                f.close()
+
+
+                        # Process playbook
+
+                        if from_files:
+
+                            # Get playbooks from files
+
+                            MySqlConFail = False
+
+                            version = None
+                            target_PlaybookName = None
+
+                            if '@' in PlaybookName:
+                                version = PlaybookName.split("@")[1]
+                                version = version.replace('.yml','')
+                                version = version.replace('.tar.gz','')
+
+                            onlyfiles = [f for f in listdir(ansible_path)
+                                         if isfile(join(ansible_path, f))]
+
+                            version_max = '0.00'
+                            version_target = ''
+
+                            for file in onlyfiles:
+                                if LCM in file:
+                                    temp_version = file.split("@")[1]
+                                    temp_version = temp_version.replace('.yml','')
+                                    temp_version = temp_version.replace('.tar.gz','')
+                                    if version_max < temp_version:
+                                        version_max = temp_version
+
+                                    if not version == None:
+                                        if version in PlaybookName:
+                                            version_target = version
+                                            target_PlaybookName = file
+
+                            if target_PlaybookName == None:
+                                for file in onlyfiles:
+                                    if LCM in file and version_max in file:
+                                        target_PlaybookName = file
+                                        version_target = version_max
+
+                            if target_PlaybookName:
+                                AnsiblePlaybookFail = False
+                                readversion = version_target
+                                src  = ansible_path + "/" + target_PlaybookName
+                                if ".tar.gz" in target_PlaybookName:
+                                    dest = PlaybookDir + "/" + LCM + ".tar.gz"
+                                    shutil.copy2(src, dest)
+                                    retcode = subprocess.call(['tar', '-xvzf',
+                                                   dest, "-C", PlaybookDir])
+                                    ArchiveFlag = True
+                                else:
+                                    dest = PlaybookDir + "/" + LCM + ".yml"
+                                    shutil.copy2(src, dest)
+
+                        else:
+                            # Get playbooks from mySQL
+
+                            sqlintf = AnsibleSql.mySql (host, user, passwd, db)
+                            if sqlintf.con:
+                                MySqlConFail = False
+
+                                name, readversion, AnsiblePlaybookFail, diag = \
+                                  readPlaybook (sqlintf, PlaybookName.split(".")[0],
+                                                version)
+
+                                if not AnsiblePlaybookFail:
+
+                                    f = open(PlaybookDir + "/" + LCM + diag[1], "w")
+                                    f.write(diag[0])
+                                    f.close()
+
+                                    if ".tar.gz" in diag[1]:
+                                        retcode = subprocess.call(['tar', '-xvzf',
+                                         PlaybookDir + "/" + LCM + diag[1], "-C", PlaybookDir])
+                                        f.close()
+                                        ArchiveFlag = True
+                            else:
+                                MySqlConFailCause = sqlintf.error
+                            sqlintf.Close()
+
+                        if MySqlConFail:
+                            if os.path.exists(PlaybookDir):
+                                shutil.rmtree (PlaybookDir)
+                            del TestRecord[Id]
+                            return {"StatusCode": 101,
+                                    "StatusMessage": "CANNOT CONNECT TO MYSQL: " \
+                                    + MySqlConFailCause}
+                        elif AnsiblePlaybookFail:
+                            if os.path.exists(PlaybookDir):
+                                shutil.rmtree (PlaybookDir)
+                            del TestRecord[Id]
+                            return {"StatusCode": 101,
+                                    "StatusMessage": "PLAYBOOK NOT FOUND"}
+                        elif AnsibleInvFail:
+                            if os.path.exists(PlaybookDir):
+                                shutil.rmtree (PlaybookDir)
+                            del TestRecord[Id]
+                            return {"StatusCode": 101,
+                                    "StatusMessage": "NODE LIST CREDENTIALS NOT FOUND"}
+                        else:
+
+                            # Test EnvParameters
+                            playbook_path = None
+                            if ArchiveFlag:
+                                for dName, sdName, fList in os.walk(PlaybookDir):
+                                    if LCM+".yml" in fList:
+                                        playbook_path = dName
+                            else:
+                                playbook_path = PlaybookDir
+
+                            # Store local vars
+                            if not os.path.exists(playbook_path + "/vars"):
+                                os.mkdir(playbook_path + "/vars")
+                            if not os.path.isfile(playbook_path + "/vars/defaults.yml"):
+                                os.mknod(playbook_path + "/vars/defaults.yml")
+
+                            for key in TestRecord[Id]['LocalParameters']:
+                                host_index = []
+                                for i in range(len(TestRecord[Id]['HostNameList'])):
+                                    if key in TestRecord[Id]['HostNameList'][i]:
+                                        host_index.append(i)
+                                if len(host_index) == 0:
+                                    for i in range(len(TestRecord[Id]['HostGroupList'])):
+                                        if key in TestRecord[Id]['HostGroupList'][i]:
+                                            host_index.append(i)
+                                if len(host_index) > 0:
+                                    for i in range(len(host_index)):
+                                        f = open(playbook_path + "/vars/" +
+                                                 TestRecord[Id]['HostNameList'][host_index[i]] +
+                                                 ".yml", "a")
+                                        for param in TestRecord[Id]['LocalParameters'][key]:
+                                            f.write(param + ": " +
+                                             str (TestRecord[Id]['LocalParameters'][key][param]) +
+                                                  "\n")
+                                        f.close()
+
+                            # Get mandatory parameters from playbook
+                            Mandatory = []
+                            with open(playbook_path + "/" + LCM + ".yml") as origin_file:
+                                for line in origin_file:
+                                    if "Mandatory" in line:
+                                        temp = line.split(":")[1].strip().replace(' ', '')
+                                        if len(temp) > 0:
+                                            Mandatory = temp.split(",")
+
+                            TestRecord[Id] = {'PlaybookName': TestRecord[Id]['PlaybookName'],
+                                              'LCM': TestRecord[Id]['LCM'],
+                                              'Version': readversion,
+                                              'NodeList': TestRecord[Id]['NodeList'],
+                                              'HostGroupList': TestRecord[Id]['HostGroupList'],
+                                              'HostNameList': TestRecord[Id]['HostNameList'],
+                                              'Time': TestRecord[Id]['Time'],
+                                              'Timeout': TestRecord[Id]['Timeout'],
+                                              'Duration': TestRecord[Id]['Duration'],
+                                              'EnvParameters': TestRecord[Id]['EnvParameters'],
+                                              'LocalParameters': TestRecord[Id]['LocalParameters'],
+                                              'FileParameters': TestRecord[Id]['FileParameters'],
+                                              'CallBack': TestRecord[Id]['CallBack'],
+                                              'Result': TestRecord[Id]['Result'],
+                                              'Log': TestRecord[Id]['Log'],
+                                              'Output': TestRecord[Id]['Output'],
+                                              'Path': TestRecord[Id]['Path'],
+                                              'Mandatory': Mandatory}
+
+                            TestKey = False
+
+                            if Mandatory:
+                                for val in Mandatory:
+                                    if EnvParam:
+                                        if val in EnvParam:
+                                            TestKey = True
+                                        else:
+                                            if LocalParam:
+                                                for key in TestRecord[Id]['NodeList']:
+                                                    if key in LocalParam:
+                                                        if val in LocalParam[key]:
+                                                            TestKey = True
+                                    else:
+                                        if LocalParam:
+                                            for key in TestRecord[Id]['NodeList']:
+                                                if key in LocalParam:
+                                                    if val in LocalParam[key]:
+                                                        TestKey = True
+
+                                if not TestKey:
+                                    if os.path.exists(PlaybookDir):
+                                        shutil.rmtree (PlaybookDir)
+                                    del TestRecord[Id]
+                                    return {"StatusCode": 101,
+                                            "StatusMessage": "MISSING MANDATORY PARAMETER: " + \
+                                            " ".join(str(x) for x in Mandatory)}
+
+
+                            # Cannot use thread because ansible module uses
+                            # signals which are only supported in main thread.
+                            # So use multiprocess with shared object
+
+                            p = Process(target = RunAnsible_Playbook,
+                                        args = (callback, Id,  PlaybookDir + "/" + AnsibleInv,
+                                                playbook_path + "/" + LCM + ".yml",
+                                                NodeList, TestRecord, PlaybookDir,
+                                                ArchiveFlag))
+                            p.start()
+                            ActiveProcess[Id] = p
+                            return TestRecord[Id]['Result']
+                    else:
+                        return {"StatusCode": 101, "StatusMessage": "TEST ID ALREADY DEFINED"}
+
+                else:
+                    return {"StatusCode": 500, "StatusMessage": "REQUEST MUST INCLUDE: NODELIST"}
+
+            else:
+                return {"StatusCode": 500, "StatusMessage": "JSON OBJECT MUST INCLUDE: ID, PLAYBOOKNAME"}
+
+        elif 'GET' in cherrypy.request.method:
+
+            input_data = parse_query_string(cherrypy.request.query_string)
+
+            print "***> in RestServer.GET"
+            print "   Payload: ", input_data, input_data['Type']
+
+            if 'Id' in input_data and 'Type' in input_data:
+                if not ('GetResult' in input_data['Type'] or 'GetOutput' in input_data['Type'] or 'GetLog' in input_data['Type']):
+                    return {"StatusCode": 500, "StatusMessage": "RESULTS TYPE UNDEFINED"}
+                if input_data['Id'] in TestRecord:
+
+                    if 'GetResult' in input_data['Type']:
+
+                        print "Result:", TestRecord[input_data['Id']]['Result']
+
+                        if 'StatusMessage' in TestRecord[input_data['Id']]['Result'] and getresults_block:
+
+                            print "*** Request blocked", input_data['Id']
+
+                            while ActiveProcess[input_data['Id']].is_alive():
+                                time.sleep(5)
+
+                            print "*** Request released ", input_data['Id']
+
+                        print TestRecord[input_data['Id']]['Result']
+                        if TestRecord[input_data['Id']]['Result']['StatusCode'] == 500:
+                            out_obj = TestRecord[input_data['Id']]['Result']['Results']
+                        else:
+                            out_obj = {"StatusCode": 200,
+                                   "StatusMessage": "FINISHED",
+                                   "PlaybookName": TestRecord[input_data['Id']]["PlaybookName"],
+                                   "Version": TestRecord[input_data['Id']]["Version"],
+                                   "Duration": TestRecord[input_data['Id']]["Duration"],
+                                   "Results": TestRecord[input_data['Id']]['Result']['Results']}
+                        if not TestRecord[input_data['Id']]['Output']['Output'] == {}:
+                            for key in out_obj["Results"]:
+                                if key in TestRecord[input_data['Id']]['Output']['Output']:
+                                    out_obj["Results"][key]["Output"] = TestRecord[input_data['Id']]['Output']['Output'][key]
+
+                        return out_obj
+
+                    elif 'GetOutput' in input_data['Type']:
+
+                        if TestRecord[input_data['Id']]['Output'] == {} and \
+                               getresults_block:
+
+                            print "*** Request blocked", input_data['Id']
+
+                            while TestRecord[input_data['Id']]['Output'] == {} \
+                                      or 'StatusMessage' in TestRecord[input_data['Id']]['Result']:
+                                time.sleep(5)
+
+                            print "*** Request released ", input_data['Id']
+
+                        print "Output:", TestRecord[input_data['Id']]['Output']
+                        return {"Output": TestRecord[input_data['Id']]['Output']['Output']}
+                    else:
+                        # GetLog
+
+                        if TestRecord[input_data['Id']]['Log'] == '' and \
+                               getresults_block:
+
+                            print "*** Request blocked", input_data['Id']
+
+                            while TestRecord[input_data['Id']]['Log'] == '' \
+                                      or 'StatusMessage' in TestRecord[input_data['Id']]['Result']:
+                                time.sleep(5)
+
+                            print "*** Request released ", input_data['Id']
+
+                        print "Log:", TestRecord[input_data['Id']]['Log']
+                        return {"Log": TestRecord[input_data['Id']]['Log']}
+                else:
+                    return {"StatusCode": 500, "StatusMessage": "TEST ID UNDEFINED"}
+            else:
+                return {"StatusCode": 500, "StatusMessage": "MALFORMED REQUEST"}
+        elif 'DELETE' in cherrypy.request.method:
+            input_data = parse_query_string(cherrypy.request.query_string)
+
+            print "***> in RestServer.DELETE"
+            print "   Payload: ", input_data
+
+            if input_data['Id'] in TestRecord:
+                if not 'PENDING' in TestRecord[input_data['Id']]['Result']:
+                    print "   Path:", TestRecord[input_data['Id']]['Path']
+                    if os.path.exists(TestRecord[input_data['Id']]['Path']):
+                        shutil.rmtree (TestRecord[input_data['Id']]['Path'])
+                    TestRecord.pop (input_data['Id'])
+                    if input_data['Id'] in ActiveProcess:
+                        ActiveProcess.pop (input_data['Id'])
+
+                    return {"StatusCode": 200, "StatusMessage": "PLAYBOOK EXECUTION RECORDS DELETED"}
+                else:
+                    return {"StatusCode": 200, "StatusMessage": "PENDING"}
+            else:
+                return {"StatusCode": 500, "StatusMessage": "TEST ID UNDEFINED"}
+
+
+if __name__ == '__main__':
+
+    # Read configuration
+
+    config_file_path = "RestServer_config"
+
+    if not os.path.exists(config_file_path):
+        print '[INFO] The config file does not exist'
+        sys.exit(0)
+
+    ip = 'na'
+    port = 'na'
+    tls = False
+    auth = False
+    pub = 'na'
+    id = 'na'
+    priv = 'na'
+    psswd = 'na'
+    timeout_seconds = 'na'
+    ansible_path = 'na'
+    ansible_inv = 'na'
+    ansible_temp = 'na'
+    host = 'na'
+    user = 'na'
+    passwd = 'na'
+    db = 'na'
+    getresults_block = False
+    from_files = False
+
+    file = open(config_file_path, 'r')
+    for line in file.readlines():
+        if '#' not in line:
+            if 'ip:' in line:
+                ip = line.split(':')[1].strip()
+            elif 'port:' in line:
+                port = line.split(':')[1].strip()
+            elif 'tls:' in line:
+                tls = 'YES' in line.split(':')[1].strip().upper()
+            elif 'auth:' in line:
+                auth = 'YES' in line.split(':')[1].strip().upper()
+            if tls and 'priv:' in line:
+                priv = line.split(':')[1].strip()
+            if tls and 'pub:' in line:
+                pub = line.split(':')[1].strip()
+            if auth and 'id:' in line:
+                id = line.split(':')[1].strip()
+            if auth and 'psswd:' in line:
+                psswd = line.split(':')[1].strip()
+            if 'timeout_seconds' in line:
+                timeout_seconds = int (line.split(':')[1].strip())
+            if 'ansible_path' in line:
+                ansible_path = line.split(':')[1].strip()
+            if 'ansible_inv' in line:
+                ansible_inv = line.split(':')[1].strip()
+                if not os.path.exists(ansible_path + "/" + ansible_inv):
+                    print '[INFO] The ansible_inv file does not exist'
+                    sys.exit(0)
+            if 'ansible_temp' in line:
+                ansible_temp = line.split(':')[1].strip()
+            if 'host' in line:
+                host = line.split(':')[1].strip()
+            if 'user' in line:
+                user = line.split(':')[1].strip()
+            if 'passwd' in line:
+                passwd = line.split(':')[1].strip()
+            if 'db' in line:
+                db = line.split(':')[1].strip()
+            if 'getresults_block' in line:
+                getresults_block = 'YES' in line.split(':')[1].strip().upper()
+            if 'from_files' in line:
+                from_files = 'YES' in line.split(':')[1].strip().upper()
+    file.close()
+
+    # Initialization
+
+    global_conf = {
+        'global': {
+            'server.socket_host': ip,
+            'server.socket_port': int(port),
+            'server.protocol_version': 'HTTP/1.1'
+            }
+        }
+
+    if tls:
+        # Use pythons built-in SSL
+        cherrypy.server.ssl_module = 'builtin'
+
+        # Point to certificate files
+
+        if not os.path.exists(pub):
+            print '[INFO] The public certificate does not exist'
+            sys.exit(0)
+
+        if not os.path.exists(priv):
+            print '[INFO] The private key does not exist'
+            sys.exit(0)
+
+        cherrypy.server.ssl_certificate = pub
+        cherrypy.server.ssl_private_key = priv
+
+    if auth:
+        userpassdict = {id: psswd}
+        checkpassword = cherrypy.lib.auth_basic.checkpassword_dict(userpassdict)
+
+        app_conf = {'/':
+                    {'tools.auth_basic.on': True,
+                     'tools.auth_basic.realm': 'earth',
+                     'tools.auth_basic.checkpassword': checkpassword,
+                     }
+                    }
+
+        cherrypy.tree.mount(TestManager(), '/', app_conf)
+    else:
+        cherrypy.tree.mount(TestManager(), '/')
+
+    cherrypy.config.update(global_conf)
+
+    # Start server
+
+    cherrypy.engine.start()
+    cherrypy.engine.block()
@@ -46,7 +46,7 @@ db:   ansible
 
 # Playbooks
 from_files:            yes
-ansible_path:          /opt/onap/ccsdk/Playbooks
+ansible_path:          /opt/ansible-server/Playbooks
 ansible_inv:           Ansible_inventory
 ansible_temp:          PlaybooksTemp
 timeout_seconds:       60
diff --git a/ansible-server/src/main/ansible-server/requirements.txt b/ansible-server/src/main/ansible-server/requirements.txt
new file mode 100644 (file)
index 0000000..3d508f4
--- /dev/null
@@ -0,0 +1,4 @@
+PyMySQL
+cherrypy<18.0.0
+requests
+ansible
\ No newline at end of file
diff --git a/ansible-server/src/main/configuration/ansible.cfg b/ansible-server/src/main/configuration/ansible.cfg
new file mode 100644 (file)
index 0000000..14c8065
--- /dev/null
@@ -0,0 +1,2 @@
+[defaults]
+host_key_checking = False
diff --git a/ansible-server/src/main/scripts/RestServer.py b/ansible-server/src/main/scripts/RestServer.py
deleted file mode 100755 (executable)
index 1469e59..0000000
+++ /dev/null
@@ -1,1065 +0,0 @@
-'''
-/*-
-* ============LICENSE_START=======================================================
-* ONAP : APPC
-* ================================================================================
-* Copyright (C) 2017-2019 AT&T Intellectual Property.  All rights reserved.
-* ================================================================================
-* Copyright (C) 2017 Amdocs
-* =============================================================================
-* Licensed under the Apache License, Version 2.0 (the "License");
-* you may not use this file except in compliance with the License.
-* You may obtain a copy of the License at
-*
-*      http://www.apache.org/licenses/LICENSE-2.0
-*
-* Unless required by applicable law or agreed to in writing, software
-* distributed under the License is distributed on an "AS IS" BASIS,
-* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-* See the License for the specific language governing permissions and
-* limitations under the License.
-* 
-* ECOMP is a trademark and service mark of AT&T Intellectual Property.
-* ============LICENSE_END=========================================================
-*/
-'''
-
-import time, datetime, json, os, sys, subprocess, re
-import uuid
-import tarfile
-import shutil
-import glob
-import crypt
-
-import requests
-
-import cherrypy
-from cherrypy.lib.httputil import parse_query_string
-from cherrypy.lib import auth_basic
-
-from multiprocessing import Process, Manager
-
-from AnsibleModule import ansibleSysCall
-from BuildHostFile import buildHostsSysCall
-
-from os import listdir
-from os.path import isfile, join
-
-TestRecord = Manager().dict()
-ActiveProcess = {}
-
-def validate_password(realm, username, password):
-    comp = crypt.crypt(password, salt)
-    if username in userpassdict and userpassdict[username] == comp:
-       return True
-    return False
-
-def sys_call (cmd):
-    p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
-    output = p.stdout.readlines()
-    retval = p.wait()
-    if len (output) > 0:
-        for i in range(len(output)):
-            output[i] = output[i].strip()
-    return retval, output
-
-def callback (Id, Result, Output, Log, returncode):
-    
-    print "***> in RestServer.callback"
-
-    if Id in TestRecord:
-        time_now = datetime.datetime.utcnow()
-        delta_time = (time_now - TestRecord[Id]['Time']).total_seconds()
-        Result['PlaybookName'] = TestRecord[Id]['PlaybookName']
-        Result['Version'] = TestRecord[Id]['Version']
-        if returncode == 137:
-            Result['StatusCode'] = 500
-            Result['StatusMessage'] = "TERMINATED"
-        else:
-            Result['StatusCode'] = 200
-            Result['StatusMessage'] = "FINISHED"
-
-        # Need to update the whole data structure for key=Id otherwise Manager is not updated
-        TestRecord[Id] = {'PlaybookName': TestRecord[Id]['PlaybookName'],
-                          'Version': TestRecord[Id]['Version'],
-                          'NodeList': TestRecord[Id]['NodeList'],
-                          'HostGroupList': TestRecord[Id]['HostGroupList'],
-                          'HostNameList': TestRecord[Id]['HostNameList'],
-                          'Time': TestRecord[Id]['Time'],
-                          'Timeout': TestRecord[Id]['Timeout'],
-                          'Duration': str(delta_time),
-                          'EnvParameters': TestRecord[Id]['EnvParameters'],
-                          'LocalParameters': TestRecord[Id]['LocalParameters'],
-                          'FileParameters': TestRecord[Id]['FileParameters'],
-                          'CallBack': TestRecord[Id]['CallBack'],
-                          'Result': Result,
-                          'Log': Log, 
-                          'Output': Output, 
-                          'Path': TestRecord[Id]['Path'],
-                          'Mandatory': TestRecord[Id]['Path']}
-
-        if not TestRecord[Id]['CallBack'] == None:
-            
-            # Posting results to callback server
-
-            data = {"StatusCode": 200,
-                    "StatusMessage": "FINISHED",
-                    "PlaybookName": TestRecord[Id]["PlaybookName"],
-                    "Version": TestRecord[Id]["Version"],
-                    "Duration": TestRecord[Id]["Duration"],
-                    "Results": TestRecord[Id]['Result']['Results']}
-
-            cherrypy.log("CALLBACK: TestRecord[Id]['Output']['Output']:", str(TestRecord[Id]['Output']['Output']))
-            cherrypy.log("CALLBACK: Results:", str(data["Results"]))
-
-            if not TestRecord[Id]['Output']['Output'] == {}:
-                for key in data["Results"]:
-                    if key in TestRecord[Id]['Output']['Output']:
-                        data["Results"][key]["Output"] = TestRecord[Id]['Output']['Output'][key]
-
-            print "     Posting to", TestRecord[Id]['CallBack']
-            
-            s = requests.Session()
-            r = s.post(TestRecord[Id]['CallBack'], data = json.dumps(data),
-                       headers = {'content-type': 'application/json'})
-            print  "     Response", r.status_code, r.text
-
-def RunAnsible_Playbook (callback, Id, Inventory, Playbook, NodeList, TestRecord,
-                         Path, ArchiveFlag):
-
-    print "***> in RestServer.RunAnsible_Playbook"
-
-    # Run test in playbook for given target
-    Result = ''
-
-    retval, log, returncode = ansibleSysCall (Inventory, Playbook, NodeList,
-                                              TestRecord[Id]['Mandatory'],
-                                              TestRecord[Id]['EnvParameters'],
-                                              TestRecord[Id]['LocalParameters'],
-                                              TestRecord[Id]['Timeout'],
-                                              Path)
-
-
-    cherrypy.log("Return code:" + str(returncode))
-    cherrypy.log("Return val:" +  str(retval))
-    
-    Log = ''.join(log)
-    #Output = {'Output': {}}
-    Output = {}
-    
-    onlyfiles = [f for f in listdir(Path)
-                 if isfile(join(Path, f))]
-
-    cherrypy.log("Checking for results.txt files: ")
-    for file in onlyfiles:
-        if "results.txt" in file:
-#       if file.endswith("results.txt"):
-            cherrypy.log("results file: " + file)
-            f = open(Path + "/" + file, "r")
-            resultsData =  f.read()  # Not to pass vnf instance name
-            OutputP = json.loads(resultsData)
-            Output['Output'] =  OutputP 
-           cherrypy.log("Output = " + str(Output['Output']))
-            #Output['Output'][key] = f.read() # To pass vnf instance name
-            f.close()
-
-    if Output == {}:
-      Output = {'Output': {}}
-
-    Result = {'Results': {}}
-    if 'could not be found' in Log:
-        Result['Results'] = {"StatusCode": 101,
-                             "StatusMessage": "PLAYBOOK NOT FOUND"}
-    if returncode == 137:
-        Result['Results'] = {"StatusCode": 500,
-                             "StatusMessage": "TERMINATED"}
-
-    elif TestRecord[Id]['NodeList'] == []:
-        
-        host_index = None
-        
-        if 'TargetNode' in TestRecord[Id]['EnvParameters']:
-            targetlist = TestRecord[Id]['EnvParameters']['TargetNode'].split(' ')
-        else:
-            targetlist = ["localhost"]
-            
-        for key in retval:
-            for i in range (len(targetlist)):
-                if key in targetlist[i]:
-                    host_index = i
-    
-            if int(retval[key][0]) > 0 and int(retval[key][2]) == 0 and \
-                   int(retval[key][3]) == 0:
-
-                if host_index:
-                    Result['Results'][targetlist[host_index]] = \
-                             {"GroupName": 'na', "StatusCode": 200, \
-                              "StatusMessage": "SUCCESS"}
-                else:
-                    Result['Results'][key] = \
-                             {"GroupName": 'na', "StatusCode": 200, \
-                              "StatusMessage": "SUCCESS"}                    
-            elif int(retval[key][2]) > 0:
-                if host_index:
-                    Result['Results'][targetlist[host_index]] = \
-                       {"GroupName": 'na', "StatusCode": 400, \
-                        "StatusMessage": "NOT REACHABLE"}
-                else:
-                    Result['Results'][key] = \
-                       {"GroupName": 'na', "StatusCode": 400, \
-                        "StatusMessage": "NOT REACHABLE"}                    
-            elif int(retval[key][3]) > 0:
-                if host_index:                
-                    Result['Results'][targetlist[host_index]] = \
-                       {"GroupName": 'na', "StatusCode": 400, \
-                        "StatusMessage": "FAILURE"}
-                else:
-                    Result['Results'][key] = \
-                       {"GroupName": 'na', "StatusCode": 400, \
-                        "StatusMessage": "FAILURE"}                    
-    else:
-        
-        for key in retval:
-
-            if len(TestRecord[Id]['HostNameList']) > 0:
-
-                host_index = []
-                for i in range (len(TestRecord[Id]['HostNameList'])):
-                    if key in TestRecord[Id]['HostNameList'][i]:
-                        host_index.append(i)
-
-                if int(retval[key][0]) > 0 and int(retval[key][2]) == 0 and \
-                       int(retval[key][3]) == 0:
-
-                    if len(host_index) > 0:
-                        Result['Results'][TestRecord[Id]['HostNameList'][host_index[0]]] = \
-                          {"GroupName": TestRecord[Id]['HostGroupList'][host_index[0]],
-                           "StatusCode": 200, "StatusMessage": "SUCCESS"}
-                    
-                        for i in range (1, len(host_index)):
-                            Result['Results'][TestRecord[Id]['HostNameList'][host_index[i]]]["GroupName"]+=\
-                             "," + TestRecord[Id]['HostGroupList'][host_index[i]]
-                    else:
-                       Result['Results'][key] = \
-                          {"GroupName": key,
-                           "StatusCode": 200, "StatusMessage": "SUCCESS"}  
-
-                elif int(retval[key][2]) > 0:
-
-                    if len(host_index) > 0:
-                        Result['Results'][TestRecord[Id]['HostNameList'][host_index[0]]] = \
-                          {"GroupName": TestRecord[Id]['HostGroupList'][host_index[0]],
-                           "StatusCode": 400, "StatusMessage": "NOT REACHABLE"}
-                    
-                        for i in range (1, len(host_index)):
-                            Result['Results'][TestRecord[Id]['HostNameList'][host_index[i]]]["GroupName"]+=\
-                             "," + TestRecord[Id]['HostGroupList'][host_index[i]]
-                    else:
-                       Result['Results'][key] = \
-                          {"GroupName": key,
-                           "StatusCode": 200, "StatusMessage": "NOT REACHABLE"}  
-                    
-                elif int(retval[key][3]) > 0:
-
-                    if len(host_index) > 0:
-                        Result['Results'][TestRecord[Id]['HostNameList'][host_index[0]]] = \
-                          {"GroupName": TestRecord[Id]['HostGroupList'][host_index[0]],
-                           "StatusCode": 400, "StatusMessage": "FAILURE"}
-                    
-                        for i in range (1, len(host_index)):
-                            Result['Results'][TestRecord[Id]['HostNameList'][host_index[i]]]["GroupName"]+=\
-                             "," + TestRecord[Id]['HostGroupList'][host_index[i]]
-                    else:
-                       Result['Results'][key] = \
-                          {"GroupName": key,
-                           "StatusCode": 200, "StatusMessage": "FAILURE"}                          
-            else:
-                host_index = None
-                for i in range (len(TestRecord[Id]['NodeList'])):
-                    if key in TestRecord[Id]['NodeList'][i]:
-                        host_index = i
-    
-                if int(retval[key][0]) > 0 and int(retval[key][2]) == 0 and \
-                       int(retval[key][3]) == 0:
-                    Result['Results'][TestRecord[Id]['NodeList'][host_index]] = \
-                             {"GroupName": 'na', "StatusCode": 200, \
-                             "StatusMessage": "SUCCESS"}
-                elif int(retval[key][2]) > 0:
-                    Result['Results'][TestRecord[Id]['NodeList'][host_index]] = \
-                       {"GroupName": 'na', "StatusCode": 400, "StatusMessage": "NOT REACHABLE"}
-                elif int(retval[key][3]) > 0:
-                    Result['Results'][TestRecord[Id]['NodeList'][host_index]] = \
-                       {"GroupName": 'na', "StatusCode": 400, "StatusMessage": "FAILURE"}
-    
-    cherrypy.log("TESTRECORD: " + str(TestRecord[Id]))
-    #cherrypy.log("Output: " + str(Output))
-    callback (Id, Result, Output, Log, returncode)
-
-class TestManager (object):
-
-    @cherrypy.expose
-    @cherrypy.tools.json_out()
-    @cherrypy.tools.json_in()
-    @cherrypy.tools.allow(methods=['POST', 'GET', 'DELETE'])
-
-    def Dispatch(self, **kwargs):
-
-        # Let cherrypy error handler deal with malformed requests
-        # No need for explicit error handler, we use default ones
-
-        time_now = datetime.datetime.utcnow()
-
-        # Erase old test results (2x timeout)
-        # Do cleanup too of ActiveProcess list and old Records - PAP
-        if TestRecord:
-            for key in TestRecord.copy():
-                cherrypy.log( "LOOKING AT ALL TestRecords: " + str(key))
-                if key in ActiveProcess:
-                   if not ActiveProcess[key].is_alive(): # Just to cleanup defunct processes
-                      cherrypy.log( "Not ActiveProcess for ID: " + str(key))
-                delta_time = (time_now - TestRecord[key]['Time']).seconds
-                if delta_time > 2*TestRecord[key]['Timeout']:
-                    cherrypy.log( "DELETED HISTORY for ID: " + str(key))
-                    if key in ActiveProcess:
-                      if not ActiveProcess[key].is_alive():
-                          ActiveProcess.pop (key)
-                          cherrypy.log( "DELETED ActiveProcess for ID: " + str(key))
-                    #if os.path.exists(TestRecord[key]['Path']):
-                        # don't remove run dirrectory
-                        #shutil.rmtree (TestRecord[key]['Path'])
-                    del TestRecord[key]
-                    
-        cherrypy.log("RestServer.Dispatch: " + cherrypy.request.method)
-
-
-        if 'POST' in cherrypy.request.method:
-            
-            input_json = cherrypy.request.json
-            cherrypy.log("Payload: " + str(input_json))
-
-            if 'Id' in input_json and 'PlaybookName' in input_json and 'EnvParameters' in input_json:
-
-                if True:
-
-                    if not input_json['Id'] in TestRecord:
-                       # check if Id exists in previous run dirctory
-                        # if so retun error
-                        s_cmd = 'ls ' + ansible_temp + '/*_' + input_json['Id']
-                        #if subprocess.check_output([s_cmd, ]):
-                        Id = input_json['Id']
-                        if glob.glob(  ansible_temp + '/*_' + input_json['Id']):
-                            cherrypy.log("Old directory found for ID: " + Id)
-                            return {"StatusCode": 101, "StatusMessage": "TEST ID FILE ALREADY DEFINED"}
-                    
-                        PlaybookName = input_json['PlaybookName']
-                        # if required it should be passed as an argument
-                        EnvParameters = input_json['EnvParameters']
-
-                        # The lines below are to test multiple EnvParameters being passed
-                        #for i in EnvParameters:
-                        #  cherrypy.log("EnvParameter object: " + i)
-                        #  cherrypy.log("  EnvParameter Value: " + EnvParameters[ i ])
-
-                       # Now get things out of EnvParameters
-                        VNF_instance   = None
-                        VNF_instance = EnvParameters.get('vnf_instance')
-
-                        # Get Version if present
-                        version = None
-                        if 'Version' in input_json:
-                            version = input_json['Version']
-
-                        # GetInventoryNames
-                        HaveNodeList         = False
-                        HaveInventoryNames   = False 
-                        inventory_names = None
-                        if 'InventoryNames' in input_json:
-                           inventory_names = input_json['InventoryNames']
-                           HaveInventoryNames   = True 
-                    
-                        #AnsibleInvFail = True
-                        AnsiblePlaybookFail = True
-
-                        LocalNodeList = None
-
-                        str_uuid = str (uuid.uuid4())
-                        
-
-                        VnfType= PlaybookName.split("/")[0] 
-                        if auth:
-                           cherrypy.log( "Request USER  :                  " + cherrypy.request.login)
-                        cherrypy.log( "Request Decode: ID               " + Id)
-                        cherrypy.log( "Request Decode: VnfType          " + VnfType)
-                        cherrypy.log( "Request Decode: EnvParameters    " + json.dumps(EnvParameters))
-                        
-                        # Verify VNF_instance was passed in EnvParameters
-                        if VNF_instance != None:
-                         cherrypy.log( "Request Decode: VnfInstance      " + VNF_instance)
-                        else:
-                         cherrypy.log( "StatusCode: 107, StatusMessage: VNF_instance NOT PROVIDED" )
-                          return {"StatusCode": 107,
-                                    "StatusMessage": "VNF_instance NOT PROVIDED"}
-
-                        if inventory_names != None:
-                         cherrypy.log( "Request Decode: Inventory Names  " + inventory_names)
-                        else:
-                         cherrypy.log( "Request Decode: Inventory Names  " + "Not provided")
-
-                       cherrypy.log( "Request Decode: PlaybookName     " + PlaybookName)
-                       PlayBookFunction = PlaybookName.rsplit("/",2)[1]
-                       PlayBookFile = PlayBookFunction + "/site.yml"
-                       cherrypy.log( "Request Decode: PlaybookFunction " + PlayBookFunction)
-                       cherrypy.log( "Request Decode: Playbook file    " + PlayBookFile)
-                       
-                        BaseDir = ansible_path + "/" + PlaybookName.rsplit("/",1)[0]
-                        CopyDir = ansible_path + "/" + PlaybookName.rsplit("/",2)[0]
-                       cherrypy.log( "Request Decode: Basedir          " + BaseDir)
-                       cherrypy.log( "Request Decode: Copydir          " + CopyDir)
-                       
-
-                        PlaybookDir = ansible_temp + "/" + \
-                                      VNF_instance + "_" + str_uuid + "_" + str(Id)
-
-                       # AnsibleInv is the directory where the host file to be run exsists
-                        AnsibleInv = ansible_path + "/" + VnfType + "/latest/ansible/inventory/" + VNF_instance
-                        ArchiveFlag = False
-
-                       # Create base run directory if it doesn't exist
-                        if not os.path.exists(ansible_temp):
-                            cherrypy.log( "Creating Base Run Directory: "  + ansible_temp)
-                            os.makedirs(ansible_temp)
-
-                       if not os.path.exists( CopyDir ):
-                            cherrypy.log("Playbook Not Found")
-                            return {"StatusCode": 101,
-                                    "StatusMessage": "PLAYBOOK NOT FOUND"}
-
-                        # copy static playbook dir to run dir
-                        cherrypy.log("Copying from " + CopyDir + " to " + PlaybookDir)
-                        shutil.copytree(CopyDir, PlaybookDir)
-                        cmd="/usr/bin/find " + PlaybookDir + " -exec /usr/bin/touch {} \;"
-                        cmd="/usr/bin/find " + PlaybookDir + " -exec chmod +rx  {} \;"
-                        sys_call(cmd)
-                        cherrypy.log(cmd)
-
-                        cherrypy.log( "PlaybookDir:    " + PlaybookDir)
-                        cherrypy.log( "AnsibleInv:     " + AnsibleInv)
-
-                        # Process inventory file for target
-                    
-                        hostgrouplist = []
-                        hostnamelist = []
-
-                        NodeList = []
-                        if 'NodeList' in input_json:
-                            NodeList = input_json['NodeList']
-
-                        cherrypy.log("NodeList: " + str(NodeList));
-
-                        # if NodeList empty 
-                        if NodeList == []:
-                                cherrypy.log( "*** NodeList - Empty ***")
-
-                        else:
-                                HaveNodeList = True
-
-                       ###############################################################################
-                       ##### Host file processing                          ###########################
-                       ##### 1. Use file delivered with playbook           ###########################
-                       ##### 2. If HostNames + NodeList generate and use   ###########################
-                       ###############################################################################
-
-                        #Verify inventory directory exists
-                        path = PlaybookDir + "/inventory/"
-                        if not os.path.isdir(path):
-                            cherrypy.log ("Inventory directory %s does not exist - create it" % path)
-                            try:
-                                os.mkdir(path)
-                            except OSError:
-                                cherrypy.log ("Creation of the directory %s failed" % path)
-                            else:
-                                cherrypy.log ("Successfully created the directory %s " % path)
-
-                       #location of host file - Default
-                        HostFile = PlaybookDir + "/inventory/" + VNF_instance + "hosts"
-                        cherrypy.log("HostFile: " +  HostFile)
-
-                        # if NodeList and InventoryNames need to build host file
-                        if HaveInventoryNames & HaveNodeList:
-                           cherrypy.log("Build host file from NodeList")
-                           ret = buildHostsSysCall (input_json, PlaybookDir, inventory_names)
-                           if (ret < 0):
-                              cherrypy.log("Returning Error: Not running Playbook")
-                              return {"StatusCode": 105,
-                                    "StatusMessage": "NodeList: Missing vnfc-type field"}
-
-                           # Having been built now copy new file to correct file
-                           shutil.copy(PlaybookDir + "/host_file.txt", HostFile)
-                           cherrypy.log("Copying Generated host file to: " + HostFile)
-
-                        timeout = timeout_seconds
-                        if 'Timeout' in input_json:
-                            timeout = int (input_json['Timeout'])
-                            cherrypy.log("Timeout from API: " + str(timeout))
-
-                        else:
-                            cherrypy.log("Timeout not passed from API using default: " + str(timeout))
-
-                        EnvParam = {}
-                        if 'EnvParameters' in input_json:
-                            EnvParam = input_json['EnvParameters']
-
-                        LocalParam = {}
-                        if 'LocalParameters' in input_json:
-                            LocalParam = input_json['LocalParameters']
-
-                        FileParam = {}
-                        if 'FileParameters' in input_json:
-                            FileParam = input_json['FileParameters']
-                    
-                        callback_flag = None
-                        if 'CallBack' in input_json:
-                            callback_flag = input_json['CallBack']
-
-                       # if AnsibleServer is not set to 'na'  don't send AnsibleServer in PENDING responce.
-                       if AnsibleServer != 'na':
-                               TestRecord[Id] = {'PlaybookName': PlaybookName,
-                                          'Version': version,
-                                          'NodeList': NodeList,
-                                          'HostGroupList': hostgrouplist,
-                                          'HostNameList': hostnamelist,
-                                          'Time': time_now,
-                                          'Duration': timeout,
-                                          'Timeout': timeout,
-                                          'EnvParameters': EnvParam,
-                                          'LocalParameters': LocalParam,
-                                          'FileParameters': FileParam,
-                                          'CallBack': callback_flag,
-                                          'Result': {"StatusCode": 100,
-                                                     "StatusMessage": 'PENDING',
-                                                     "AnsibleServer": str(AnsibleServer),
-                                                     "ExpectedDuration": str(timeout) + "sec"},
-                                          'Log': '',
-                                          'Output': {},
-                                          'Path': PlaybookDir,
-                                          'Mandatory': None}
-                       else:
-                               TestRecord[Id] = {'PlaybookName': PlaybookName,
-                                          'Version': version,
-                                          'NodeList': NodeList,
-                                          'HostGroupList': hostgrouplist,
-                                          'HostNameList': hostnamelist,
-                                          'Time': time_now,
-                                          'Duration': timeout,
-                                          'Timeout': timeout,
-                                          'EnvParameters': EnvParam,
-                                          'LocalParameters': LocalParam,
-                                          'FileParameters': FileParam,
-                                          'CallBack': callback_flag,
-                                          'Result': {"StatusCode": 100,
-                                                     "StatusMessage": 'PENDING',
-                                                     "ExpectedDuration": str(timeout) + "sec"},
-                                          'Log': '',
-                                          'Output': {},
-                                          'Path': PlaybookDir,
-                                          'Mandatory': None}
-
-                        cherrypy.log("Test_Record: " +  str(TestRecord[Id]))
-                        # Write files
-                        
-                        if not TestRecord[Id]['FileParameters'] == {}:
-                            for key in TestRecord[Id]['FileParameters']:
-                                filename = key
-                                filecontent = TestRecord[Id]['FileParameters'][key]
-                                f = open(PlaybookDir + "/" + filename, "w")
-                                f.write(filecontent)
-                                f.close()
-                                
-                        
-                        # Process playbook
-                       if os.path.exists( ansible_path + '/' + PlaybookName):
-                            AnsiblePlaybookFail = False
-                                    
-                        if AnsiblePlaybookFail:
-                            #if os.path.exists(PlaybookDir):
-                                #shutil.rmtree (PlaybookDir)
-                            del TestRecord[Id]
-                            return {"StatusCode": 101,
-                                    "StatusMessage": "PLAYBOOK NOT FOUND"}
-                        else:
-
-                            # Test EnvParameters
-                            playbook_path = PlaybookDir
-
-                            # Store local vars
-                            if not os.path.exists(playbook_path + "/vars"):
-                                os.mkdir(playbook_path + "/vars")
-                            if not os.path.isfile(playbook_path + "/vars/defaults.yml"):
-                                os.mknod(playbook_path + "/vars/defaults.yml")
-
-                            ###################################################
-                            # PAP
-                            #write local parameters passed into defaults.yml
-                            # PAP
-                            f = open(playbook_path + "/vars/defaults.yml","a")
-                            #for id, record in TestRecord.items():
-                            print TestRecord[Id]['LocalParameters']
-                            local_parms = TestRecord[Id]['LocalParameters']
-                            for key, value in local_parms.items():
-                                f.write(key +"=" + value + "\n");
-                            f.close()
-                            ###################################################
-                                 
-                            for key in TestRecord[Id]['LocalParameters']:
-                                host_index = []
-                                for i in range(len(TestRecord[Id]['HostNameList'])):
-                                    if key in TestRecord[Id]['HostNameList'][i]:
-                                        host_index.append(i)
-                                if len(host_index) == 0:
-                                    for i in range(len(TestRecord[Id]['HostGroupList'])):
-                                        if key in TestRecord[Id]['HostGroupList'][i]:
-                                            host_index.append(i)
-                                if len(host_index) > 0:
-                                    for i in range(len(host_index)):
-                                        f = open(playbook_path + "/vars/" +
-                                                 TestRecord[Id]['HostNameList'][host_index[i]] +
-                                                 ".yml", "a")
-                                        for param in TestRecord[Id]['LocalParameters'][key]:
-                                            f.write(param + ": " +
-                                             str (TestRecord[Id]['LocalParameters'][key][param]) +
-                                                  "\n")
-                                        f.close()
-
-    
-                            # write some info out to files before running
-                            if auth:
-                               f = open(playbook_path + "/User.txt", "a")
-                               f.write(cherrypy.request.login)
-                               f.close()
-                            f = open(playbook_path + "/PlaybookName.txt", "a")
-                            f.write(PlaybookName)
-                            f.close()
-                            f = open(playbook_path + "/PlaybookExDir.txt", "a")
-                            f.write(PlaybookDir + "/" + PlayBookFunction)
-                            f.close()
-                            f = open(playbook_path + "/JsonRequest.txt", "w")
-                            #f.write(str(input_json))
-                            #print( json.dumps(input_json, indent=4, sort_keys=True))
-                            f.write( json.dumps(input_json, indent=4, sort_keys=True))
-                            f.close()
-
-
-                            # Check that HostFile exists
-                            if not os.path.isfile(HostFile):
-                              cherrypy.log("Inventory file Not Found: " + HostFile)
-                              return {"StatusCode": 101,
-                                   "StatusMessage": "PLAYBOOK INVENTORY FILE NOT FOUND"}
-   
-                            # Cannot use thread because ansible module uses
-                            # signals which are only supported in main thread.
-                            # So use multiprocess with shared object
-                                       # args = (callback, Id,  PlaybookDir + "/" + AnsibleInv,
-
-                            p = Process(target = RunAnsible_Playbook,
-                                        args = (callback, Id,  HostFile,
-                                                PlaybookDir + '/' + PlayBookFile,
-                                                NodeList, TestRecord, PlaybookDir + "/" + PlayBookFunction,
-                                                ArchiveFlag))
-                            p.start()
-                            ActiveProcess[Id] = p
-                            return TestRecord[Id]['Result']
-                    else:
-                        cherrypy.log("TEST ID ALREADY DEFINED")
-                        return {"StatusCode": 101, "StatusMessage": "TEST ID ALREADY DEFINED"}
-
-                else:
-                    return {"StatusCode": 500, "StatusMessage": "REQUEST MUST INCLUDE: NODELIST"}
-                
-            else:
-                return {"StatusCode": 500, "StatusMessage": "JSON OBJECT MUST INCLUDE: ID, PLAYBOOKNAME, EnvParameters"}
-
-        elif 'GET' in cherrypy.request.method:
-            
-            # Lets pause for a second just incase the resquest was just kicked off
-            time.sleep(1)
-
-            input_data = parse_query_string(cherrypy.request.query_string)
-             
-            # Verify we have a Type passed in GET request
-            if not ( 'Type' in input_data):
-                return {"StatusCode": 500, "StatusMessage": "RESULTS TYPE UNDEFINED"}
-
-            if auth:
-               cherrypy.log( "Request USER:             " + cherrypy.request.login)
-            cherrypy.log("Payload: " + str(input_data) + " Type " + input_data['Type'])
-
-            if 'LogRest' in input_data['Type']:
-               sys.stdout.close()
-               sys.stdout = open("/var/log/RestServer.log", "w")
-
-            # Just a debug to dump any records
-            if 'GetStatus' in input_data['Type']:
-                cherrypy.log( "******** Dump Records **********")
-                if TestRecord.items():
-                  for id, record in TestRecord.items():
-                    cherrypy.log( "    Id: " + id)
-                    cherrypy.log( "Record: " + str(record))
-                else:
-                  cherrypy.log(" No Records to dump")
-            
-            if 'Id' in input_data and 'Type' in input_data:
-                if not ('GetResult' in input_data['Type'] or 'GetOutputLog' in input_data['Type'] or'GetTheOutput' in input_data['Type'] or 'GetLog' in input_data['Type']):
-                    return {"StatusCode": 500, "StatusMessage": "RESULTS TYPE UNDEFINED"}
-                if input_data['Id'] in TestRecord:
-                    
-                    if 'GetResult' in input_data['Type']:
-                        
-                        cherrypy.log( " ** GetResult for: " + str (input_data['Id']))
-
-                        if 'StatusMessage' in TestRecord[input_data['Id']]['Result'] and getresults_block:
-
-
-                            #check if playbook is still running
-                            while ActiveProcess[input_data['Id']].is_alive():
-                                cherrypy.log( "*** Playbook running returning PENDING for " + str(input_data['Id']))
-                                ##
-                                ## If still running return PENDING response
-                                ##
-                                if AnsibleServer != 'na':
-                                   return {"StatusCode": 100,
-                                                     "StatusMessage": 'PENDING',
-                                                     "AnsibleServer": str(AnsibleServer)}
-                                else: 
-                                   return {"StatusCode": 100,
-                                                     "StatusMessage": 'PENDING'}
-                                #time.sleep(5)
-
-                            #cherrypy.log( "*** Request released " + input_data['Id'])
-
-                        cherrypy.log(str( TestRecord[input_data['Id']]['Result']))
-                        cherrypy.log("Output: " + str( TestRecord[input_data['Id']]['Output']))
-                        cherrypy.log("StatusCode: " + str( TestRecord[input_data['Id']]['Result']['StatusCode']))
-                        cherrypy.log("StatusMessage: " + str( TestRecord[input_data['Id']]['Result']['StatusMessage']))
-
-                        #out_obj gets returned to GET request
-                        if TestRecord[input_data['Id']]['Result']['StatusCode'] == 500:
-                            out_obj = TestRecord[input_data['Id']]['Result']['Results']
-                        else:
-                            out_obj = {"StatusCode": 200,
-                                   "StatusMessage": "FINISHED",
-                                   "PlaybookName": TestRecord[input_data['Id']]["PlaybookName"],
-                                   "Version": TestRecord[input_data['Id']]["Version"],
-                                   "Duration": TestRecord[input_data['Id']]["Duration"],
-                                   "Output": TestRecord[input_data['Id']]["Output"]["Output"],
-                                   "Results": TestRecord[input_data['Id']]['Result']['Results']}
-                        if not TestRecord[input_data['Id']]['Output']['Output'] == {}:
-                            cherrypy.log("TestRecord has Output:" + str(TestRecord[input_data['Id']]['Output']['Output']))
-                            # PAP                
-                            for key in out_obj["Results"]:
-                                cherrypy.log("Output key: " + str(key))
-                                if key in TestRecord[input_data['Id']]['Output']['Output']:
-                                    out_obj["Results"][key]["Output"] = TestRecord[input_data['Id']]['Output']['Output'][key]
-
-                        cherrypy.log("***** GET RETURNING RESULTS Back ****")
-                        cherrypy.log(str(out_obj))
-                        return out_obj
-
-                    elif 'GetStatus' in input_data['Type']:
-                        print " Dump Records"
-                        for id, record in TestRecord,items():
-                           print " id: " + id
-                           print "   Record:" + str(reecord)
-
-                    elif 'GetTheOutput' in input_data['Type']:
-
-                        if TestRecord[input_data['Id']]['Output'] == {} and \
-                               getresults_block:
-
-                            cherrypy.log( "*** Request blocked " + input_data['Id'])
-                            
-                            while TestRecord[input_data['Id']]['Output'] == {} \
-                                      or 'StatusMessage' in TestRecord[input_data['Id']]['Result']:
-                                time.sleep(5)
-
-                            cherrypy.log( "*** Request released " + input_data['Id'])
-                        
-                        cherrypy.log( "Output: " + str(TestRecord[input_data['Id']]['Output']))
-                        return {"Output": TestRecord[input_data['Id']]['Output']['Output']}
-
-                    elif 'GetOutputLog' in input_data['Type']:
-                       cherrypy.log("GetOutputLog: processing.")
-                       if glob.glob(  ansible_temp + '/*_' + input_data['Id']):
-                          id = input_data['Id']
-                          cherrypy.log("Old directory found for ID: " + id)
-                          run_dir = glob.glob(  ansible_temp + '/*_' + input_data['Id'])
-                          for dir in run_dir:
-                              rdir=dir
-                          if os.path.exists (rdir + "/PlaybookExDir.txt"):
-                               cherrypy.log("Found PlaybookExDir.txt file")
-                               f = open( rdir + '/PlaybookExDir.txt', 'r')
-                               playbookexdir =  f.readline()
-                               rdir = playbookexdir
-                               f.close()
-                          cherrypy.log("Id:     " + id)
-                          cherrypy.log("RunDir: " + rdir)
-                          if os.path.exists( rdir + "/output.log"):
-                               cherrypy.log("Found output.log file")
-                               f = open( rdir + '/output.log', 'r')
-                               output_log =  f.readline()
-                               f.close()
-                               return output_log
-                       else:
-                         cherrypy.log("Globglob failed:")
-                         return
-
-                    else:
-                        # GetLog
-
-                        if TestRecord[input_data['Id']]['Log'] == '' and \
-                               getresults_block:
-
-                            cherrypy.log( "*** Request blocked " + input_data['Id'])
-                            
-                            while TestRecord[input_data['Id']]['Log'] == '' \
-                                      or 'StatusMessage' in TestRecord[input_data['Id']]['Result']:
-                                time.sleep(5)
-
-                            cherrypy.log( "*** Request released " + input_data['Id'])
-                            
-                        cherrypy.log( "Log:" + str(TestRecord[input_data['Id']]['Log']))
-                        return {"Log": TestRecord[input_data['Id']]['Log']}
-                else:
-                   # Not in memory check for a file 
-                   if glob.glob(  ansible_temp + '/*_' + input_data['Id']):
-                       id = input_data['Id']
-                       cherrypy.log("Old directory found for ID: " + id)
-                       run_dir = glob.glob(  ansible_temp + '/*_' + input_data['Id'])
-                       for dir in run_dir:
-                           rdir=dir
-                       if os.path.exists (rdir + "/PlaybookExDir.txt"):
-                            cherrypy.log("Found PlaybookExDir.txt file")
-                            f = open( rdir + '/PlaybookExDir.txt', 'r')
-                            playbookexdir =  f.readline()
-                            rdir = playbookexdir
-                            f.close()
-                       cherrypy.log("Id:     " + id)
-                       cherrypy.log("RunDir: " + rdir)
-                       if 'GetLog' in input_data['Type']:
-                         if os.path.exists( rdir + "/output.log"):
-                            cherrypy.log("Found output.log file")
-                            f = open( rdir + '/output.log', 'r')
-                            output_log =  f.readline()
-                            f.close()
-                            return output_log
-                       elif 'GetOutputLog' in input_data['Type']:
-                            if os.path.exists( rdir + "/output.log"):
-                               cherrypy.log("Found output.log file")
-                               f = open( rdir + '/output.log', 'r')
-                               output_log =  f.readline()
-                               f.close()
-                               return output_log
-                       elif 'GetResult' in input_data['Type']:
-                            if os.path.exists (rdir + "/PlaybookName.txt"):
-                               cherrypy.log("Found PlaybookName.txt file")
-                               f = open( rdir + '/PlaybookName.txt', 'r')
-                               playbooknametxt =  f.readline()
-                               f.close()
-                            else:
-                               playbooknametxt = "NA"
-
-                           # Add code to get other items not just output.log from files
-                            if os.path.exists( rdir + "/log.file"):
-                               cherrypy.log("Found log.file")
-                               out_results = "NA:"
-                               f = open( rdir + '/log.file', 'r')
-                               
-                               line =  f.readline()
-                               while line :
-                                if "fatal" in line:
-                                   out_results = out_results +  line
-                                 elif "RECAP" in line:
-                                   out_results = out_results +  line
-                                   recap_line =  f.readline()
-                                   while recap_line :
-                                     out_results = out_results +  recap_line
-                                     recap_line =  f.readline()
-                                 line = f.readline()
-                               f.close()
-                            out_obj = {"StatusCode": 200,
-                                     "StatusMessage": "FINISHED",
-                                     "PlaybookName": playbooknametxt,
-                                     "Version": "Version",
-                                     "Duration": 200,
-                                     "Results": out_results}
-                            return out_obj
-                       else:
-                          return {"StatusCode": 500, "StatusMessage": "PLAYBOOK FAILED "}
-                           
-
-                   return {"StatusCode": 500, "StatusMessage": "TEST ID UNDEFINED"}
-            else:
-                return {"StatusCode": 500, "StatusMessage": "MALFORMED REQUEST"}
-        elif 'DELETE' in cherrypy.request.method:
-            input_data = parse_query_string(cherrypy.request.query_string)
-            
-            cherrypy.log( "***> in RestServer.DELETE")
-            cherrypy.log("Payload: " + str(input_data))
-            
-            if input_data['Id'] in TestRecord:
-                if not 'PENDING' in TestRecord[input_data['Id']]['Result']:
-                    cherrypy.log(" Path: " + str(TestRecord[input_data['Id']]['Path']))
-                    TestRecord.pop (input_data['Id'])
-                    if input_data['Id'] in ActiveProcess:
-                        ActiveProcess.pop (input_data['Id'])
-
-                    return {"StatusCode": 200, "StatusMessage": "PLAYBOOK EXECUTION RECORDS DELETED"}
-                else:
-                    return {"StatusCode": 200, "StatusMessage": "PENDING"}
-            else:
-                return {"StatusCode": 500, "StatusMessage": "TEST ID UNDEFINED"}
-
-
-if __name__ == '__main__':
-
-    # Read configuration
-
-    config_file_path = "RestServer_config"
-
-    if not os.path.exists(config_file_path):
-        print '[INFO] The config file does not exist'
-        sys.exit(0)
-
-    ip = 'na'
-    AnsibleServer = 'na'
-    port = 'na'
-    tls = False
-    auth = False
-    pub = 'na'
-    priv = 'na'
-    timeout_seconds = 'na'
-    ansible_path = 'na'
-    ansible_temp = 'na'    
-    host = 'na'
-    users= 'na'
-    getresults_block = False
-    from_files = False
-    
-    file = open(config_file_path, 'r')
-    for line in file.readlines():
-        if '#' not in line:
-            if 'ip:' in line:
-                ip = line.split(':')[1].strip()
-            elif 'AnsibleServer:' in line:
-                AnsibleServer = line.split(':')[1].strip()
-            elif 'port:' in line:
-                port = line.split(':')[1].strip()
-            elif 'ksalt:' in line:
-                salt = line.split(':')[1].strip()
-            elif 'tls:' in line:
-                tls = 'YES' in line.split(':')[1].strip().upper()
-            elif 'auth:' in line:
-                auth = 'YES' in line.split(':')[1].strip().upper()
-            if tls and 'priv:' in line:
-                priv = line.split(':')[1].strip()
-            if tls and 'pub:' in line:
-                pub = line.split(':')[1].strip()
-            if tls and 'inter_cert:' in line:
-                intermediate = line.split(':')[1].strip()
-            if 'timeout_seconds' in line:
-                timeout_seconds = int (line.split(':')[1].strip())
-            if 'ansible_path' in line:
-                ansible_path = line.split(':')[1].strip()
-            if 'ansible_temp' in line:
-                ansible_temp = line.split(':')[1].strip()
-            if 'host' in line:
-                host = line.split(':')[1].strip()
-            if 'users' in line:
-                users = line.split(':')[1].strip()
-            if 'getresults_block' in line:
-                getresults_block = 'YES' in line.split(':')[1].strip().upper()
-            if 'from_files' in line:
-                from_files = 'YES' in line.split(':')[1].strip().upper()
-    file.close()
-
-    # Initialization
-    
-    global_conf = {
-        'global': {
-            'log.screen': True,
-            'response.timeout': 5400,
-            'server.socket_host': ip,
-            'server.socket_port': int(port),
-            'server.protocol_version': 'HTTP/1.1'
-            }
-        }
-
-    if tls:
-        # Use pythons built-in SSL
-        cherrypy.server.ssl_module = 'builtin'
-
-        # Point to certificate files
-
-        if not os.path.exists(pub):
-            print '[INFO] The public certificate does not exist'
-            sys.exit(0)
-
-        if not os.path.exists(priv):
-            print '[INFO] The private key does not exist'
-            sys.exit(0)            
-
-        if not os.path.exists(intermediate):
-            print '[INFO] The intermediate certificate does not exist'
-            sys.exit(0)
-
-        
-        cherrypy.server.ssl_certificate = pub
-        cherrypy.server.ssl_certificate_chain = intermediate
-        cherrypy.server.ssl_private_key = priv
-
-    if auth:
-        # Read in and build user dictionary
-        if not os.path.exists(users):
-          print '[INFO] The users file does not exist: ' + users
-          sys.exit(0)
-        userpassdict = {}
-        user_file = open(users, 'r')
-        for line in user_file.readlines():
-          if '#' not in line:
-            id = line.split(':')[0].strip()
-            pw = line.split(':')[1].strip()
-            userpassdict[id] = pw
-            #print str(userpassdict)
-
-        app_conf = {'/':
-                    {'tools.auth_basic.on': True,
-                     'tools.auth_basic.realm': 'earth',
-                     'tools.auth_basic.checkpassword': validate_password,
-                     }
-                    }
-
-        application = cherrypy.tree.mount(TestManager(), '/', app_conf)
-    else:
-        application = cherrypy.tree.mount(TestManager(), '/')
-        
-    cherrypy.config.update({
-        'log.access_file': "/var/log/RestServer.access"
-    })
-    accessLogName = "/var/log/RestServer.access"
-    applicationLogName = "/var/log/RestServer.log"
-    cherrypy.config.update(global_conf)
-
-    log = application.log
-    log.error_file = ""
-    log.access_file = ""
-    from logging import handlers
-    applicationLogFileHandler = handlers.RotatingFileHandler(applicationLogName, 'a', 1000000, 5000)
-    accessLogFileHandler = handlers.RotatingFileHandler(accessLogName, 'a', 1000000, 5000)
-    import logging
-    applicationLogFileHandler.setLevel(logging.DEBUG)
-    log.error_log.addHandler(applicationLogFileHandler)
-    log.access_log.addHandler(accessLogFileHandler)
-
-    # Start server
-    
-    cherrypy.engine.start()
-    cherrypy.engine.block()
diff --git a/ansible-server/src/main/scripts/startAnsibleServer.sh b/ansible-server/src/main/scripts/startAnsibleServer.sh
deleted file mode 100644 (file)
index 966a29a..0000000
+++ /dev/null
@@ -1,15 +0,0 @@
-#!/bin/bash
-exec &> >(tee -a "/var/log/ansible-server.log")
-
-if [ ! -f /tmp/.ansible-server-installed ]
-then
-    pip install 'cherrypy<18.0.0'
-    pip install requests
-
-    cp /etc/ansible/ansible.cfg /etc/ansible/ansible.cfg.orig
-    cat /etc/ansible/ansible.cfg.orig | sed -e 's/#host_key_checking/host_key_checking/' > /etc/ansible/ansible.cfg
-    date > /tmp/.ansible-server-installed 2>&1
-fi
-
-cd /opt/onap/ccsdk
-exec /usr/bin/python RestServer.py

© 2017 ONAP. Copyright © The Linux Foundation ®. All Rights Reserved.
The Linux Foundation has registered trademarks and uses trademarks.
For a list of trademarks of The Linux Foundation, please see our Trademark Usage page.
Linux is a registered trademark of Linus Torvalds.
Privacy Policy and Terms of Use