docs/APPC Logging Guide/APPC Logging Guide.rst

   1 .. ============LICENSE_START==========================================
   2 .. ===================================================================
   3 .. Copyright © 2017 AT&T Intellectual Property. All rights reserved.
   4 .. ===================================================================
   5 .. Licensed under the Creative Commons License, Attribution 4.0 Intl.  (the "License");
   6 .. you may not use this documentation except in compliance with the License.
   7 .. You may obtain a copy of the License at
   8 ..
   9 ..  https://creativecommons.org/licenses/by/4.0/
  10 ..
  11 .. Unless required by applicable law or agreed to in writing, software
  12 .. distributed under the License is distributed on an "AS IS" BASIS,
  13 .. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  14 .. See the License for the specific language governing permissions and
  15 .. limitations under the License.
  16 .. ============LICENSE_END============================================
  17 .. ECOMP is a trademark and service mark of AT&T Intellectual Property.
  18
  19 APPC Logging Guide
  20 ==================
  21
  22 The APPC modules manage logging functionality according to the logging
  23 configuration file ``org.ops4j.pax.logging.cfg``. APPC configuration defines
  24 the location of the logging configuration file. Each APPC module
  25 configured to generate logs reads the configuration file periodically
  26 and generates logs according to the current logging level.
  27
  28 Logging Architecture
  29 ---------------------
  30
  31 The following diagram illustrates the APPC logging architecture.
  32
  33 |image0|
  34
  35 Log Types
  36 ---------
  37
  38 APPC generates several types of logs to capture information needed to
  39 operate, troubleshoot, and report on performance:
  40
  41 -  **Audit Log**: The Audit Log provides a summary view of the
  42    processing of a request – a transaction – within an application. It
  43    captures activity requests received by APPC and includes such
  44    information as the time the activity initiated, when it finishes, and
  45    the API invoked at the component.
  46
  47 -  **Metric Log**: The Metrics Log provides a more detailed view into
  48    the processing of a transaction within APP-C. It captures the
  49    beginning and ending of activities necessary to complete the
  50    transaction within APPC modules, for example, the Dispatcher module,
  51    and other ONAP components such as A&AI.
  52
  53 -  **Error Log**: The Error Log captures INFO, WARN, ERROR, and FATAL
  54    conditions sensed (“exception handled”) by the APPC software
  55    components.
  56
  57 -  **Debug Log**: The optional Debug Log captures data that may be required to debug and correct abnormal conditions of the application.
  58
  59
  60 The APPC modules manage logging functionality according to the logging
  61 configuration file ``org.ops4j.pax.logging.cfg``. APPC configuration defines
  62 the location of the logging configuration file. Each APPC module
  63 configured to generate logs reads the configuration file periodically
  64 and generates logs according to the current logging level.
  65
  66 Logging Levels
  67 ~~~~~~~~~~~~~~
  68
  69 +-------------+----------------+---------------+-------------+-------------+------------+
  70 | **Level**   | **Log type**                                                            |
  71 +=============+================+===============+=============+=============+============+
  72 |             |                                | **Errors**                             |
  73 +-------------+----------------+---------------+-------------+-------------+------------+
  74 |             | **Audit**      | **Metrics**   | **FATAL**   | **ERROR**   | **WARN**   |
  75 +-------------+----------------+---------------+-------------+-------------+------------+
  76 | OFF         | No             | No            | No          | No          | No         |
  77 +-------------+----------------+---------------+-------------+-------------+------------+
  78 | FATAL       | Yes            | Yes           | Yes         | No          | No         |
  79 +-------------+----------------+---------------+-------------+-------------+------------+
  80 | ERROR       | Yes            | Yes           | Yes         | Yes         | No         |
  81 +-------------+----------------+---------------+-------------+-------------+------------+
  82 | WARN        | Yes            | Yes           | Yes         | Yes         | Yes        |
  83 +-------------+----------------+---------------+-------------+-------------+------------+
  84
  85 Response Codes
  86 ~~~~~~~~~~~~~~
  87
  88 +----------------------------+--------------------------------+
  89 | **Response code ranges**   | **Error type**                 |
  90 +============================+================================+
  91 | 100-199                    | Permission errors              |
  92 +----------------------------+--------------------------------+
  93 | 200-299                    | Availability errors/Timeouts   |
  94 +----------------------------+--------------------------------+
  95 | 300-399                    | Data errors                    |
  96 +----------------------------+--------------------------------+
  97 | 400-499                    | Schema errors                  |
  98 +----------------------------+--------------------------------+
  99 | 500-599                    | Business process errors        |
 100 +----------------------------+--------------------------------+
 101 | 900-999                    | Unknown errors                 |
 102 +----------------------------+--------------------------------+
 103
 104 Logging Output
 105 ---------------
 106
 107 APPC generates logging output according to Event and Error Logging Framework(EELF)requirements in the
 108 following format:
 109
 110 Error Log
 111 ~~~~~~~~~~
 112
 113 +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 114 |     **Field**           |     **Description**                                                                                                                                                                                                                             |
 115 +=========================+=================================================================================================================================================================================================================================================+
 116 |     Timestamp           |     Date-time when error occurs in UTC.                                                                                                                                                                                                         |
 117 +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 118 |     RequestID           |     Universally unique transaction requestID (UUID).                                                                                                                                                                                            |
 119 +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 120 |     ThreadId            |     Blank                                                                                                                                                                                                                                       |
 121 |                         |                                                                                                                                                                                                                                                 |
 122 |                         |     This traces processing of a request over a number of threads of a single ONAP component.                                                                                                                                                    |
 123 +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 124 |     ServiceName         |     Externally advertised API invoked by clients of this component, for example, "Audit", "HealthCheck".                                                                                                                                        |
 125 +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 126 |     PartnerName         |     Client or user invoking the API.                                                                                                                                                                                                            |
 127 +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 128 |     TargetEntity        |     ONAP component/subcomponent or non-ONAP entity invoked for this suboperation’s activities, for example, A&AI, VF, VM, MySql.                                                                                                                |
 129 +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 130 |     TargetServiceName   |     Known name of External API/operation activities invoked on TargetEntity (ONAP component/subcomponent or non-ONAP entity), for example, VM UUID, VF UUID.                                                                                    |
 131 +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 132 |     ErrorCategory       |     WARN or ERROR                                                                                                                                                                                                                               |
 133 +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 134 |     ErrorCode           |     Code representing the error condition according to the error categories.                                                                                                                                                                    |
 135 +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 136 |     ErrorDescription    |     Human readable description of the error - Error name, for example: "CONFIGURATION\_STARTED".                                                                                                                                                |
 137 +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 138 |     detailMessage       |     This field may contain any additional information useful in describing the error condition, for example:                                                                                                                                    |
 139 |                         |                                                                                                                                                                                                                                                 |
 140 |                         |     ``Application Controller (APP-C) initialization started at {0}|\``                                                                                                                                                                          |
 141 |                         |     ``No resolution is required, this is an informational message|\``                                                                                                                                                                           |
 142 |                         |     ``The APP-C component configuration has been started because the component is being initialized or loaded for the first time, or a new instance of the component is being created. This message indicates that the component is starting.`` |
 143 +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 144
 145 Debug Log
 146 ~~~~~~~~~
 147
 148 +-----------------+------------------------------------------------------------+
 149 |     **Field**   |     **Description**                                        |
 150 +=================+============================================================+
 151 | Timestamp       | Date-time of the log record.                               |
 152 +-----------------+------------------------------------------------------------+
 153 | RequestID       | Universally unique transaction requestID (UUID).           |
 154 +-----------------+------------------------------------------------------------+
 155 | DebugInfo       | Debug Information                                          |
 156 +-----------------+------------------------------------------------------------+
 157 | End of Record   | Designates the logical end of a multi-line debug record.   |
 158 +-----------------+------------------------------------------------------------+
 159
 160 Audit Log
 161 ~~~~~~~~~
 162
 163 +--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 164 |     **Field**                        |     **Description**                                                                                                                                                                                       |
 165 +======================================+===========================================================================================================================================================================================================+
 166 |     BeginTimestamp                   |     Date-time of the start of a request activity.                                                                                                                                                         |
 167 +--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 168 |     EndTimestamp                     |     Date-time of the end of a request activity.                                                                                                                                                           |
 169 +--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 170 |     RequestID                        |     Universally unique transaction request ID (UUID).                                                                                                                                                     |
 171 +--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 172 |     serviceInstanceID                |     Uniquely identifies a service instance, for example, “service graph”. The primary key, for example, in A&AI, to reference or manage the service instance as a unit.                                   |
 173 +--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 174 |     threadId                         |     Empty                                                                                                                                                                                                 |
 175 +--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 176 |     physical/virtual server name     |     Empty (the value added by the log files collecting agent).                                                                                                                                            |
 177 +--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 178 |     serviceName                      |     Externally advertised API invoked by clients of this component, for example, "Audit", "HealthCheck".                                                                                                  |
 179 +--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 180 |     PartnerName                      |     Client or user invoking the API.                                                                                                                                                                      |
 181 +--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 182 |     StatusCode                       |     High-level success or failure of the request (COMPLETE or ERROR).                                                                                                                                     |
 183 +--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 184 |     ResponseCode                     |     Application specific response code - LCM API error codes categorized according to the logging categories.                                                                                             |
 185 +--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 186 |     ResponseDescription              |     Human readable description of the application specific response code, for example, "INVALID INPUT PARAMETER - ${detailedErrorMsg}".                                                                   |
 187 +--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 188 |     instanceUUID                     |     Universally unique identifier to differentiate between multiple instances of the same (named), log writing component - the specific APPC instance UUID.                                               |
 189 +--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 190 |     Category log level               |     Enum: “INFO” \| “WARN” \|”DEBUG” \| “ERROR” \| “FATAL”. Current log level for the entire APP-C.                                                                                                       |
 191 +--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 192 |     Severity                         |     Blank                                                                                                                                                                                                 |
 193 +--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 194 |     Server IP address                |     Blank                                                                                                                                                                                                 |
 195 +--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 196 |     ElapsedTime                      |     Elapsed time to complete processing of an API or request at the granularity available to the component system. This value should be the difference between BeginTimestamp and EndTimestamp fields.    |
 197 +--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 198 |     Server                           |     VM FQDN if virtualized, otherwise the host name of the logging component.                                                                                                                             |
 199 +--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 200 |     ClientIPaddress                  |     Requesting remote client application’s IP address if known, otherwise empty.                                                                                                                          |
 201 +--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 202
 203 Metrics Log
 204 ~~~~~~~~~~~
 205
 206 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 207 |     **Field**                                 |     **Description**                                                                                                                                                                                              |
 208 +===============================================+==================================================================================================================================================================================================================+
 209 | BeginTimestamp                                | Date-time when a suboperation activity is begun in UTC                                                                                                                                                           |
 210 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 211 | EndTimestamp                                  | Date-time when a supoperation activity is completed in UTC                                                                                                                                                       |
 212 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 213 | RequestID                                     | Universally unique transaction request ID (UUID)                                                                                                                                                                 |
 214 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 215 | serviceInstanceID                             | VMUUID, VFUUID                                                                                                                                                                                                   |
 216 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 217 | threadId                                      | Optional                                                                                                                                                                                                         |
 218 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 219 | physical/virtual server name                  | Empty if its value can be added by the log files collecting agent.                                                                                                                                               |
 220 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 221 | serviceName                                   | For example: "Audit", "HealthCheck" etc                                                                                                                                                                          |
 222 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 223 | PartnerName Client or user invoking the API   |                                                                                                                                                                                                                  |
 224 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 225 | TargetEntity                                  | APPC internal subcomponent, for example, MD-SAL, or external component, for example, A&AI, SSH, Netconf, invoked for this suboperation.                                                                          |
 226 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 227 | TargetServiceName                             | Operation activities invoked on TargetEntity e.g. A&AI GET generic- vnf                                                                                                                                          |
 228 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 229 | StatusCode                                    | High level success or failure of the suboperation activities (COMPLETE or ERROR)                                                                                                                                 |
 230 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 231 | ResponseCode                                  | Specific response code returned by the suboperation activities.                                                                                                                                                  |
 232 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 233 | ResponseDescription                           | Human readable description of the response code.                                                                                                                                                                 |
 234 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 235 | instanceUUID                                  | APPC instance ID.                                                                                                                                                                                                |
 236 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 237 | Category log level                            | Enumerated values: “INFO” \| “WARN” \|”DEBUG” \| “ERROR” \| “FATAL”.                                                                                                                                             |
 238 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 239 | Severity                                      | Empty                                                                                                                                                                                                            |
 240 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 241 | Server IP address                             | The logging component host server’s IP address.                                                                                                                                                                  |
 242 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 243 | ElapsedTime                                   | Elapsed time to complete processing of the sub operation activities at the granularity available to the component system.  This value should be the difference between EndTimestamp and BeginTimestamp fields.   |
 244 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 245 | Server                                        | VM FQDN if virtualized, otherwise the host name of the logging component.                                                                                                                                        |
 246 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 247 | ClientIP                                      | Requesting remote client application’s IP address.                                                                                                                                                               |
 248 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 249 | class name                                    | Optional. The name of the class that has caused the log record creation. For OO programing languages that support this concept.                                                                                  |
 250 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 251 | Unused                                        |                                                                                                                                                                                                                  |
 252 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 253 | ProcessKey                                    | Optional                                                                                                                                                                                                         |
 254 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 255 | TargetVirtualEntity                           | Empty                                                                                                                                                                                                            |
 256 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 257 | CustomField1                                  | Empty (specific attributes exposed by developers)                                                                                                                                                                |
 258 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 259 | CustomField2                                  | Empty                                                                                                                                                                                                            |
 260 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 261 | CustomField3                                  | Empty                                                                                                                                                                                                            |
 262 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 263 | CustomField4                                  | Empty                                                                                                                                                                                                            |
 264 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 265 | detailMessage                                 | Empty                                                                                                                                                                                                            |
 266 +-----------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 267
 268 Log File Locations
 269 ------------------
 270
 271 The logging configuration file, ``org.ops4j.pax.logging.cfg`` are located in
 272 appc Git repository:
 273
 274 ``/appc/appc-common/src/main/resources/org/onap/appc/org.ops4j.pax.logging.cfg``
 275
 276
 277 The logs are stored at the location defined by the appropriate appender:
 278
 279 ``log4j.appender.error.File=${karaf.data}/log/APPC/appc-error.log``
 280
 281 ``log4j.appender.debug.file=${karaf.data}/log/APPC/appc-debug.log``
 282
 283 ``log4j.appender.metric.File=${karaf.data}/log/APPC/appc-metric.log``
 284
 285 ``log4j.appender.audit.File=${karaf.data}/log/APPC/appc-audit.log``
 286
 287
 288 Enabling APPC Logging
 289 ----------------------
 290
 291 APPC uses Event and Error Logging Framework (EELF) for application logs.
 292 To enable EELF logging:
 293
 294 1. Replace the default configuration file located at
 295    ``/opt/opendaylight/current/etc/org.ops4j.pax.logging.cfg``
 296
 297    with the configuration file that is checked into git:
 298
 299    ``/appc/appc-common/src/main/resources/org/onap/appc/org.ops4j.pax.logging.cfg``
 300
 301 2. Stop and restart ODL controller for the configuration changes to take
 302    effect.
 303
 304 3. Verify logging changes at the following log paths:
 305
 306    -  ``/opt/opendaylight/current/data/log/eelf/karaf.log``
 307       This log contains the regular karaf.log output reformatted to use
 308       the EELF MDC properties and the pattern that is configured in the
 309       ``org.ops4j.pax.logging.cfg`` file.
 310    -  ``/opt/opendaylight/current/data/log/APPC/<package-name>``
 311       This directory contains the audit, metric, error, and debug logs that are configured in the ``org.ops4j.pax.logging.cfg`` file.
 312
 313
 314 **Note:**
 315    ``/opt/opendaylight/current/data/log/APPC/controller`` contains the logs generated from the package ``org.openecomp.\*`` (all APPC logs)
 316
 317 -  Error.log: alarms –ERROR level logs and above
 318
 319 -  Info.log: INFO level logs only
 320
 321 -  Debug.log: debugging – DEBUG level and above
 322
 323 -  Audit – AUDIT level and above
 324
 325 Log Rotation
 326 ------------
 327
 328 Log rotation is performed after every 100 MB size limit is reached. The
 329 log rotation interval is defined as part of the EELF framework.
 330
 331 .. |image0| image:: APPCLoggingArchitecturediagram.png
 332    :width: 6.49097in
 333    :height: 3.46181in
 334