1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
2 .. http://creativecommons.org/licenses/by/4.0
3 .. Copyright 2017 AT&T Intellectual Property. All rights reserved.
5 Monitoring & Management
6 -----------------------
8 This section addresses data collection and event processing
9 functionality that is directly dependent on the interfaces
10 provided by the VNFs’ APIs. These can be in the form of asynchronous
11 interfaces for event, fault notifications, and autonomous data streams.
12 They can also be synchronous interfaces for on-demand requests to
13 retrieve various performance, usage, and other event information.
15 The target direction for VNF interfaces is to employ APIs that are
16 implemented utilizing standardized messaging and modeling protocols
17 over standardized transports. Migrating to a virtualized environment
18 presents a tremendous opportunity to eliminate the need for proprietary
19 interfaces for VNF provider equipment while removing the traditional
20 boundaries between Network Management Systems and Element Management
21 Systems. Additionally, VNFs provide the ability to instrument the
22 networking applications by creating event records to test and monitor
23 end-to-end data flow through the network, similar to what physical or
24 virtual probes provide without the need to insert probes at various
25 points in the network. The VNF providers must be able to provide the
26 aforementioned set of required data directly to the ONAP collection
27 layer using standardized interfaces.
29 Data Model for Event Records
30 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
32 This section describes the data model for the collection of telemetry
33 data from VNFs by Service Providers (SPs) to manage VNF health and
34 runtime lifecycle. This data model is referred to as the VNF Event
35 Streaming (VES) specifications. While this document is focused on
36 specifying some of the records from the ONAP perspective, there may
37 be other external bodies using the same framework to specify additional
38 records. For example, OPNFV has a VES project that is looking to specify
39 records for OpenStack’s internal telemetry to manage Application (VNFs),
40 physical and virtual infrastructure (compute, storage, network devices),
41 and virtual infrastructure managers (cloud controllers, SDN controllers).
42 Note that any configurable parameters for these data records (e.g.,
43 frequency, granularity, policy-based configuration) will be managed
44 using the “Configuration” framework described in the prior sections
47 The Data Model consists of:
49 - Common Header Record: This data structure precedes each of the
50 Technology Independent and Technology Specific records sections of
53 - Technology Independent Records: This version of the document
54 specifies the model for Fault, Heartbeat, State Change, Syslog,
55 Threshold Crossing Alerts, and VNF Scaling* (short for
56 measurementForVfScalingFields – actual name used in JSON
57 specification) records. In the future, these may be extended to
58 support other types of technology independent records. Each of
59 these records allows additional fields (name/ value pairs) for
60 extensibility. The VNF provider can use these VNF Provider-specific
61 additional fields to provide additional information that may be
62 relevant to the managing systems.
64 - Technology Specific Records: This version of the document specifies
65 the model for Mobile Flow records, Signaling and Voice Quality records.
66 In the future, these may be extended to support other types of records
67 (e.g. Network Fabric, Security records, etc.). Each of these records
68 allows additional fields (name/value pairs) for extensibility. The VNF
69 providers can use these VNF-specific additional fields to provide
70 additional information that may be relevant to the managing systems.
71 A placeholder for additional technology specific areas of interest to
72 be defined in the future documents has been depicted.
76 Figure 1. Data Model for Event Records
78 Event Records - Data Structure Description
79 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
81 The data structure for event records consists of:
83 - a Common Event Header block;
85 - zero or more technology independent domain blocks; and
87 - e.g., Fault domain, State Change domain, Syslog domain, etc.
89 - zero or more technology specific domain blocks.
91 - e.g., Mobile Flow domain, Signaling domain, Voice Quality domain,
97 The common header that precedes any of the domain-specific records contains
98 information identifying the type of record to follow, information about
99 the sender and other identifying characteristics related to timestamp,
100 sequence number, etc.
102 Technology Independent Records – Fault Fields
103 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
105 The Fault Record, describing a condition in the Fault domain, contains
106 information about the fault such as the entity under fault, the
107 severity, resulting status, etc.
109 Technology Independent Records – Heartbeat Fields
110 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
112 The Heartbeat Record provides an optional structure for communicating
113 information about heartbeat or watchdog signaling events. It can
114 contain information about service intervals, status information etc.
115 as required by the heartbeat implementation.
117 Note: Heartbeat records would only have the Common Event Header block.
118 An optional heartbeat domain is available if required by the heartbeat
121 Technology Independent Records – State Change Fields
122 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
124 The State Change Record provides a structure for communicating information
125 about data flow through the VNF. It can contain information about state
126 change related to physical device that is reported by VNF. As an example,
127 when cards or port name of the entity that has changed state.
129 Technology Independent Records – Syslog Fields
130 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
132 The Syslog Record provides a structure for communicating any type of
133 information that may be logged by the VNF. It can contain information
134 about system internal events, status, errors, etc.
136 Technology Independent Records – Threshold Crossing Alert Fields
137 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
139 The Threshold Crossing Alert (TCA) Record provides a structure for
140 communicating information about threshold crossing alerts. It can
141 contain alert definitions and types, actions, events, timestamps
142 and physical or logical details.
144 Technology Independent Records - VNF Scaling Fields
145 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
147 The VNF Scaling\* (short for measurementForVfScalingFields –
148 actual name used in JSON specification) Record contains information
149 about VNF and VNF resource structure and its condition to help in
150 the management of the resources for purposes of elastic scaling.
152 Technology Independent Records – otherFields
153 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
155 The otherFields Record defines fields for events belonging to the
156 otherFields domain of the Technology Independent domain enumeration.
157 This record provides a mechanism to convey a complex set of fields
158 (possibly nested or opaque) and is purely intended to address
159 miscellaneous needs such as addressing time-to-market considerations
160 or other proof-of-concept evaluations. Hence, use of this record
161 type is discouraged and should be minimized.
163 Technology Specific Records – Mobile Flow Fields
164 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
166 The Mobile Flow Record provides a structure for communicating
167 information about data flow through the VNF. It can contain
168 information about connectivity and data flows between serving
169 elements for mobile service, such as between LTE reference points, etc.
171 Technology Specific Records – Signaling Fields
172 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
174 The Signaling Record provides a structure for communicating information
175 about signaling messages, parameters and signaling state. It can
176 contain information about data flows for signaling and controlling
177 multimedia communication sessions such as voice and video calls.
179 Technology Specific Records – Voice Quality Fields
180 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
181 The Voice Quality Record provides a structure for communicating information
182 about voice quality statistics including media connection information,
183 such as transmitted octet and packet counts, packet loss, packet delay
184 variation, round-trip delay, QoS parameters and codec selection.
186 Technology Specific Records – Future Domains
187 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
189 The futureDomains Record is a placeholder for additional technology
190 specific areas of interest that will be defined and described
191 in the future documents.
193 Data Structure Specification of the Event Record
194 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
196 For additional information on the event record formats of the data
197 structures mentioned above, please refer to `VES Event
198 Listener <https://github.com/att/evel-test-collector/tree/master/docs/att_interface_definition>`__.
200 Transports and Protocols Supporting Resource Interfaces
201 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
203 Delivery of data from VNFs to ONAP must use the common transport
204 mechanisms and protocols for all VNFs as defined in this document.
205 Transport mechanisms and protocols have been selected to enable both
206 high volume and moderate volume datasets, as well as asynchronous and
207 synchronous communications over secure connections. The specified
208 encoding provides self-documenting content, so data fields can be
209 changed as needs evolve, while minimizing changes to data delivery.
211 The term ‘Event Record’ is used throughout this document to represent
212 various forms of telemetry or instrumentation made available by the
213 VNF including, faults, status events, various other types of VNF
214 measurements and logs. Headers received by themselves must be used
215 as heartbeat indicators. Common structures and delivery protocols for
216 other types of data will be given in future versions of this document
217 as we get more insight into data volumes and required processing.
219 In the following sections, we provide options for encoding, serialization
220 and data delivery. Agreements between Service Providers and VNF providers
221 shall determine which encoding, serialization and delivery method to use
222 for particular data sets. The selected methods must be agreed to prior to
223 the on-boarding of the VNF into ONAP design studio.
225 VNF Telemetry using VES/JSON Model
226 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
228 The preferred model for data delivery from a VNF to ONAP DCAE is
229 the JSON driven model as depicted in Figure 2.
233 Figure 2. VES/JSON Driven Model
235 VNF providers will provide a YAML artifact to the Service Provider
238 * standard VES/JSON model information elements (key/values) that
240 * any additional non-standard (custom) VES/JSON model information
241 elements (key/values) that the VNF provides
243 Using the semantics and syntax supported by YAML, VNF providers
244 will indicate specific conditions that may arise, and recommend
245 actions that should be taken at specific thresholds, or if specific
246 conditions repeat within a specified time interval.
248 Based on the VNF provider's recommendations, the Service Provider may
249 create additional YAML artifacts (using ONAP design Studio), which
250 finalizes Service Provider engineering rules for the processing of
251 the VNF events. The Service Provider may alter the threshold levels
252 recommended by the VNF providor, and may modify and more clearly
253 specify actions that should be taken when specified conditions arise.
254 The Service Provider-created version of the YAML artifact will be
255 distributed to ONAP applications by the Design framework.
257 VNF Telemetry using YANG Model
258 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
260 In addition to the JSON driven model described above, a YANG
261 driven model can also be supported, as depicted in Figure 3.
265 Figure 3. YANG Driven Model
267 VNF providers will provide to the Service Provider the following
268 YANG model artifacts:
270 * common IETF YANG modules that support the VNF
271 * native (VNF provider-supplied) YANG modules that support the VNF
272 * open (OpenConfig) YANG modules and the following
273 configuration-related information, including:
275 * telemetry configuration and operational state data; such as:
278 * subscription bindings
281 * transport mechanisms
284 * a YAML artifact that provides all necessary mapping relationships
285 between YANG model data types to VES/JSON information elements
286 * YANG helper or decoder functions that automate the conversion between
287 YANG model data types to VES/JSON information elements
288 * OPTIONAL: YANG Telemetry modules in JSON format per RFC 7951
290 Using the semantics and syntax supported by YANG, VNF providers
291 will indicate specific conditions that may arise, and recommend
292 actions that should be taken at specific thresholds, or if specific
293 conditions repeat within a specified time interval.
295 Based on the VNF provider's recommendations, the Service Provider may
296 create additional YAML artifacts (using ONAP design Studio), which
297 finalizes Service Provider engineering rules for the processing of the
298 VNF events. The Service Provider may alter the threshold levels recommended
299 by the VNF provider, and may modify and more clearly specify actions that
300 should be taken when specified conditions arise. The Service
301 Provided-created version of the YAML will be distributed to ONAP
302 applications by the Design framework.
304 Note: While supporting the YANG model described above, we are still
305 leveraging the VES JSON based model in DCAE. The purpose of the
306 diagram above is to illustrate the concept only and not to imply a
307 specific implementation.
309 VNF Telemetry using Google Protocol Buffers
310 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
312 In addition to the data delivery models described above, support for
313 delivery of VNF telemetry using Google Protocol Buffers (GPB) can
314 also be supported, as depicted in Figure 4.
316 VNF providers will provide to the Service Provider the additional
317 following artifacts to support the delivery of VNF telemetry to DCAE
318 via the open-source gRPC mechanism using Google's Protocol Buffers:
320 * the YANG model artifacts described in support of the
321 "VNF Telemetry using YANG Model"
322 * valid definition file(s) for all GPB / KV-GPB encoded messages
323 * valid definition file(s) for all gRPC services
324 * gRPC method parameters and return types specified as Protocol
329 Figure 4. Protocol Buffers Driven Model
331 Note: if Google Protocol Buffers are employed for delivery of VNF
332 telemetry, Key-Value Google Protocol Buffers (KV-GPB) is the
333 preferred serialization method. Details of specifications and
334 versioning corresponding to a release can be found at:
335 `VES Event Listener <https://github.com/att/evel-test-collector/tree/master/docs/att_interface_definition>`__.
337 Note: While supporting the VNF telemetry delivery approach described above,
338 we are still leveraging the VES JSON based model in DCAE. The purpose of
339 the diagram above is to illustrate the concept only and not to imply a
340 specific implementation.
342 Monitoring & Management Requirements
343 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
345 VNF telemetry via standardized interface
346 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
354 The xNF **MUST** provide all telemetry (e.g., fault event
355 records, syslog records, performance records etc.) to ONAP using the
356 model, format and mechanisms described in this section.
358 Encoding and Serialization
359 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
361 Content delivered from VNFs to ONAP is to be encoded and serialized using JSON:
372 The xNF **MUST** encode and serialize content delivered to
373 ONAP using JSON (RFC 7159) plain text format. High-volume data
374 is to be encoded and serialized using `Avro <http://avro.apache.org/>`_,
375 where the Avro [1]_ data format are described using JSON.
379 - JSON plain text format is preferred for moderate volume data sets
380 (option 1), as JSON has the advantage of having well-understood simple
381 processing and being human-readable without additional decoding. Examples
382 of moderate volume data sets include the fault alarms and performance
383 alerts, heartbeat messages, measurements used for xNF scaling and syslogs.
384 - Binary format using Avro is preferred for high volume data sets
385 (option 2) such as mobility flow measurements and other high-volume
386 streaming events (such as mobility signaling events or SIP signaling)
387 or bulk data, as this will significantly reduce the volume of data
388 to be transmitted. As of the date of this document, all events are
389 reported using plain text JSON and REST.
390 - Avro content is self-documented, using a JSON schema. The JSON schema is
391 delivered along with the data content
392 (http://avro.apache.org/docs/current/ ). This means the presence and
393 position of data fields can be recognized automatically, as well as the
394 data format, definition and other attributes. Avro content can be
395 serialized as JSON tagged text or as binary. In binary format, the
396 JSON schema is included as a separate data block, so the content is
397 not tagged, further compressing the volume. For streaming data, Avro
398 will read the schema when the stream is established and apply the
399 schema to the received content.
401 In addition to the preferred method (JSON), content can be delivered
402 from xNFs to ONAP can be encoded and serialized using Google Protocol
408 Telemetry data delivered using Google Protocol Buffers v3 (proto3)
409 can be serialized in one of the following methods:
411 * Key-value Google Protocol Buffers (KV-GPB) is also known as
414 * keys are strings that correspond to the path of the system
415 resources for the VNF being monitored.
416 * values correspond to integers or strings that identify the
417 operational state of the VNF resource, such a statistics counters
418 and the state of a VNF resource.
420 * VNF providers must supply valid KV-GPB definition file(s) to allow
421 for the decoding of all KV-GPB encoded telemetry messages.
423 * Native Google Protocol Buffers (GPB) is also known as compact GPB:
425 * keys are represented as integers pointing to the system resources for
426 the VNF being monitored.
427 * values correspond to integers or strings that identify the operational
428 state of the VNF resource, such a statistics counters and the state
431 * Google Protocol Buffers (GPB) requires metadata in the form of .proto
432 files. VNF providers must supply the necessary GPB .proto files such that
433 GPB telemetry messages can be encoded and decoded.
435 * In the future, we may consider support for other types of
436 encoding & serialization methods based on industry demand.
440 ~~~~~~~~~~~~~~~~~~~~~
447 The xNF **MUST** vary the frequency that asynchronous data
448 is delivered based on the content and how data may be aggregated or
453 - For example, alarms and alerts are expected to be delivered as
454 soon as they appear. In contrast, other content, such as performance
455 measurements, KPIs or reported network signaling may have various
456 ways of packaging and delivering content. Some content should be
457 streamed immediately; or content may be monitored over a time
458 interval, then packaged as collection of records and delivered
459 as block; or data may be collected until a package of a certain
460 size has been collected; or content may be summarized statistically
461 over a time interval, or computed as a KPI, with the summary or KPI
463 - We expect the reporting frequency to be configurable depending on
464 the virtual network functions needs for management. For example,
465 Service Provider may choose to vary the frequency of collection
466 between normal and trouble-shooting scenarios.
467 - Decisions about the frequency of data reporting will affect
468 the size of delivered data sets, recommended delivery method,
469 and how the data will be interpreted by ONAP. These considerations
470 should not affect deserialization and decoding of the data, which
471 will be guided by the accompanying JSON schema or GPB definition
474 Addressing and Delivery Protocol
475 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
477 ONAP destinations can be addressed by URLs for RESTful data PUT. Future
478 data sets may also be addressed by host name and port number for TCP
479 streaming, or by host name and landing zone directory for SFTP transfer
487 The xNF **SHOULD** use REST using HTTPS delivery of plain
488 text JSON for moderate sized asynchronous data sets, and for high
489 volume data sets when feasible.
496 The xNF **MUST** have the capability of maintaining a primary
497 and backup DNS name (URL) for connecting to ONAP collectors, with the
498 ability to switch between addresses based on conditions defined by policy
499 such as time-outs, and buffering to store messages until they can be
500 delivered. At its discretion, the service provider may choose to populate
501 only one collector address for a xNF. In this case, the network will
502 promptly resolve connectivity problems caused by a collector or network
503 failure transparently to the xNF.
510 The xNF **MUST** be configured with initial address(es) to use
511 at deployment time. Subsequently, address(es) may be changed through
512 ONAP-defined policies delivered from ONAP to the xNF using PUTs to a
513 RESTful API, in the same manner that other controls over data reporting
514 will be controlled by policy.
521 The xNF **MAY** use another option which is expected to include REST
522 delivery of binary encoded data sets.
529 The xNF **MAY** use another option which is expected to include TCP
530 for high volume streaming asynchronous data sets and for other high volume
531 data sets. TCP delivery can be used for either JSON or binary encoded data
539 The xNF **MAY** use another option which is expected to include SFTP
540 for asynchronous bulk files, such as bulk files that contain large volumes
541 of data collected over a long time interval or data collected across many
542 xNFs. (Preferred is to reorganize the data into more frequent or more focused
543 data sets, and deliver these by REST or TCP as appropriate.)
550 The xNF **MAY** use another option which is expected to include REST
551 for synchronous data, using RESTCONF (e.g., for xNF state polling).
558 The xNF **MUST**, by ONAP Policy, provide the ONAP addresses
559 as data destinations for each xNF, and may be changed by Policy while
560 the xNF is in operation. We expect the xNF to be capable of redirecting
561 traffic to changed destinations with no loss of data, for example from
562 one REST URL to another, or from one TCP host and port to another.
564 Asynchronous and Synchronous Data Delivery
565 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
572 The xNF **MUST** deliver asynchronous data as data becomes
573 available, or according to the configured frequency.
580 The xNF **MUST** must encode, address and deliver the data
581 as described in the previous paragraphs.
588 The xNF **MUST** respond to data requests from ONAP as soon
589 as those requests are received, as a synchronous response.
596 The xNF **MUST** use the RESTCONF/NETCONF framework used by
597 the ONAP configuration subsystem for synchronous communication.
604 The xNF **MUST** use the YANG configuration models and RESTCONF
605 [RFC8040] (https://tools.ietf.org/html/rfc8040).
612 The xNF **MUST** respond with content encoded in JSON, as
613 described in the RESTCONF specification. This way the encoding of a
614 synchronous communication will be consistent with Avro.
621 The xNF **MUST** respond to an ONAP request to deliver the
622 current data for any of the record types defined in
623 `Event Records - Data Structure Description`_ by returning the requested
624 record, populated with the current field values. (Currently the defined
625 record types include fault fields, mobile flow fields, measurements for
626 xNF scaling fields, and syslog fields. Other record types will be added
627 in the future as they become standardized and are made available.)
634 The xNF **MUST** respond to an ONAP request to deliver granular
635 data on device or subsystem status or performance, referencing the YANG
636 configuration model for the xNF by returning the requested data elements.
643 The xNF **SHOULD** use `Modeling JSON text with YANG
644 <https://tools.ietf.org/html/rfc7951>`_, If YANG models need to be
645 translated to and from JSON{RFC7951]. YANG configuration and content can
646 be represented via JSON, consistent with Avro, as described in "Encoding
647 and Serialization" section.
657 The xNF **MUST** support secure connections and transports such as
658 Transport Layer Security (TLS) protocol
659 [`RFC5246 <https://tools.ietf.org/html/rfc5246>`_] and should adhere to
660 the best current practices outlined in
661 `RFC7525 <https://tools.ietf.org/html/rfc7525>`_.
668 The xNF **MUST** control access to ONAP and to xNFs, and creation
669 of connections, through secure credentials, log-on and exchange mechanisms.
676 The xNF **MUST** carry data in motion only over secure connections.
683 The xNF **MUST** encrypt any content containing Sensitive Personal
684 Information (SPI) or certain proprietary data, in addition to applying the
685 regular procedures for securing access and delivery.
688 This option is not currently supported in ONAP and it is currently
691 .. |image0| image:: ../Data_Model_For_Event_Records.png
695 .. |image1| image:: ../VES_JSON_Driven_Model.png
699 .. |image2| image:: ../YANG_Driven_Model.png
703 .. |image3| image:: ../Protocol_Buffers_Driven_Model.png