1 .. Modifications Copyright © 2017-2018 AT&T Intellectual Property.
3 .. Licensed under the Creative Commons License, Attribution 4.0 Intl.
4 (the "License"); you may not use this documentation except in compliance
5 with the License. You may obtain a copy of the License at
7 .. https://creativecommons.org/licenses/by/4.0/
9 .. Unless required by applicable law or agreed to in writing, software
10 distributed under the License is distributed on an "AS IS" BASIS,
11 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 See the License for the specific language governing permissions and
13 limitations under the License.
16 Monitoring & Management
17 -----------------------
19 This section addresses data collection and event processing
20 functionality that is directly dependent on the interfaces
21 provided by the VNFs' APIs. These can be in the form of asynchronous
22 interfaces for event, fault notifications, and autonomous data streams.
23 They can also be synchronous interfaces for on-demand requests to
24 retrieve various performance, usage, and other event information.
26 The target direction for VNF interfaces is to employ APIs that are
27 implemented utilizing standardized messaging and modeling protocols
28 over standardized transports. Migrating to a virtualized environment
29 presents a tremendous opportunity to eliminate the need for proprietary
30 interfaces for VNF provider equipment while removing the traditional
31 boundaries between Network Management Systems and Element Management
32 Systems. Additionally, VNFs provide the ability to instrument the
33 networking applications by creating event records to test and monitor
34 end-to-end data flow through the network, similar to what physical or
35 virtual probes provide without the need to insert probes at various
36 points in the network. The VNF providers must be able to provide the
37 aforementioned set of required data directly to the ONAP collection
38 layer using standardized interfaces.
40 Data Model for Event Records
41 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
43 This section describes the data model for the collection of telemetry
44 data from VNFs by Service Providers (SPs) to manage VNF health and
45 runtime lifecycle. This data model is referred to as the VNF Event
46 Streaming (VES) specifications. While this document is focused on
47 specifying some of the records from the ONAP perspective, there may
48 be other external bodies using the same framework to specify additional
49 records. For example, OPNFV has a VES project that is looking to specify
50 records for OpenStack's internal telemetry to manage Application (VNFs),
51 physical and virtual infrastructure (compute, storage, network devices),
52 and virtual infrastructure managers (cloud controllers, SDN controllers).
53 Note that any configurable parameters for these data records (e.g.,
54 frequency, granularity, policy-based configuration) will be managed
55 using the “Configuration” framework described in the prior sections
58 The Data Model consists of:
60 - Common Header Record: This data structure precedes each of the
61 Technology Independent and Technology Specific records sections of
64 - Technology Independent Records: This version of the document
65 specifies the model for Fault, Heartbeat, State Change, Syslog,
66 Threshold Crossing Alerts, and VNF Scaling* (short for
67 measurementForVfScalingFields – actual name used in JSON
68 specification) records. In the future, these may be extended to
69 support other types of technology independent records. Each of
70 these records allows additional fields (name/ value pairs) for
71 extensibility. The VNF provider can use these VNF Provider-specific
72 additional fields to provide additional information that may be
73 relevant to the managing systems.
75 - Technology Specific Records: This version of the document specifies
76 the model for Mobile Flow records, Signaling and Voice Quality records.
77 In the future, these may be extended to support other types of records
78 (e.g. Network Fabric, Security records, etc.). Each of these records
79 allows additional fields (name/value pairs) for extensibility. The VNF
80 providers can use these VNF-specific additional fields to provide
81 additional information that may be relevant to the managing systems.
82 A placeholder for additional technology specific areas of interest to
83 be defined in the future documents has been depicted.
87 Figure 1. Data Model for Event Records
89 Event Records - Data Structure Description
90 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
92 The data structure for event records consists of:
94 - a Common Event Header block;
96 - zero or more technology independent domain blocks; and
98 - e.g., Fault domain, State Change domain, Syslog domain, etc.
100 - zero or more technology specific domain blocks.
102 - e.g., Mobile Flow domain, Signaling domain, Voice Quality domain,
106 ~~~~~~~~~~~~~~~~~~~~~
108 The common header that precedes any of the domain-specific records contains
109 information identifying the type of record to follow, information about
110 the sender and other identifying characteristics related to timestamp,
111 sequence number, etc.
113 Technology Independent Records – Fault Fields
114 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
116 The Fault Record, describing a condition in the Fault domain, contains
117 information about the fault such as the entity under fault, the
118 severity, resulting status, etc.
120 Technology Independent Records – Heartbeat Fields
121 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
123 The Heartbeat Record provides an optional structure for communicating
124 information about heartbeat or watchdog signaling events. It can
125 contain information about service intervals, status information etc.
126 as required by the heartbeat implementation.
128 Note: Heartbeat records would only have the Common Event Header block.
129 An optional heartbeat domain is available if required by the heartbeat
132 Technology Independent Records – State Change Fields
133 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
135 The State Change Record provides a structure for communicating information
136 about data flow through the VNF. It can contain information about state
137 change related to physical device that is reported by VNF. As an example,
138 when cards or port name of the entity that has changed state.
140 Technology Independent Records – Syslog Fields
141 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
143 The Syslog Record provides a structure for communicating any type of
144 information that may be logged by the VNF. It can contain information
145 about system internal events, status, errors, etc.
147 Technology Independent Records – Threshold Crossing Alert Fields
148 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
150 The Threshold Crossing Alert (TCA) Record provides a structure for
151 communicating information about threshold crossing alerts. It can
152 contain alert definitions and types, actions, events, timestamps
153 and physical or logical details.
155 Technology Independent Records - VNF Scaling Fields
156 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
158 The VNF Scaling\* (short for measurementForVfScalingFields –
159 actual name used in JSON specification) Record contains information
160 about VNF and VNF resource structure and its condition to help in
161 the management of the resources for purposes of elastic scaling.
163 Technology Independent Records – otherFields
164 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
166 The otherFields Record defines fields for events belonging to the
167 otherFields domain of the Technology Independent domain enumeration.
168 This record provides a mechanism to convey a complex set of fields
169 (possibly nested or opaque) and is purely intended to address
170 miscellaneous needs such as addressing time-to-market considerations
171 or other proof-of-concept evaluations. Hence, use of this record
172 type is discouraged and should be minimized.
174 Technology Specific Records – Mobile Flow Fields
175 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
177 The Mobile Flow Record provides a structure for communicating
178 information about data flow through the VNF. It can contain
179 information about connectivity and data flows between serving
180 elements for mobile service, such as between LTE reference points, etc.
182 Technology Specific Records – Signaling Fields
183 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
185 The Signaling Record provides a structure for communicating information
186 about signaling messages, parameters and signaling state. It can
187 contain information about data flows for signaling and controlling
188 multimedia communication sessions such as voice and video calls.
190 Technology Specific Records – Voice Quality Fields
191 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
192 The Voice Quality Record provides a structure for communicating information
193 about voice quality statistics including media connection information,
194 such as transmitted octet and packet counts, packet loss, packet delay
195 variation, round-trip delay, QoS parameters and codec selection.
197 Technology Specific Records – Future Domains
198 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
200 The futureDomains Record is a placeholder for additional technology
201 specific areas of interest that will be defined and described
202 in the future documents.
204 Data Structure Specification of the Event Record
205 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
207 For additional information on the event record formats of the data
208 structures mentioned above, please refer to `VES Event
209 Listener <https://onap.readthedocs.io/en/latest/submodules/vnfsdk/model.git/docs/files/VESEventListener.html>`__.
211 Transports and Protocols Supporting Resource Interfaces
212 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
214 Delivery of data from VNFs to ONAP must use the common transport
215 mechanisms and protocols for all VNFs as defined in this document.
216 Transport mechanisms and protocols have been selected to enable both
217 high volume and moderate volume datasets, as well as asynchronous and
218 synchronous communications over secure connections. The specified
219 encoding provides self-documenting content, so data fields can be
220 changed as needs evolve, while minimizing changes to data delivery.
222 The term 'Event Record' is used throughout this document to represent
223 various forms of telemetry or instrumentation made available by the
224 VNF including, faults, status events, various other types of VNF
225 measurements and logs. Headers received by themselves must be used
226 as heartbeat indicators. Common structures and delivery protocols for
227 other types of data will be given in future versions of this document
228 as we get more insight into data volumes and required processing.
230 In the following sections, we provide options for encoding, serialization
231 and data delivery. Agreements between Service Providers and VNF providers
232 shall determine which encoding, serialization and delivery method to use
233 for particular data sets. The selected methods must be agreed to prior to
234 the on-boarding of the VNF into ONAP design studio.
236 VNF Telemetry using VES/JSON Model
237 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
239 The preferred model for data delivery from a VNF to ONAP DCAE is
240 the JSON driven model as depicted in Figure 2.
244 Figure 2. VES/JSON Driven Model
246 VNF providers will provide a YAML artifact to the Service Provider
249 * standard VES/JSON model information elements (key/values) that
251 * any additional non-standard (custom) VES/JSON model information
252 elements (key/values) that the VNF provides
254 Using the semantics and syntax supported by YAML, VNF providers
255 will indicate specific conditions that may arise, and recommend
256 actions that should be taken at specific thresholds, or if specific
257 conditions repeat within a specified time interval.
259 Based on the VNF provider's recommendations, the Service Provider may
260 create additional YAML artifacts (using ONAP design Studio), which
261 finalizes Service Provider engineering rules for the processing of
262 the VNF events. The Service Provider may alter the threshold levels
263 recommended by the VNF providor, and may modify and more clearly
264 specify actions that should be taken when specified conditions arise.
265 The Service Provider-created version of the YAML artifact will be
266 distributed to ONAP applications by the Design framework.
268 VNF Telemetry using YANG Model
269 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
271 In addition to the JSON driven model described above, a YANG
272 driven model can also be supported, as depicted in Figure 3.
276 Figure 3. YANG Driven Model
278 VNF providers will provide to the Service Provider the following
279 YANG model artifacts:
281 * common IETF YANG modules that support the VNF
282 * native (VNF provider-supplied) YANG modules that support the VNF
283 * open (OpenConfig) YANG modules and the following
284 configuration-related information, including:
286 * telemetry configuration and operational state data; such as:
289 * subscription bindings
292 * transport mechanisms
295 * a YAML artifact that provides all necessary mapping relationships
296 between YANG model data types to VES/JSON information elements
297 * YANG helper or decoder functions that automate the conversion between
298 YANG model data types to VES/JSON information elements
299 * OPTIONAL: YANG Telemetry modules in JSON format per RFC 7951
301 Using the semantics and syntax supported by YANG, VNF providers
302 will indicate specific conditions that may arise, and recommend
303 actions that should be taken at specific thresholds, or if specific
304 conditions repeat within a specified time interval.
306 Based on the VNF provider's recommendations, the Service Provider may
307 create additional YAML artifacts (using ONAP design Studio), which
308 finalizes Service Provider engineering rules for the processing of the
309 VNF events. The Service Provider may alter the threshold levels recommended
310 by the VNF provider, and may modify and more clearly specify actions that
311 should be taken when specified conditions arise. The Service
312 Provided-created version of the YAML will be distributed to ONAP
313 applications by the Design framework.
315 Note: While supporting the YANG model described above, we are still
316 leveraging the VES JSON based model in DCAE. The purpose of the
317 diagram above is to illustrate the concept only and not to imply a
318 specific implementation.
320 VNF Telemetry using Google Protocol Buffers
321 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
323 In addition to the data delivery models described above, support for
324 delivery of VNF telemetry using Google Protocol Buffers (GPB) can
325 also be supported, as depicted in Figure 4.
327 VNF providers will provide to the Service Provider the additional
328 following artifacts to support the delivery of VNF telemetry to DCAE
329 via the open-source gRPC mechanism using Google's Protocol Buffers:
331 * the YANG model artifacts described in support of the
332 "VNF Telemetry using YANG Model"
333 * valid definition file(s) for all GPB / KV-GPB encoded messages
334 * valid definition file(s) for all gRPC services
335 * gRPC method parameters and return types specified as Protocol
340 Figure 4. Protocol Buffers Driven Model
342 Note: if Google Protocol Buffers are employed for delivery of VNF
343 telemetry, Key-Value Google Protocol Buffers (KV-GPB) is the
344 preferred serialization method. Details of specifications and
345 versioning corresponding to a release can be found at:
346 `VES Event Listener <https://onap.readthedocs.io/en/latest/submodules/vnfsdk/model.git/docs/files/VESEventListener.html>`__.
348 Note: While supporting the VNF telemetry delivery approach described above,
349 we are still leveraging the VES JSON based model in DCAE. The purpose of
350 the diagram above is to illustrate the concept only and not to imply a
351 specific implementation.
353 Monitoring & Management Requirements
354 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
356 VNF telemetry via standardized interface
357 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
365 The xNF **MUST** provide all telemetry (e.g., fault event
366 records, syslog records, performance records etc.) to ONAP using the
367 model, format and mechanisms described in this section.
369 Encoding and Serialization
370 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
372 Content delivered from VNFs to ONAP is to be encoded and serialized using JSON:
383 The xNF **MUST** encode and serialize content delivered to
384 ONAP using JSON (RFC 7159) plain text format. High-volume data
385 is to be encoded and serialized using `Avro <http://avro.apache.org/>`_,
386 where the Avro [#7.4.1]_ data format are described using JSON.
390 - JSON plain text format is preferred for moderate volume data sets
391 (option 1), as JSON has the advantage of having well-understood simple
392 processing and being human-readable without additional decoding. Examples
393 of moderate volume data sets include the fault alarms and performance
394 alerts, heartbeat messages, measurements used for xNF scaling and syslogs.
395 - Binary format using Avro is preferred for high volume data sets
396 (option 2) such as mobility flow measurements and other high-volume
397 streaming events (such as mobility signaling events or SIP signaling)
398 or bulk data, as this will significantly reduce the volume of data
399 to be transmitted. As of the date of this document, all events are
400 reported using plain text JSON and REST.
401 - Avro content is self-documented, using a JSON schema. The JSON schema is
402 delivered along with the data content
403 (http://avro.apache.org/docs/current/ ). This means the presence and
404 position of data fields can be recognized automatically, as well as the
405 data format, definition and other attributes. Avro content can be
406 serialized as JSON tagged text or as binary. In binary format, the
407 JSON schema is included as a separate data block, so the content is
408 not tagged, further compressing the volume. For streaming data, Avro
409 will read the schema when the stream is established and apply the
410 schema to the received content.
412 In addition to the preferred method (JSON), content can be delivered
413 from xNFs to ONAP can be encoded and serialized using Google Protocol
419 Telemetry data delivered using Google Protocol Buffers v3 (proto3)
420 can be serialized in one of the following methods:
422 * Key-value Google Protocol Buffers (KV-GPB) is also known as
425 * keys are strings that correspond to the path of the system
426 resources for the VNF being monitored.
427 * values correspond to integers or strings that identify the
428 operational state of the VNF resource, such a statistics counters
429 and the state of a VNF resource.
431 * VNF providers must supply valid KV-GPB definition file(s) to allow
432 for the decoding of all KV-GPB encoded telemetry messages.
434 * Native Google Protocol Buffers (GPB) is also known as compact GPB:
436 * keys are represented as integers pointing to the system resources for
437 the VNF being monitored.
438 * values correspond to integers or strings that identify the operational
439 state of the VNF resource, such a statistics counters and the state
442 * Google Protocol Buffers (GPB) requires metadata in the form of .proto
443 files. VNF providers must supply the necessary GPB .proto files such that
444 GPB telemetry messages can be encoded and decoded.
446 * In the future, we may consider support for other types of
447 encoding & serialization methods based on industry demand.
451 ~~~~~~~~~~~~~~~~~~~~~
458 The xNF **MUST** vary the frequency that asynchronous data
459 is delivered based on the content and how data may be aggregated or
464 - For example, alarms and alerts are expected to be delivered as
465 soon as they appear. In contrast, other content, such as performance
466 measurements, KPIs or reported network signaling may have various
467 ways of packaging and delivering content. Some content should be
468 streamed immediately; or content may be monitored over a time
469 interval, then packaged as collection of records and delivered
470 as block; or data may be collected until a package of a certain
471 size has been collected; or content may be summarized statistically
472 over a time interval, or computed as a KPI, with the summary or KPI
474 - We expect the reporting frequency to be configurable depending on
475 the virtual network functions needs for management. For example,
476 Service Provider may choose to vary the frequency of collection
477 between normal and trouble-shooting scenarios.
478 - Decisions about the frequency of data reporting will affect
479 the size of delivered data sets, recommended delivery method,
480 and how the data will be interpreted by ONAP. These considerations
481 should not affect deserialization and decoding of the data, which
482 will be guided by the accompanying JSON schema or GPB definition
485 Addressing and Delivery Protocol
486 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
488 ONAP destinations can be addressed by URLs for RESTful data PUT. Future
489 data sets may also be addressed by host name and port number for TCP
490 streaming, or by host name and landing zone directory for SFTP transfer
498 The xNF **SHOULD** use REST using HTTPS delivery of plain
499 text JSON for moderate sized asynchronous data sets, and for high
500 volume data sets when feasible.
507 The xNF **MUST** have the capability of maintaining a primary
508 and backup DNS name (URL) for connecting to ONAP collectors, with the
509 ability to switch between addresses based on conditions defined by policy
510 such as time-outs, and buffering to store messages until they can be
511 delivered. At its discretion, the service provider may choose to populate
512 only one collector address for a xNF. In this case, the network will
513 promptly resolve connectivity problems caused by a collector or network
514 failure transparently to the xNF.
521 The xNF **MUST** be configured with initial address(es) to use
522 at deployment time. Subsequently, address(es) may be changed through
523 ONAP-defined policies delivered from ONAP to the xNF using PUTs to a
524 RESTful API, in the same manner that other controls over data reporting
525 will be controlled by policy.
532 The xNF **MAY** use another option which is expected to include REST
533 delivery of binary encoded data sets.
540 The xNF **MAY** use another option which is expected to include TCP
541 for high volume streaming asynchronous data sets and for other high volume
542 data sets. TCP delivery can be used for either JSON or binary encoded data
550 The xNF **MAY** use another option which is expected to include SFTP
551 for asynchronous bulk files, such as bulk files that contain large volumes
552 of data collected over a long time interval or data collected across many
553 xNFs. (Preferred is to reorganize the data into more frequent or more focused
554 data sets, and deliver these by REST or TCP as appropriate.)
561 The xNF **MAY** use another option which is expected to include REST
562 for synchronous data, using RESTCONF (e.g., for xNF state polling).
569 The xNF **MUST**, by ONAP Policy, provide the ONAP addresses
570 as data destinations for each xNF, and may be changed by Policy while
571 the xNF is in operation. We expect the xNF to be capable of redirecting
572 traffic to changed destinations with no loss of data, for example from
573 one REST URL to another, or from one TCP host and port to another.
575 Asynchronous and Synchronous Data Delivery
576 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
583 The xNF **MUST** deliver asynchronous data as data becomes
584 available, or according to the configured frequency.
591 The xNF **MUST** must encode, address and deliver the data
592 as described in the previous paragraphs.
599 The xNF **MUST** respond to data requests from ONAP as soon
600 as those requests are received, as a synchronous response.
607 The xNF **MUST** use the RESTCONF/NETCONF framework used by
608 the ONAP configuration subsystem for synchronous communication.
615 The xNF **MUST** use the YANG configuration models and RESTCONF
616 [RFC8040] (https://tools.ietf.org/html/rfc8040).
623 The xNF **MUST** respond with content encoded in JSON, as
624 described in the RESTCONF specification. This way the encoding of a
625 synchronous communication will be consistent with Avro.
632 The xNF **MUST** respond to an ONAP request to deliver the
633 current data for any of the record types defined in
634 `Event Records - Data Structure Description`_ by returning the requested
635 record, populated with the current field values. (Currently the defined
636 record types include fault fields, mobile flow fields, measurements for
637 xNF scaling fields, and syslog fields. Other record types will be added
638 in the future as they become standardized and are made available.)
645 The xNF **MUST** respond to an ONAP request to deliver granular
646 data on device or subsystem status or performance, referencing the YANG
647 configuration model for the xNF by returning the requested data elements.
654 The xNF **SHOULD** use `Modeling JSON text with YANG
655 <https://tools.ietf.org/html/rfc7951>`_, If YANG models need to be
656 translated to and from JSON{RFC7951]. YANG configuration and content can
657 be represented via JSON, consistent with Avro, as described in "Encoding
658 and Serialization" section.
668 The xNF **MUST** support secure connections and transports such as
669 Transport Layer Security (TLS) protocol
670 [`RFC5246 <https://tools.ietf.org/html/rfc5246>`_] and should adhere to
671 the best current practices outlined in
672 `RFC7525 <https://tools.ietf.org/html/rfc7525>`_.
679 The xNF **MUST** control access to ONAP and to xNFs, and creation
680 of connections, through secure credentials, log-on and exchange mechanisms.
687 The xNF **MUST** carry data in motion only over secure connections.
694 The xNF **MUST** encrypt any content containing Sensitive Personal
695 Information (SPI) or certain proprietary data, in addition to applying the
696 regular procedures for securing access and delivery.
698 Bulk Performance Measurement
705 :introduced: casablanca
706 :impacts: dcae, dmaap
708 The xNF **SHOULD** support FileReady VES event for event-driven bulk transfer
716 :introduced: casablanca
717 :impacts: dcae, dmaap
719 The xNF **SHOULD** support File transferring protocol, such as FTPES or SFTP,
720 when supporting the event-driven bulk transfer of monitoring data.
726 :introduced: casablanca
727 :impacts: dcae, dmaap
729 The xNF **SHOULD** support the data schema defined in 3GPP TS 32.435, when
730 supporting the event-driven bulk transfer of monitoring data.
733 This option is not currently supported in ONAP and it is currently
736 .. |image0| image:: ../Data_Model_For_Event_Records.png
740 .. |image1| image:: ../VES_JSON_Driven_Model.png
744 .. |image2| image:: ../YANG_Driven_Model.png
748 .. |image3| image:: ../Protocol_Buffers_Driven_Model.png