-=======\r
-Filter\r
-======= \r
-\r
-A filter examine an event and decides if it matches or doesn't. Filters\r
-are mainly used in rules to decide if the processing entries should be\r
-executed on the given event. They're also used for settings, and systems\r
-like the Graph Correlator re-use Highland Park's filter mechanism to\r
-specify which alarms fit in a correlation. Some publishers may produce\r
-topics with a lot of volume and a subscriber may want only a portion of\r
-those messages. The subscriber can certainly filter messages after\r
-receiving them, but it may be substantially more efficient to ask the\r
-API server to filter the messages before sending them to the\r
-subscriber.The standard library includes a number of simple filters. The\r
-Equals filter, for example, compares a field's value with some other\r
-value and returns true if the values match. The standard library also\r
-includes filter classes called And, Or, and Not, so you can compose more\r
-complex filters. For example, written in the standard JSON config\r
-format: \r
-\r
-+-----------------------------------------+\r
-| "filter":{ |\r
-| "class":"And", |\r
-| "filters": |\r
-| [ |\r
-| { "class":"Equals", "foo":"abc" }, |\r
-| { "class":"Assigned", "field":"bar" } |\r
-| ] |\r
-| } |\r
-+-----------------------------------------+\r
-\r
-- This filter matches events in which the field "foo" has value "abc"\r
- and the field "bar" is assigned to some value (as opposed to not being\r
- present on the event). Filters are used by the consumers to filter out\r
- data and consume only specific set of data which matches the\r
- conditions mentioned in filter. Filters can be passed as a query\r
- parameter by the consumer in consume request as mentioned below:\r
- **http://localhost:8080/DMaaP/dmaaprest/events/DMaaP/consumergroup/mHOeNFY4XiWx4CBa?filter=\\{"class":"Equals",\r
- "field":"email", "value":"test@abc.com" }** \r
-- Filters can be applied only on data in JSON format i.e. if applied,\r
- filters will automatically ignore any non-json data. While consuming,\r
- request CONTENT\_TYPE is not relevant to filter.\r
-\r
-All the supported filter can be found below.\r
-\r
-Types of Filters \r
-----------------\r
-\r
-- DMaaP Message Router supports all the filters which were supported by\r
- DMaaP Message Router and are mentioned below:- \r
-- **All Alarms:**\r
- - Match all alarms. \r
-- **And:**\r
- - Create a set of filters. This filter matches when all of them matches.\r
-\r
-+-------------+--------------------+------------+---------------------+\r
-| **Field** | **Description** | **Type** | **Notes** |\r
-+=============+====================+============+=====================+\r
-| filters | Combined Filters | LIST | A list of filters |\r
-+-------------+--------------------+------------+---------------------+\r
-| | | | |\r
-+-------------+--------------------+------------+---------------------+\r
-\r
-- **Assigned:**\r
- - Choose a field from the event to check for assignment. This filter\r
- matches when the field is assigned.\r
-\r
-+-------------------+--------------------------------------------------------+------------+-----------------+\r
-| **Field** | **Description** | **Type** | **Notes** |\r
-+===================+========================================================+============+=================+\r
-| field | The field to check for on the event. | STRING | A field name |\r
-+-------------------+--------------------------------------------------------+------------+-----------------+\r
-| emptyIsAssigned | If true, an empty value is considered an assignment. | BOOLEAN | True or False |\r
-+-------------------+--------------------------------------------------------+------------+-----------------+\r
-\r
-- **Contains:**\r
- - Check if a search string contains another string.\r
-\r
-+-------------+---------------------------------------------------+------------+--------------+\r
-| **Field** | **Description** | **Type** | **Notes** |\r
-+=============+===================================================+============+==============+\r
-| String | The value to search. Supports ${} notation. | STRING | Any string |\r
-+-------------+---------------------------------------------------+------------+--------------+\r
-| Value | The value to search for. Supports ${} notation. | STRING | Any string |\r
-+-------------+---------------------------------------------------+------------+--------------+\r
-| | | | |\r
-+-------------+---------------------------------------------------+------------+--------------+\r
-\r
-- **EndsWith**:\r
- - Check if a search string ends with another string.\r
-\r
-+-------------+---------------------------------------------------+------------+--------------+\r
-| **Field** | **Description** | **Type** | **Notes** |\r
-+=============+===================================================+============+==============+\r
-| string | The value to search. Supports ${} notation. | STRING | Any string |\r
-+-------------+---------------------------------------------------+------------+--------------+\r
-| value | The value to search for. Supports ${} notation. | STRING | Any string |\r
-+-------------+---------------------------------------------------+------------+--------------+\r
-\r
-- **Equals:**\r
-\r
- - Choose a field from the event and a value to check for equality.\r
-\r
-+-------------+----------------------------------------------+------------+--------------+\r
-| **Field** | **Description** | **Type** | **Notes** |\r
-+=============+==============================================+============+==============+\r
-| field | The field to check. Supports ${} notation. | STRING | Any string |\r
-+-------------+----------------------------------------------+------------+--------------+\r
-| value | The value to match. Supports ${} notation. | STRING | Any string |\r
-+-------------+----------------------------------------------+------------+--------------+\r
-\r
-- **FlatironObjectExists**\r
-\r
- - Matches when the given object exists in the given Flatiron instance.\r
-\r
-+-------------+---------------------------------------------+------------+--------------+\r
-| **Field** | **Description** | **Type** | **Notes** |\r
-+=============+=============================================+============+==============+\r
-| oid | The OID of the object to look for. | STRING | Any string |\r
-+-------------+---------------------------------------------+------------+--------------+\r
-| flatiron | The name of the Flatiron client instance. | STRING | Any string |\r
-+-------------+---------------------------------------------+------------+--------------+\r
-\r
-- **IsAging**\r
- - Choose a field to test. This filter matches if the expression is\r
- numeric.\r
-\r
-+-------------+---------------------------------------------+------------+--------------+\r
-| **Field** | **Description** | **Type** | **Notes** |\r
-+=============+=============================================+============+==============+\r
-| field | The field to test. Supports ${} notation. | STRING | Any string |\r
-+-------------+---------------------------------------------+------------+--------------+\r
-\r
-- **IsNumeric**\r
- - Choose a field to test. This filter matches if the expression is\r
- numeric.\r
-\r
-+-------------+---------------------------------------------+------------+--------------+\r
-| **Field** | **Description** | **Type** | **Notes** |\r
-+=============+=============================================+============+==============+\r
-| field | The field to test. Supports ${} notation. | STRING | Any string |\r
-+-------------+---------------------------------------------+------------+--------------+\r
-\r
-- **MathCondition**\r
- - Choose a field from the event and a value for logical math conditions.\r
-\r
-+-------------+-------------------------------------------------+------------+-----------------------------------+\r
-| **Field** | **Description** | **Type** | **Notes** |\r
-+=============+=================================================+============+===================================+\r
-| Field | The field to check. Supports ${} notation. | STRING | Any string |\r
-+-------------+-------------------------------------------------+------------+-----------------------------------+\r
-| Value | The value to consider. Supports ${} notation. | STRING | Any string |\r
-+-------------+-------------------------------------------------+------------+-----------------------------------+\r
-| operator | The operation. | STRING | One of { "<=", ">=", ">", "<" } |\r
-+-------------+-------------------------------------------------+------------+-----------------------------------+\r
-| | | | |\r
-+-------------+-------------------------------------------------+------------+-----------------------------------+\r
-\r
-- **NoAlarms**\r
- - Don't match any alarms. \r
-- **Not**\r
- - Negate the configured filter.\r
-\r
-+-------------+-------------------------+------------+-------------+\r
-| **Field** | **Description** | **Type** | **Notes** |\r
-+=============+=========================+============+=============+\r
-| filter | The filter to negate. | FILTER | A filter |\r
-+-------------+-------------------------+------------+-------------+\r
-\r
-- **NotEqual**\r
- - Choose a field from the event and a value to check for inequality.\r
-\r
-+-------------+----------------------------------------------+------------+--------------+\r
-| **Field** | **Description** | **Type** | **Notes** |\r
-+=============+==============================================+============+==============+\r
-| field | The field to check. Supports ${} notation. | STRING | Any string |\r
-+-------------+----------------------------------------------+------------+--------------+\r
-| value | The value to match. Supports ${} notation. | STRING | Any string |\r
-+-------------+----------------------------------------------+------------+--------------+\r
-\r
-- **NotOneOf**\r
- - Match when the specified field does not have a value from the given\r
- list.\r
-\r
-+-------------+---------------------------------------------+------------+---------------------+\r
-| **Field** | **Description** | **Type** | **Notes** |\r
-+=============+=============================================+============+=====================+\r
-| field | The field to test. Supports ${} notation. | STRING | Any string |\r
-+-------------+---------------------------------------------+------------+---------------------+\r
-| values | The matching values. | LIST | A list of strings |\r
-+-------------+---------------------------------------------+------------+---------------------+\r
-\r
-- **OneOf**\r
- - Match when the specified field has a value from the given list.\r
-\r
-+-------------+---------------------------------------------+------------+---------------------+\r
-| **Field** | **Description** | **Type** | **Notes** |\r
-+=============+=============================================+============+=====================+\r
-| field | The field to test. Supports ${} notation. | STRING | Any string |\r
-+-------------+---------------------------------------------+------------+---------------------+\r
-| values | The matching values. | LIST | A list of strings |\r
-+-------------+---------------------------------------------+------------+---------------------+\r
-\r
-- **Or**\r
- - Create a set of filters. This filter matches when any one of them\r
- matches.\r
-\r
-+-------------+--------------------+------------+---------------------+\r
-| **Field** | **Description** | **Type** | **Notes** |\r
-+=============+====================+============+=====================+\r
-| filters | Combined Filters | LIST | A list of filters |\r
-+-------------+--------------------+------------+---------------------+\r
-\r
-- **RegEx**\r
- - Choose a field from the event to match against the regular expression\r
- you provide.\r
-\r
-+-------------+---------------------------------------------------------+------------+--------------+\r
-| **Field** | **Description** | **Type** | **Notes** |\r
-+=============+=========================================================+============+==============+\r
-| field | The text to check for a match. Supports ${} notation. | STRING | Any string |\r
-+-------------+---------------------------------------------------------+------------+--------------+\r
-| value | The regular expression (pattern) to match. | STRING | Any string |\r
-+-------------+---------------------------------------------------------+------------+--------------+\r
-\r
-- **StartsWith**\r
- - Check if a search string starts with another string.\r
-\r
-+-------------+---------------------------------------------------+------------+--------------+\r
-| **Field** | **Description** | **Type** | **Notes** |\r
-+=============+===================================================+============+==============+\r
-| string | The value to search. Supports ${} notation. | STRING | Any string |\r
-+-------------+---------------------------------------------------+------------+--------------+\r
-| Value | The value to search for. Supports ${} notation. | STRING | Any string |\r
-+-------------+---------------------------------------------------+------------+--------------+\r
-\r
-- **Unassigned**\r
- - Choose a field from the event to check for assignment. This filter\r
- matches when the field is not assigned.\r
-\r
-+-------------------+--------------------------------------------------------+------------+-----------------+\r
-| **Field** | **Description** | **Type** | **Notes** |\r
-+===================+========================================================+============+=================+\r
-| field | The field to check for on the event. | STRING | A field name |\r
-+-------------------+--------------------------------------------------------+------------+-----------------+\r
-| emptyIsAssigned | If true, an empty value is considered an assignment. | BOOLEAN | True or False |\r
-+-------------------+--------------------------------------------------------+------------+-----------------+\r
-\r
-- **WithinSecondsFrom**\r
- - This filter matches when the specified epoch time value is within the\r
- given number of seconds from the baseline time value. Both time values\r
- are assumed to be in seconds. If a value is in milliseconds, set\r
- baselineTimeInMillis and/or eventTimeInMillis to true.\r
-\r
-+------------------------+------------------------------------------------------------+------------+-----------------+\r
-| **Field** | **Description** | **Type** | **Notes** |\r
-+========================+============================================================+============+=================+\r
-| field | The time value to test. Supports ${} | STRING | A field name |\r
-+------------------------+------------------------------------------------------------+------------+-----------------+\r
-| eventTimeInMillis | Whether to convert the event value from milliseconds. | BOOLEAN | True or False |\r
-+------------------------+------------------------------------------------------------+------------+-----------------+\r
-| seconds | The number of seconds. | NUMBER | A number |\r
-+------------------------+------------------------------------------------------------+------------+-----------------+\r
-| baselineTimeInMillis | Whether to convert the baseline value from milliseconds. | BOOLEAN | True or False |\r
-+------------------------+------------------------------------------------------------+------------+-----------------+\r
-| baseline | The baseline time value. Supports ${}. | STRING | Any string |\r
-+------------------------+------------------------------------------------------------+------------+-----------------+\r
-\r
-- **WithinTimeFromNow**\r
- - This filter matches when the named field has an epoch time value\r
- within the given number of seconds from the current time. The event's\r
- time value is assumed to be in seconds. If it's in milliseconds, set\r
- eventTimeInMillis to true.\r
-\r
-+---------------------+---------------------------------------------------------+------------+-----------------+\r
-| **Field** | **Description** | **Type** | **Notes** |\r
-+=====================+=========================================================+============+=================+\r
-| field | The field to check on the event. | STRING | A field name |\r
-+---------------------+---------------------------------------------------------+------------+-----------------+\r
-| eventTimeInMillis | Whether to convert the event value from milliseconds. | BOOLEAN | True or False |\r
-+---------------------+---------------------------------------------------------+------------+-----------------+\r
-| seconds | The number of seconds. | NUMBER | A number |\r
-+---------------------+---------------------------------------------------------+------------+-----------------+\r
-\r
-- **Limit:** \r
-\r
- - Limit is the integer value and DMaaP Message Router will consumes\r
- only that set of message which are specified in limit.\r
- \r
- .. code:: bash\r
- \r
- Suppose if we set limit=2, then only 2 sets of data will be consumed. \r
- *Get \ **<<topicName>>/group/2?limit=4*** \r
- Let us suppose if \r
- **No of data available** = 4\r
- **Set limit** = 6\r
- i.e. limit>no of data\r
- In this scenario all 4 sets of data will be consumed. \r
-\r
-- If limit is not passed with the url then by default limit is set to\r
- 4096.\r
- \r
- .. code:: bash\r
-\r
- i.e. 4096 sets of data will be consumed. \r
- **Timeout and Long Poll:** \r
-\r
-- Timeout is the integer value which will be treated by DMaaP Message\r
- Router as time in millisecond.\r
-\r
- \r
-\r
-- Get\r
-\r
-+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+\r
-| `***http://localhost/DMaaP/dmaaprest/events/<<topicName>>/group/2?timeout=20000*** <http://localhost/DMaaP/dmaaprest/events/%3c%3ctopicName%3e%3e/group/2?timeout=20000>`__ |\r
-+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+\r
-\r
-- If there is no data available to be consumed, then DMaaP Message\r
- Router will poll for the particular period of time specified in\r
- timeout this mechanism is known as Long Poll.\r
-\r
-- If timeout is not passed with url then DMaaP Message Router will set\r
- the value of timeout =10000\r
-\r
-- i.e. if no set of data are available then DMaaP Message Router will\r
- poll for 10000 ms.\r
-\r
-***Meta:***\r
-\r
-- Meta is a Boolean value.\r
-\r
-- DMaaP Message Router reads the value of meta from\r
- MRConfiguration.properties file at the time of startup.\r
-\r
-- If the value of meta is not null and if value of meta is one of these\r
- values true, yes, on, 1, y, checked then DMaaP Message Router will\r
- take meta flag as true, else it will be false.\r
-\r
-- If meta is set to true then consumer will get the value of message\r
- offset along with message.\r
-\r
- |image1|\r
-\r
- .. |image1| image:: images/image1.png\r
- \r
-- **Pretty**:\r
-\r
-- Pretty is a Boolean value.\r
-\r
-- DMaaP Message Router reads the value of pretty from\r
- MRConfiguration.properties file at the time of startup.\r
-\r
-- If the value of pretty is not null and if value of pretty is one of\r
- these values true, yes, on, 1, y, checked then DMaaP Message Router\r
- will take pretty flag as true, else it will be false.\r
-\r
-- If pretty is set to true then different sets of messages will be\r
- printed in next line separated by comma (,).\r
-\r
- |image2|\r
- \r
- .. |image2| image:: images/image2.png\r
- \r
-- **Filter**\r
-\r
- - A filter examine an event and decides if it matches or doesn't. \r
- - Filters are mainly used in rules to decide if the processing entries\r
- should be executed on the given event. They're also used for settings,\r
- and systems like the Graph Correlator re-use Highland Park's filter\r
- mechanism to specify which alarms fit in a correlation. \r
- - The standard library includes a number of simple filters. The Equals\r
- filter, for example, compares a field's value with some other value\r
- and returns true if the values match. \r
- - The standard library also includes filter classes called And, Or, and\r
- Not, so you can compose more complex filters. For example, written in\r
- the standard JSON config format: \r
-\r
- +-----------------------------------------+\r
- | "filter":{ |\r
- | "class":"And", |\r
- | "filters": |\r
- | [ |\r
- | { "class":"Equals", "foo":"abc" }, |\r
- | { "class":"Assigned", "field":"bar" } |\r
- | ] |\r
- | } |\r
- +-----------------------------------------+\r
-\r
- - This filter matches events in which the field "foo" has value "abc"\r
- and the field "bar" is assigned to some value (as opposed to not being\r
- present on the event).\r
- - Filters are used by the consumers to filter out data and consume only\r
- specific set of data which matches the conditions mentioned in filter.\r
- - Filters can be passed as a query parameter by the consumer in consume\r
- request as mentioned below:\r
- - **http://localhost:8080/DMaaP/dmaaprest/events/DMaaP/consumergroup/mHOeNFY4XiWx4CBa?filter=\\{"class":"Equals",\r
- "field":"email", "value":"`test@abc.com <mailto:test@att.com>`__" }** \r
- - Filters can be applied only on data in JSON format i.e. if applied,\r
- filters will automatically ignore any non-json data. \r
- - While consuming, request CONTENT\_TYPE is not relevant to filter.\r
-\r
- The MR API allows a subscriber pass a Highland Park filter as part of\r
- the GET request. This will filter the stream of messages sent back to\r
- the subscriber, but for this to work, there are some requirements: \r
-\r
- - The message payload must be JSON\r
-\r
- - Only a filter built from Highland Park's Standard Library may be\r
- used. (The Cambria API server doesn't have access to plugged in\r
- filters.)\r
-\r
- - The filter must be encoded properly in the URL path.\r
-\r
- Server-side filtering can also be setup in the Java client as\r
-illustrated below\r
-\r
-**Filtering Consumer**\r
-\r
-You can also provide a Highland Park filter to your consumer instance,\r
-and this filter is passed on to the server in the GET request. One way\r
-to create the filter is programmatically. In your code, instantiate a\r
-filter from the Highland Park Standard Library Then create a String\r
-representation of the filter using the FilterIo.write utility. This\r
-String can then be passed to the Cambria client instance for use on the\r
-server.\r
-\r
-Remember, only Highland Park standard library filter components can be\r
-used -- no plug-ins are available in the Cambria server context.\r
- \r
- .. code:: bash\r
- \r
- package org.onap.sa.highlandPark.integration;\r
-\r
- import java.io.IOException;\r
-\r
- import java.util.UUID;\r
-\r
- import org.onap.nsa.cambria.client.CambriaClientFactory;\r
-\r
- import org.onap.nsa.cambria.client.CambriaConsumer;\r
-\r
- import org.onap.sa.highlandPark.processor.HpEvent;\r
-\r
- import org.onap.sa.highlandPark.stdlib.filters.FilterIo;\r
-\r
- import org.onap.sa.highlandPark.stdlib.filters.OneOf;\r
-\r
- public class ExampleFilteringConsumer\r
-\r
- {\r
-\r
- public static void main ( String[] args ) throws IOException,\r
- InterruptedException\r
-\r
- {\r
-\r
- // Cambria clients take a set of 1 or more servers to use in round-robin\r
- fashion.\r
-\r
- // If a server becomes unreachable, another in the group is used.\r
-\r
- final String\r
- serverGroup="ueb01hydc.it.att.com,ueb02hydc.it.att.com,ueb03hydc.it.att.com";\r
-\r
- // choose a topic\r
-\r
- final String topic = "TEST-TOPIC";\r
-\r
- // Cambria clients can run in a cooperative group to handle high-volume\r
- topics.\r
-\r
- // Here, we create a random group name, which means this client is not\r
- re-startable.\r
-\r
- final String consumerGroup = UUID.randomUUID ().toString ();\r
-\r
- final String consumerId = "0";\r
-\r
- // Cambria clients can sit in a tight loop on the client side, using a\r
- long-poll\r
-\r
- // to wait for messages, and a limit to tell the server the most to send\r
- at a time.\r
-\r
- final int longPollMs = 30\*1000;\r
-\r
- final int limit = -1;\r
-\r
- // The Cambria server can filter the returned message stream using\r
- filters from the\r
-\r
- // Highland Park system. Here, we create a simple filter to test for the\r
- AlarmID\r
-\r
- // value being one of the Mobility power alarms.\r
-\r
- final OneOf oneOf = new OneOf ( "AlarmId", kPowerAlarms );\r
-\r
- // create the consumer\r
-\r
- final CambriaConsumer cc = CambriaClientFactory.createConsumer (\r
- serverGroup, topic,\r
-\r
- consumerGroup, consumerId, longPollMs, limit, FilterIo.write ( oneOf )\r
- );\r
-\r
- // now loop reading messages. Note that cc.fetch() will wait in its HTTP\r
- receive\r
-\r
- // method for up to 30 seconds (longPollMs) when nothing's available at\r
- the server.\r
-\r
- long count = 0;\r
-\r
- while ( true )\r
-\r
- {\r
-\r
- for ( String msg : cc.fetch () )\r
-\r
- {\r
-\r
- System.out.println ( "" + (++count) + ": " + msg );\r
-\r
- }\r
-\r
- }\r
-\r
- }\r
-\r
- private static final String[] kPowerAlarms =\r
-\r
- {\r
-\r
- "HUB COMMERCIAL POWER FAIL\_FWD",\r
-\r
- "HUB COMMERCIAL POWER FAIL",\r
-\r
- "RBS COMMERCIAL POWER FAIL - Fixed\_FWD",\r
-\r
- "RBS COMMERCIAL POWER FAIL\_FWD",\r
-\r
- "RBS COMMERCIAL POWER FAIL - No Generator\_FWD",\r
-\r
- "RBS COMMERCIAL POWER FAIL - Portable\_FWD",\r
-\r
- "RBS COMMERCIAL POWER FAIL - Shared\_FWD",\r
-\r
- "RBS COMMERCIAL POWER FAIL - Yes\_FWD",\r
-\r
- "RBS COMMERCIAL POWER FAIL - YES\_FWD",\r
-\r
- "RBS COMMERCIAL POWER FAIL - Fixed",\r
-\r
- "RBS COMMERCIAL POWER FAIL - No Generator",\r
-\r
- "RBS COMMERCIAL POWER FAIL - Portable",\r
-\r
- "RBS COMMERCIAL POWER FAIL - Shared",\r
-\r
- "RBS COMMERCIAL POWER FAIL - YES",\r
-\r
- "RBS COMMERCIAL POWER FAIL - Yes",\r
-\r
- "RBS COMMERCIAL POWER FAIL",\r
-\r
- "HUB COMMERCIAL POWER FAIL - Fixed",\r
-\r
- "HUB COMMERCIAL POWER FAIL - No Generator",\r
-\r
- "HUB COMMERCIAL POWER FAIL - Portable",\r
-\r
- "HUB COMMERCIAL POWER FAIL - Shared",\r
-\r
- "HUB COMMERCIAL POWER FAIL - Fixed\_FWD",\r
-\r
- "HUB COMMERCIAL POWER FAIL - No Generator\_FWD",\r
-\r
- "HUB COMMERCIAL POWER FAIL - Portable\_FWD",\r
-\r
- "HUB COMMERCIAL POWER FAIL - Shared\_FWD",\r
-\r
- };\r
-\r
- }\r
-\r
- \r
-\r
-**Filter Builder**\r
-\r
- MR server-side filtering allows a consumer to filter the stream of\r
- messages returned from the GET call. The following link provide details\r
- of building some of the filter to illustrate Filter Builder. It is not\r
- meant to cover and provide examples of every filter\r
-\r
-\r
-\r
-\r
+=======
+Filter
+=======
+
+A filter examine an event and decides if it matches or doesn't. Filters
+are mainly used in rules to decide if the processing entries should be
+executed on the given event. They're also used for settings, and systems
+like the Graph Correlator re-use Highland Park's filter mechanism to
+specify which alarms fit in a correlation. Some publishers may produce
+topics with a lot of volume and a subscriber may want only a portion of
+those messages. The subscriber can certainly filter messages after
+receiving them, but it may be substantially more efficient to ask the
+API server to filter the messages before sending them to the
+subscriber.The standard library includes a number of simple filters. The
+Equals filter, for example, compares a field's value with some other
+value and returns true if the values match. The standard library also
+includes filter classes called And, Or, and Not, so you can compose more
+complex filters. For example, written in the standard JSON config
+format:
+
++-----------------------------------------+
+| "filter":{ |
+| "class":"And", |
+| "filters": |
+| [ |
+| { "class":"Equals", "foo":"abc" }, |
+| { "class":"Assigned", "field":"bar" } |
+| ] |
+| } |
++-----------------------------------------+
+
+- This filter matches events in which the field "foo" has value "abc"
+ and the field "bar" is assigned to some value (as opposed to not being
+ present on the event). Filters are used by the consumers to filter out
+ data and consume only specific set of data which matches the
+ conditions mentioned in filter. Filters can be passed as a query
+ parameter by the consumer in consume request as mentioned below:
+ **http://localhost:8080/DMaaP/dmaaprest/events/DMaaP/consumergroup/mHOeNFY4XiWx4CBa?filter=\\{"class":"Equals",
+ "field":"email", "value":"test@abc.com" }**
+- Filters can be applied only on data in JSON format i.e. if applied,
+ filters will automatically ignore any non-json data. While consuming,
+ request CONTENT\_TYPE is not relevant to filter.
+
+All the supported filter can be found below.
+
+Types of Filters
+----------------
+
+- DMaaP Message Router supports all the filters which were supported by
+ DMaaP Message Router and are mentioned below:-
+- **All Alarms:**
+ - Match all alarms.
+- **And:**
+ - Create a set of filters. This filter matches when all of them matches.
+
++-------------+--------------------+------------+---------------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+====================+============+=====================+
+| filters | Combined Filters | LIST | A list of filters |
++-------------+--------------------+------------+---------------------+
+| | | | |
++-------------+--------------------+------------+---------------------+
+
+- **Assigned:**
+ - Choose a field from the event to check for assignment. This filter
+ matches when the field is assigned.
+
++-------------------+--------------------------------------------------------+------------+-----------------+
+| **Field** | **Description** | **Type** | **Notes** |
++===================+========================================================+============+=================+
+| field | The field to check for on the event. | STRING | A field name |
++-------------------+--------------------------------------------------------+------------+-----------------+
+| emptyIsAssigned | If true, an empty value is considered an assignment. | BOOLEAN | True or False |
++-------------------+--------------------------------------------------------+------------+-----------------+
+
+- **Contains:**
+ - Check if a search string contains another string.
+
++-------------+---------------------------------------------------+------------+--------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+===================================================+============+==============+
+| String | The value to search. Supports ${} notation. | STRING | Any string |
++-------------+---------------------------------------------------+------------+--------------+
+| Value | The value to search for. Supports ${} notation. | STRING | Any string |
++-------------+---------------------------------------------------+------------+--------------+
+| | | | |
++-------------+---------------------------------------------------+------------+--------------+
+
+- **EndsWith**:
+ - Check if a search string ends with another string.
+
++-------------+---------------------------------------------------+------------+--------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+===================================================+============+==============+
+| string | The value to search. Supports ${} notation. | STRING | Any string |
++-------------+---------------------------------------------------+------------+--------------+
+| value | The value to search for. Supports ${} notation. | STRING | Any string |
++-------------+---------------------------------------------------+------------+--------------+
+
+- **Equals:**
+
+ - Choose a field from the event and a value to check for equality.
+
++-------------+----------------------------------------------+------------+--------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+==============================================+============+==============+
+| field | The field to check. Supports ${} notation. | STRING | Any string |
++-------------+----------------------------------------------+------------+--------------+
+| value | The value to match. Supports ${} notation. | STRING | Any string |
++-------------+----------------------------------------------+------------+--------------+
+
+- **FlatironObjectExists**
+
+ - Matches when the given object exists in the given Flatiron instance.
+
++-------------+---------------------------------------------+------------+--------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+=============================================+============+==============+
+| oid | The OID of the object to look for. | STRING | Any string |
++-------------+---------------------------------------------+------------+--------------+
+| flatiron | The name of the Flatiron client instance. | STRING | Any string |
++-------------+---------------------------------------------+------------+--------------+
+
+- **IsAging**
+ - Choose a field to test. This filter matches if the expression is
+ numeric.
+
++-------------+---------------------------------------------+------------+--------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+=============================================+============+==============+
+| field | The field to test. Supports ${} notation. | STRING | Any string |
++-------------+---------------------------------------------+------------+--------------+
+
+- **IsNumeric**
+ - Choose a field to test. This filter matches if the expression is
+ numeric.
+
++-------------+---------------------------------------------+------------+--------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+=============================================+============+==============+
+| field | The field to test. Supports ${} notation. | STRING | Any string |
++-------------+---------------------------------------------+------------+--------------+
+
+- **MathCondition**
+ - Choose a field from the event and a value for logical math conditions.
+
++-------------+-------------------------------------------------+------------+-----------------------------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+=================================================+============+===================================+
+| Field | The field to check. Supports ${} notation. | STRING | Any string |
++-------------+-------------------------------------------------+------------+-----------------------------------+
+| Value | The value to consider. Supports ${} notation. | STRING | Any string |
++-------------+-------------------------------------------------+------------+-----------------------------------+
+| operator | The operation. | STRING | One of { "<=", ">=", ">", "<" } |
++-------------+-------------------------------------------------+------------+-----------------------------------+
+| | | | |
++-------------+-------------------------------------------------+------------+-----------------------------------+
+
+- **NoAlarms**
+ - Don't match any alarms.
+- **Not**
+ - Negate the configured filter.
+
++-------------+-------------------------+------------+-------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+=========================+============+=============+
+| filter | The filter to negate. | FILTER | A filter |
++-------------+-------------------------+------------+-------------+
+
+- **NotEqual**
+ - Choose a field from the event and a value to check for inequality.
+
++-------------+----------------------------------------------+------------+--------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+==============================================+============+==============+
+| field | The field to check. Supports ${} notation. | STRING | Any string |
++-------------+----------------------------------------------+------------+--------------+
+| value | The value to match. Supports ${} notation. | STRING | Any string |
++-------------+----------------------------------------------+------------+--------------+
+
+- **NotOneOf**
+ - Match when the specified field does not have a value from the given list.
+
++-------------+---------------------------------------------+------------+---------------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+=============================================+============+=====================+
+| field | The field to test. Supports ${} notation. | STRING | Any string |
++-------------+---------------------------------------------+------------+---------------------+
+| values | The matching values. | LIST | A list of strings |
++-------------+---------------------------------------------+------------+---------------------+
+
+- **OneOf**
+ - Match when the specified field has a value from the given list.
+
++-------------+---------------------------------------------+------------+---------------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+=============================================+============+=====================+
+| field | The field to test. Supports ${} notation. | STRING | Any string |
++-------------+---------------------------------------------+------------+---------------------+
+| values | The matching values. | LIST | A list of strings |
++-------------+---------------------------------------------+------------+---------------------+
+
+- **Or**
+ - Create a set of filters. This filter matches when any one of them
+ matches.
+
++-------------+--------------------+------------+---------------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+====================+============+=====================+
+| filters | Combined Filters | LIST | A list of filters |
++-------------+--------------------+------------+---------------------+
+
+- **RegEx**
+ - Choose a field from the event to match against the regular expression
+ you provide.
+
++-------------+---------------------------------------------------------+------------+--------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+=========================================================+============+==============+
+| field | The text to check for a match. Supports ${} notation. | STRING | Any string |
++-------------+---------------------------------------------------------+------------+--------------+
+| value | The regular expression (pattern) to match. | STRING | Any string |
++-------------+---------------------------------------------------------+------------+--------------+
+
+- **StartsWith**
+ - Check if a search string starts with another string.
+
++-------------+---------------------------------------------------+------------+--------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=============+===================================================+============+==============+
+| string | The value to search. Supports ${} notation. | STRING | Any string |
++-------------+---------------------------------------------------+------------+--------------+
+| Value | The value to search for. Supports ${} notation. | STRING | Any string |
++-------------+---------------------------------------------------+------------+--------------+
+
+- **Unassigned**
+ - Choose a field from the event to check for assignment. This filter
+ matches when the field is not assigned.
+
++-------------------+--------------------------------------------------------+------------+-----------------+
+| **Field** | **Description** | **Type** | **Notes** |
++===================+========================================================+============+=================+
+| field | The field to check for on the event. | STRING | A field name |
++-------------------+--------------------------------------------------------+------------+-----------------+
+| emptyIsAssigned | If true, an empty value is considered an assignment. | BOOLEAN | True or False |
++-------------------+--------------------------------------------------------+------------+-----------------+
+
+- **WithinSecondsFrom**
+ - This filter matches when the specified epoch time value is within the
+ given number of seconds from the baseline time value. Both time values
+ are assumed to be in seconds. If a value is in milliseconds, set
+ baselineTimeInMillis and/or eventTimeInMillis to true.
+
++------------------------+------------------------------------------------------------+------------+-----------------+
+| **Field** | **Description** | **Type** | **Notes** |
++========================+============================================================+============+=================+
+| field | The time value to test. Supports ${} | STRING | A field name |
++------------------------+------------------------------------------------------------+------------+-----------------+
+| eventTimeInMillis | Whether to convert the event value from milliseconds. | BOOLEAN | True or False |
++------------------------+------------------------------------------------------------+------------+-----------------+
+| seconds | The number of seconds. | NUMBER | A number |
++------------------------+------------------------------------------------------------+------------+-----------------+
+| baselineTimeInMillis | Whether to convert the baseline value from milliseconds. | BOOLEAN | True or False |
++------------------------+------------------------------------------------------------+------------+-----------------+
+| baseline | The baseline time value. Supports ${}. | STRING | Any string |
++------------------------+------------------------------------------------------------+------------+-----------------+
+
+- **WithinTimeFromNow**
+ - This filter matches when the named field has an epoch time value
+ within the given number of seconds from the current time. The event's
+ time value is assumed to be in seconds. If it's in milliseconds, set
+ eventTimeInMillis to true.
+
++---------------------+---------------------------------------------------------+------------+-----------------+
+| **Field** | **Description** | **Type** | **Notes** |
++=====================+=========================================================+============+=================+
+| field | The field to check on the event. | STRING | A field name |
++---------------------+---------------------------------------------------------+------------+-----------------+
+| eventTimeInMillis | Whether to convert the event value from milliseconds. | BOOLEAN | True or False |
++---------------------+---------------------------------------------------------+------------+-----------------+
+| seconds | The number of seconds. | NUMBER | A number |
++---------------------+---------------------------------------------------------+------------+-----------------+
+
+- **Limit:**
+
+ - Limit is the integer value and DMaaP Message Router will consumes
+ only that set of message which are specified in limit.
+
+ .. code:: bash
+
+ Suppose if we set limit=2, then only 2 sets of data will be consumed.
+ *Get \ **<<topicName>>/group/2?limit=4***
+ Let us suppose if
+ **No of data available** = 4
+ **Set limit** = 6
+ i.e. limit>no of data
+ In this scenario all 4 sets of data will be consumed.
+
+- If limit is not passed with the url then by default limit is set to
+ 4096.
+
+ .. code:: bash
+
+ i.e. 4096 sets of data will be consumed.
+ **Timeout and Long Poll:**
+
+- Timeout is the integer value which will be treated by DMaaP Message
+ Router as time in millisecond.
+
+
+
+- Get
+
++-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+| `***http://localhost/DMaaP/dmaaprest/events/<<topicName>>/group/2?timeout=20000*** <http://localhost/DMaaP/dmaaprest/events/%3c%3ctopicName%3e%3e/group/2?timeout=20000>`__ |
++-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+
+- If there is no data available to be consumed, then DMaaP Message
+ Router will poll for the particular period of time specified in
+ timeout this mechanism is known as Long Poll.
+
+- If timeout is not passed with url then DMaaP Message Router will set
+ the value of timeout =10000
+
+- i.e. if no set of data are available then DMaaP Message Router will
+ poll for 10000 ms.
+
+***Meta:***
+
+- Meta is a Boolean value.
+
+- DMaaP Message Router reads the value of meta from
+ MRConfiguration.properties file at the time of startup.
+
+- If the value of meta is not null and if value of meta is one of these
+ values true, yes, on, 1, y, checked then DMaaP Message Router will
+ take meta flag as true, else it will be false.
+
+- If meta is set to true then consumer will get the value of message
+ offset along with message.
+
+ |image1|
+
+ .. |image1| image:: images/image1.png
+
+- **Pretty**:
+
+- Pretty is a Boolean value.
+
+- DMaaP Message Router reads the value of pretty from
+ MRConfiguration.properties file at the time of startup.
+
+- If the value of pretty is not null and if value of pretty is one of
+ these values true, yes, on, 1, y, checked then DMaaP Message Router
+ will take pretty flag as true, else it will be false.
+
+- If pretty is set to true then different sets of messages will be
+ printed in next line separated by comma (,).
+
+ |image2|
+
+ .. |image2| image:: images/image2.png
+
+- **Filter**
+
+ - A filter examine an event and decides if it matches or doesn't.
+ - Filters are mainly used in rules to decide if the processing entries
+ should be executed on the given event. They're also used for settings,
+ and systems like the Graph Correlator re-use Highland Park's filter
+ mechanism to specify which alarms fit in a correlation.
+ - The standard library includes a number of simple filters. The Equals
+ filter, for example, compares a field's value with some other value
+ and returns true if the values match.
+ - The standard library also includes filter classes called And, Or, and
+ Not, so you can compose more complex filters. For example, written in
+ the standard JSON config format:
+
+ +-----------------------------------------+
+ | "filter":{ |
+ | "class":"And", |
+ | "filters": |
+ | [ |
+ | { "class":"Equals", "foo":"abc" }, |
+ | { "class":"Assigned", "field":"bar" } |
+ | ] |
+ | } |
+ +-----------------------------------------+
+
+ - This filter matches events in which the field "foo" has value "abc" and
+ the field "bar" is assigned to some value (as opposed to not being
+ present on the event).
+ - Filters are used by the consumers to filter out data and consume only
+ specific set of data which matches the conditions mentioned in filter.
+ - Filters can be passed as a query parameter by the consumer in consume
+ request as mentioned below:
+ - **http://localhost:8080/DMaaP/dmaaprest/events/DMaaP/consumergroup/mHOeNFY4XiWx4CBa?filter=\\{"class":"Equals","field":"email", "value":"`test@abc.com <mailto:test@att.com>`__" }**
+ - Filters can be applied only on data in JSON format i.e. if applied, filters will automatically ignore any non-json data.
+ - While consuming, request CONTENT\_TYPE is not relevant to filter.
+
+ The MR API allows a subscriber pass a Highland Park filter as part of
+ the GET request. This will filter the stream of messages sent back to
+ the subscriber, but for this to work, there are some requirements:
+
+ - The message payload must be JSON
+
+ - Only a filter built from Highland Park's Standard Library may be
+ used. (The Cambria API server doesn't have access to plugged in
+ filters.)
+
+ - The filter must be encoded properly in the URL path.
+
+Server-side filtering can also be setup in the Java client as illustrated below
+
+**Filtering Consumer**
+
+You can also provide a Highland Park filter to your consumer instance,
+and this filter is passed on to the server in the GET request. One way
+to create the filter is programmatically. In your code, instantiate a
+filter from the Highland Park Standard Library Then create a String
+representation of the filter using the FilterIo.write utility. This
+String can then be passed to the Cambria client instance for use on the
+server.
+
+Remember, only Highland Park standard library filter components can be
+used -- no plug-ins are available in the Cambria server context.
+
+ .. code:: bash
+
+ package org.onap.sa.highlandPark.integration;
+
+ import java.io.IOException;
+
+ import java.util.UUID;
+
+ import org.onap.nsa.cambria.client.CambriaClientFactory;
+
+ import org.onap.nsa.cambria.client.CambriaConsumer;
+
+ import org.onap.sa.highlandPark.processor.HpEvent;
+
+ import org.onap.sa.highlandPark.stdlib.filters.FilterIo;
+
+ import org.onap.sa.highlandPark.stdlib.filters.OneOf;
+
+ public class ExampleFilteringConsumer
+
+ {
+
+ public static void main ( String[] args ) throws IOException,
+ InterruptedException
+
+ {
+
+ // Cambria clients take a set of 1 or more servers to use in round-robin
+ fashion.
+
+ // If a server becomes unreachable, another in the group is used.
+
+ final String
+ serverGroup="ueb01hydc.it.att.com,ueb02hydc.it.att.com,ueb03hydc.it.att.com";
+
+ // choose a topic
+
+ final String topic = "TEST-TOPIC";
+
+ // Cambria clients can run in a cooperative group to handle high-volume
+ topics.
+
+ // Here, we create a random group name, which means this client is not
+ re-startable.
+
+ final String consumerGroup = UUID.randomUUID ().toString ();
+
+ final String consumerId = "0";
+
+ // Cambria clients can sit in a tight loop on the client side, using a
+ long-poll
+
+ // to wait for messages, and a limit to tell the server the most to send
+ at a time.
+
+ final int longPollMs = 30\*1000;
+
+ final int limit = -1;
+
+ // The Cambria server can filter the returned message stream using
+ filters from the
+
+ // Highland Park system. Here, we create a simple filter to test for the
+ AlarmID
+
+ // value being one of the Mobility power alarms.
+
+ final OneOf oneOf = new OneOf ( "AlarmId", kPowerAlarms );
+
+ // create the consumer
+
+ final CambriaConsumer cc = CambriaClientFactory.createConsumer (
+ serverGroup, topic,
+
+ consumerGroup, consumerId, longPollMs, limit, FilterIo.write ( oneOf )
+ );
+
+ // now loop reading messages. Note that cc.fetch() will wait in its HTTP
+ receive
+
+ // method for up to 30 seconds (longPollMs) when nothing's available at
+ the server.
+
+ long count = 0;
+
+ while ( true )
+
+ {
+
+ for ( String msg : cc.fetch () )
+
+ {
+
+ System.out.println ( "" + (++count) + ": " + msg );
+
+ }
+
+ }
+
+ }
+
+ private static final String[] kPowerAlarms =
+
+ {
+
+ "HUB COMMERCIAL POWER FAIL\_FWD",
+
+ "HUB COMMERCIAL POWER FAIL",
+
+ "RBS COMMERCIAL POWER FAIL - Fixed\_FWD",
+
+ "RBS COMMERCIAL POWER FAIL\_FWD",
+
+ "RBS COMMERCIAL POWER FAIL - No Generator\_FWD",
+
+ "RBS COMMERCIAL POWER FAIL - Portable\_FWD",
+
+ "RBS COMMERCIAL POWER FAIL - Shared\_FWD",
+
+ "RBS COMMERCIAL POWER FAIL - Yes\_FWD",
+
+ "RBS COMMERCIAL POWER FAIL - YES\_FWD",
+
+ "RBS COMMERCIAL POWER FAIL - Fixed",
+
+ "RBS COMMERCIAL POWER FAIL - No Generator",
+
+ "RBS COMMERCIAL POWER FAIL - Portable",
+
+ "RBS COMMERCIAL POWER FAIL - Shared",
+
+ "RBS COMMERCIAL POWER FAIL - YES",
+
+ "RBS COMMERCIAL POWER FAIL - Yes",
+
+ "RBS COMMERCIAL POWER FAIL",
+
+ "HUB COMMERCIAL POWER FAIL - Fixed",
+
+ "HUB COMMERCIAL POWER FAIL - No Generator",
+
+ "HUB COMMERCIAL POWER FAIL - Portable",
+
+ "HUB COMMERCIAL POWER FAIL - Shared",
+
+ "HUB COMMERCIAL POWER FAIL - Fixed\_FWD",
+
+ "HUB COMMERCIAL POWER FAIL - No Generator\_FWD",
+
+ "HUB COMMERCIAL POWER FAIL - Portable\_FWD",
+
+ "HUB COMMERCIAL POWER FAIL - Shared\_FWD",
+
+ };
+
+ }
+
+
+
+**Filter Builder**
+
+ MR server-side filtering allows a consumer to filter the stream of
+ messages returned from the GET call. The following link provide details
+ of building some of the filter to illustrate Filter Builder. It is not
+ meant to cover and provide examples of every filter
Code Review / dmaap / messagerouter / messageservice.git / blobdiff