+Filter \r
+\r
+A filter examine an event and decides if it matches or doesn't. Filters\r
+are mainly used in rules to decide if the processing entries should be\r
+executed on the given event. They're also used for settings, and systems\r
+like the Graph Correlator re-use Highland Park's filter mechanism to\r
+specify which alarms fit in a correlation. Some publishers may produce\r
+topics with a lot of volume and a subscriber may want only a portion of\r
+those messages. The subscriber can certainly filter messages after\r
+receiving them, but it may be substantially more efficient to ask the\r
+API server to filter the messages before sending them to the\r
+subscriber.The standard library includes a number of simple filters. The\r
+Equals filter, for example, compares a field's value with some other\r
+value and returns true if the values match. The standard library also\r
+includes filter classes called And, Or, and Not, so you can compose more\r
+complex filters. For example, written in the standard JSON config\r
+format: \r
+\r
++-----------------------------------------+\r
+| "filter":{ |\r
+| "class":"And", |\r
+| "filters": |\r
+| [ |\r
+| { "class":"Equals", "foo":"abc" }, |\r
+| { "class":"Assigned", "field":"bar" } |\r
+| ] |\r
+| } |\r
++-----------------------------------------+\r
+\r
+| This filter matches events in which the field "foo" has value "abc"\r
+ and the field "bar" is assigned to some value (as opposed to not being\r
+ present on the event). Filters are used by the consumers to filter out\r
+ data and consume only specific set of data which matches the\r
+ conditions mentioned in filter. Filters can be passed as a query\r
+ parameter by the consumer in consume request as mentioned below:\r
+ **http://localhost:8080/DMaaP/dmaaprest/events/DMaaP/consumergroup/mHOeNFY4XiWx4CBa?filter=\\{"class":"Equals",\r
+ "field":"email", "value":"test@abc.com" }** \r
+| Filters can be applied only on data in JSON format i.e. if applied,\r
+ filters will automatically ignore any non-json data. While consuming,\r
+ request CONTENT\_TYPE is not relevant to filter.\r
+\r
+All the supported filter can be found below.\r
+\r
+`Types of\r
+Filters <https://wiki.web.att.com/display/MessageRouter/Types+of+Filters>`__\r
+\r
+| DMaaP Message Router supports all the filters which were supported by\r
+ DMaaP Message Router and are mentioned below:- \r
+| **All Alarms:**\r
+| Match all alarms. \r
+| **And:**\r
+| Create a set of filters. This filter matches when all of them matches.\r
+\r
++-------------+--------------------+------------+---------------------+\r
+| **Field** | **Description** | **Type** | **Notes** |\r
++=============+====================+============+=====================+\r
+| filters | Combined Filters | LIST | A list of filters |\r
++-------------+--------------------+------------+---------------------+\r
+| | | | |\r
++-------------+--------------------+------------+---------------------+\r
+\r
+| **Assigned:**\r
+| Choose a field from the event to check for assignment. This filter\r
+ matches when the field is assigned.\r
+\r
++-------------------+--------------------------------------------------------+------------+-----------------+\r
+| **Field** | **Description** | **Type** | **Notes** |\r
++===================+========================================================+============+=================+\r
+| field | The field to check for on the event. | STRING | A field name |\r
++-------------------+--------------------------------------------------------+------------+-----------------+\r
+| emptyIsAssigned | If true, an empty value is considered an assignment. | BOOLEAN | True or False |\r
++-------------------+--------------------------------------------------------+------------+-----------------+\r
+\r
+| **Contains:**\r
+| Check if a search string contains another string.\r
+\r
++-------------+---------------------------------------------------+------------+--------------+\r
+| **Field** | **Description** | **Type** | **Notes** |\r
++=============+===================================================+============+==============+\r
+| String | The value to search. Supports ${} notation. | STRING | Any string |\r
++-------------+---------------------------------------------------+------------+--------------+\r
+| Value | The value to search for. Supports ${} notation. | STRING | Any string |\r
++-------------+---------------------------------------------------+------------+--------------+\r
+| | | | |\r
++-------------+---------------------------------------------------+------------+--------------+\r
+\r
+| **EndsWith**:\r
+| Check if a search string ends with another string.\r
+\r
++-------------+---------------------------------------------------+------------+--------------+\r
+| **Field** | **Description** | **Type** | **Notes** |\r
++=============+===================================================+============+==============+\r
+| string | The value to search. Supports ${} notation. | STRING | Any string |\r
++-------------+---------------------------------------------------+------------+--------------+\r
+| value | The value to search for. Supports ${} notation. | STRING | Any string |\r
++-------------+---------------------------------------------------+------------+--------------+\r
+\r
+| **Equals:**\r
+| Choose a field from the event and a value to check for equality.\r
+\r
++-------------+----------------------------------------------+------------+--------------+\r
+| **Field** | **Description** | **Type** | **Notes** |\r
++=============+==============================================+============+==============+\r
+| field | The field to check. Supports ${} notation. | STRING | Any string |\r
++-------------+----------------------------------------------+------------+--------------+\r
+| value | The value to match. Supports ${} notation. | STRING | Any string |\r
++-------------+----------------------------------------------+------------+--------------+\r
+\r
+| **FlatironObjectExists**\r
+| Matches when the given object exists in the given Flatiron instance.\r
+\r
++-------------+---------------------------------------------+------------+--------------+\r
+| **Field** | **Description** | **Type** | **Notes** |\r
++=============+=============================================+============+==============+\r
+| oid | The OID of the object to look for. | STRING | Any string |\r
++-------------+---------------------------------------------+------------+--------------+\r
+| flatiron | The name of the Flatiron client instance. | STRING | Any string |\r
++-------------+---------------------------------------------+------------+--------------+\r
+\r
+| **IsAging**\r
+| Choose a field to test. This filter matches if the expression is\r
+ numeric.\r
+\r
++-------------+---------------------------------------------+------------+--------------+\r
+| **Field** | **Description** | **Type** | **Notes** |\r
++=============+=============================================+============+==============+\r
+| field | The field to test. Supports ${} notation. | STRING | Any string |\r
++-------------+---------------------------------------------+------------+--------------+\r
+\r
+| **IsNumeric**\r
+| Choose a field to test. This filter matches if the expression is\r
+ numeric.\r
+\r
++-------------+---------------------------------------------+------------+--------------+\r
+| **Field** | **Description** | **Type** | **Notes** |\r
++=============+=============================================+============+==============+\r
+| field | The field to test. Supports ${} notation. | STRING | Any string |\r
++-------------+---------------------------------------------+------------+--------------+\r
+\r
+| **MathCondition**\r
+| Choose a field from the event and a value for logical math conditions.\r
+\r
++-------------+-------------------------------------------------+------------+-----------------------------------+\r
+| **Field** | **Description** | **Type** | **Notes** |\r
++=============+=================================================+============+===================================+\r
+| Field | The field to check. Supports ${} notation. | STRING | Any string |\r
++-------------+-------------------------------------------------+------------+-----------------------------------+\r
+| Value | The value to consider. Supports ${} notation. | STRING | Any string |\r
++-------------+-------------------------------------------------+------------+-----------------------------------+\r
+| operator | The operation. | STRING | One of { "<=", ">=", ">", "<" } |\r
++-------------+-------------------------------------------------+------------+-----------------------------------+\r
+| | | | |\r
++-------------+-------------------------------------------------+------------+-----------------------------------+\r
+\r
+| **NoAlarms**\r
+| Don't match any alarms. \r
+| **Not**\r
+| Negate the configured filter.\r
+\r
++-------------+-------------------------+------------+-------------+\r
+| **Field** | **Description** | **Type** | **Notes** |\r
++=============+=========================+============+=============+\r
+| filter | The filter to negate. | FILTER | A filter |\r
++-------------+-------------------------+------------+-------------+\r
+\r
+| **NotEqual**\r
+| Choose a field from the event and a value to check for inequality.\r
+\r
++-------------+----------------------------------------------+------------+--------------+\r
+| **Field** | **Description** | **Type** | **Notes** |\r
++=============+==============================================+============+==============+\r
+| field | The field to check. Supports ${} notation. | STRING | Any string |\r
++-------------+----------------------------------------------+------------+--------------+\r
+| value | The value to match. Supports ${} notation. | STRING | Any string |\r
++-------------+----------------------------------------------+------------+--------------+\r
+\r
+| **NotOneOf**\r
+| Match when the specified field does not have a value from the given\r
+ list.\r
+\r
++-------------+---------------------------------------------+------------+---------------------+\r
+| **Field** | **Description** | **Type** | **Notes** |\r
++=============+=============================================+============+=====================+\r
+| field | The field to test. Supports ${} notation. | STRING | Any string |\r
++-------------+---------------------------------------------+------------+---------------------+\r
+| values | The matching values. | LIST | A list of strings |\r
++-------------+---------------------------------------------+------------+---------------------+\r
+\r
+| **OneOf**\r
+| Match when the specified field has a value from the given list.\r
+\r
++-------------+---------------------------------------------+------------+---------------------+\r
+| **Field** | **Description** | **Type** | **Notes** |\r
++=============+=============================================+============+=====================+\r
+| field | The field to test. Supports ${} notation. | STRING | Any string |\r
++-------------+---------------------------------------------+------------+---------------------+\r
+| values | The matching values. | LIST | A list of strings |\r
++-------------+---------------------------------------------+------------+---------------------+\r
+\r
+| **Or**\r
+| Create a set of filters. This filter matches when any one of them\r
+ matches.\r
+\r
++-------------+--------------------+------------+---------------------+\r
+| **Field** | **Description** | **Type** | **Notes** |\r
++=============+====================+============+=====================+\r
+| filters | Combined Filters | LIST | A list of filters |\r
++-------------+--------------------+------------+---------------------+\r
+\r
+| **RegEx**\r
+| Choose a field from the event to match against the regular expression\r
+ you provide.\r
+\r
++-------------+---------------------------------------------------------+------------+--------------+\r
+| **Field** | **Description** | **Type** | **Notes** |\r
++=============+=========================================================+============+==============+\r
+| field | The text to check for a match. Supports ${} notation. | STRING | Any string |\r
++-------------+---------------------------------------------------------+------------+--------------+\r
+| value | The regular expression (pattern) to match. | STRING | Any string |\r
++-------------+---------------------------------------------------------+------------+--------------+\r
+\r
+| **StartsWith**\r
+| Check if a search string starts with another string.\r
+\r
++-------------+---------------------------------------------------+------------+--------------+\r
+| **Field** | **Description** | **Type** | **Notes** |\r
++=============+===================================================+============+==============+\r
+| string | The value to search. Supports ${} notation. | STRING | Any string |\r
++-------------+---------------------------------------------------+------------+--------------+\r
+| Value | The value to search for. Supports ${} notation. | STRING | Any string |\r
++-------------+---------------------------------------------------+------------+--------------+\r
+\r
+| **Unassigned**\r
+| Choose a field from the event to check for assignment. This filter\r
+ matches when the field is not assigned.\r
+\r
++-------------------+--------------------------------------------------------+------------+-----------------+\r
+| **Field** | **Description** | **Type** | **Notes** |\r
++===================+========================================================+============+=================+\r
+| field | The field to check for on the event. | STRING | A field name |\r
++-------------------+--------------------------------------------------------+------------+-----------------+\r
+| emptyIsAssigned | If true, an empty value is considered an assignment. | BOOLEAN | True or False |\r
++-------------------+--------------------------------------------------------+------------+-----------------+\r
+\r
+| **WithinSecondsFrom**\r
+| This filter matches when the specified epoch time value is within the\r
+ given number of seconds from the baseline time value. Both time values\r
+ are assumed to be in seconds. If a value is in milliseconds, set\r
+ baselineTimeInMillis and/or eventTimeInMillis to true.\r
+\r
++------------------------+------------------------------------------------------------+------------+-----------------+\r
+| **Field** | **Description** | **Type** | **Notes** |\r
++========================+============================================================+============+=================+\r
+| field | The time value to test. Supports ${} | STRING | A field name |\r
++------------------------+------------------------------------------------------------+------------+-----------------+\r
+| eventTimeInMillis | Whether to convert the event value from milliseconds. | BOOLEAN | True or False |\r
++------------------------+------------------------------------------------------------+------------+-----------------+\r
+| seconds | The number of seconds. | NUMBER | A number |\r
++------------------------+------------------------------------------------------------+------------+-----------------+\r
+| baselineTimeInMillis | Whether to convert the baseline value from milliseconds. | BOOLEAN | True or False |\r
++------------------------+------------------------------------------------------------+------------+-----------------+\r
+| baseline | The baseline time value. Supports ${}. | STRING | Any string |\r
++------------------------+------------------------------------------------------------+------------+-----------------+\r
+\r
+| **WithinTimeFromNow**\r
+| This filter matches when the named field has an epoch time value\r
+ within the given number of seconds from the current time. The event's\r
+ time value is assumed to be in seconds. If it's in milliseconds, set\r
+ eventTimeInMillis to true.\r
+\r
++---------------------+---------------------------------------------------------+------------+-----------------+\r
+| **Field** | **Description** | **Type** | **Notes** |\r
++=====================+=========================================================+============+=================+\r
+| field | The field to check on the event. | STRING | A field name |\r
++---------------------+---------------------------------------------------------+------------+-----------------+\r
+| eventTimeInMillis | Whether to convert the event value from milliseconds. | BOOLEAN | True or False |\r
++---------------------+---------------------------------------------------------+------------+-----------------+\r
+| seconds | The number of seconds. | NUMBER | A number |\r
++---------------------+---------------------------------------------------------+------------+-----------------+\r
+\r
+**Limit:** \r
+\r
+- Limit is the integer value and DMaaP Message Router will consumes\r
+ only that set of message which are specified in limit.\r
+\r
+| Suppose if we set limit=2, then only 2 sets of data will be consumed. \r
+| *Get \ **<<topicName>>/group/2?limit=4*** \r
+| Let us suppose if \r
+| **No of data available** = 4\r
+| **Set limit** = 6\r
+| i.e. limit>no of data\r
+| In this scenario all 4 sets of data will be consumed. \r
+\r
+- If limit is not passed with the url then by default limit is set to\r
+ 4096.\r
+\r
+| i.e. 4096 sets of data will be consumed. \r
+| **Timeout and Long Poll:** \r
+\r
+- Timeout is the integer value which will be treated by DMaaP Message\r
+ Router as time in millisecond.\r
+\r
+ \r
+\r
+- Get\r
+\r
++-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+\r
+| `***http://localhost/DMaaP/dmaaprest/events/<<topicName>>/group/2?timeout=20000*** <http://localhost/DMaaP/dmaaprest/events/%3c%3ctopicName%3e%3e/group/2?timeout=20000>`__ |\r
++-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+\r
+\r
+- If there is no data available to be consumed, then DMaaP Message\r
+ Router will poll for the particular period of time specified in\r
+ timeout this mechanism is known as Long Poll.\r
+\r
+- If timeout is not passed with url then DMaaP Message Router will set\r
+ the value of timeout =10000\r
+\r
+- i.e. if no set of data are available then DMaaP Message Router will\r
+ poll for 10000 ms.\r
+\r
+***Meta:***\r
+\r
+- Meta is a Boolean value.\r
+\r
+- DMaaP Message Router reads the value of meta from\r
+ MRConfiguration.properties file at the time of startup.\r
+\r
+- If the value of meta is not null and if value of meta is one of these\r
+ values true, yes, on, 1, y, checked then DMaaP Message Router will\r
+ take meta flag as true, else it will be false.\r
+\r
+- If meta is set to true then consumer will get the value of message\r
+ offset along with message.\r
+\r
+|image1|\r
+| **Pretty**:\r
+\r
+- Pretty is a Boolean value.\r
+\r
+- DMaaP Message Router reads the value of pretty from\r
+ MRConfiguration.properties file at the time of startup.\r
+\r
+- If the value of pretty is not null and if value of pretty is one of\r
+ these values true, yes, on, 1, y, checked then DMaaP Message Router\r
+ will take pretty flag as true, else it will be false.\r
+\r
+- If pretty is set to true then different sets of messages will be\r
+ printed in next line separated by comma (,).\r
+\r
+|image2|\r
+| **Filter**\r
+| A filter examine an event and decides if it matches or doesn't. \r
+| Filters are mainly used in rules to decide if the processing entries\r
+ should be executed on the given event. They're also used for settings,\r
+ and systems like the Graph Correlator re-use Highland Park's filter\r
+ mechanism to specify which alarms fit in a correlation. \r
+| The standard library includes a number of simple filters. The Equals\r
+ filter, for example, compares a field's value with some other value\r
+ and returns true if the values match. \r
+| The standard library also includes filter classes called And, Or, and\r
+ Not, so you can compose more complex filters. For example, written in\r
+ the standard JSON config format: \r
+\r
++-----------------------------------------+\r
+| "filter":{ |\r
+| "class":"And", |\r
+| "filters": |\r
+| [ |\r
+| { "class":"Equals", "foo":"abc" }, |\r
+| { "class":"Assigned", "field":"bar" } |\r
+| ] |\r
+| } |\r
++-----------------------------------------+\r
+\r
+| This filter matches events in which the field "foo" has value "abc"\r
+ and the field "bar" is assigned to some value (as opposed to not being\r
+ present on the event).\r
+| Filters are used by the consumers to filter out data and consume only\r
+ specific set of data which matches the conditions mentioned in filter.\r
+| Filters can be passed as a query parameter by the consumer in consume\r
+ request as mentioned below:\r
+| **http://localhost:8080/DMaaP/dmaaprest/events/DMaaP/consumergroup/mHOeNFY4XiWx4CBa?filter=\\{"class":"Equals",\r
+ "field":"email", "value":"`test@abc.com <mailto:test@att.com>`__" }** \r
+| Filters can be applied only on data in JSON format i.e. if applied,\r
+ filters will automatically ignore any non-json data. \r
+| While consuming, request CONTENT\_TYPE is not relevant to filter.\r
+\r
+The MR API allows a subscriber pass a Highland Park filter as part of\r
+the GET request. This will filter the stream of messages sent back to\r
+the subscriber, but for this to work, there are some requirements: \r
+\r
+- The message payload must be JSON\r
+\r
+- Only a filter built from Highland Park's Standard Library may be\r
+ used. (The Cambria API server doesn't have access to plugged in\r
+ filters.)\r
+\r
+- The filter must be encoded properly in the URL path.\r
+\r
+ Server-side filtering can also be setup in the Java client as\r
+illustrated below\r
+\r
+**Filtering Consumer**\r
+\r
+You can also provide a Highland Park filter to your consumer instance,\r
+and this filter is passed on to the server in the GET request. One way\r
+to create the filter is programmatically. In your code, instantiate a\r
+filter from the Highland Park Standard Library Then create a String\r
+representation of the filter using the FilterIo.write utility. This\r
+String can then be passed to the Cambria client instance for use on the\r
+server.\r
+\r
+Remember, only Highland Park standard library filter components can be\r
+used -- no plug-ins are available in the Cambria server context.\r
+\r
+package org.onap.sa.highlandPark.integration;\r
+\r
+import java.io.IOException;\r
+\r
+import java.util.UUID;\r
+\r
+import org.onap.nsa.cambria.client.CambriaClientFactory;\r
+\r
+import org.onap.nsa.cambria.client.CambriaConsumer;\r
+\r
+import org.onap.sa.highlandPark.processor.HpEvent;\r
+\r
+import org.onap.sa.highlandPark.stdlib.filters.FilterIo;\r
+\r
+import org.onap.sa.highlandPark.stdlib.filters.OneOf;\r
+\r
+public class ExampleFilteringConsumer\r
+\r
+{\r
+\r
+public static void main ( String[] args ) throws IOException,\r
+InterruptedException\r
+\r
+{\r
+\r
+// Cambria clients take a set of 1 or more servers to use in round-robin\r
+fashion.\r
+\r
+// If a server becomes unreachable, another in the group is used.\r
+\r
+final String\r
+serverGroup="ueb01hydc.it.att.com,ueb02hydc.it.att.com,ueb03hydc.it.att.com";\r
+\r
+// choose a topic\r
+\r
+final String topic = "TEST-TOPIC";\r
+\r
+// Cambria clients can run in a cooperative group to handle high-volume\r
+topics.\r
+\r
+// Here, we create a random group name, which means this client is not\r
+re-startable.\r
+\r
+final String consumerGroup = UUID.randomUUID ().toString ();\r
+\r
+final String consumerId = "0";\r
+\r
+// Cambria clients can sit in a tight loop on the client side, using a\r
+long-poll\r
+\r
+// to wait for messages, and a limit to tell the server the most to send\r
+at a time.\r
+\r
+final int longPollMs = 30\*1000;\r
+\r
+final int limit = -1;\r
+\r
+// The Cambria server can filter the returned message stream using\r
+filters from the\r
+\r
+// Highland Park system. Here, we create a simple filter to test for the\r
+AlarmID\r
+\r
+// value being one of the Mobility power alarms.\r
+\r
+final OneOf oneOf = new OneOf ( "AlarmId", kPowerAlarms );\r
+\r
+// create the consumer\r
+\r
+final CambriaConsumer cc = CambriaClientFactory.createConsumer (\r
+serverGroup, topic,\r
+\r
+consumerGroup, consumerId, longPollMs, limit, FilterIo.write ( oneOf )\r
+);\r
+\r
+// now loop reading messages. Note that cc.fetch() will wait in its HTTP\r
+receive\r
+\r
+// method for up to 30 seconds (longPollMs) when nothing's available at\r
+the server.\r
+\r
+long count = 0;\r
+\r
+while ( true )\r
+\r
+{\r
+\r
+for ( String msg : cc.fetch () )\r
+\r
+{\r
+\r
+System.out.println ( "" + (++count) + ": " + msg );\r
+\r
+}\r
+\r
+}\r
+\r
+}\r
+\r
+private static final String[] kPowerAlarms =\r
+\r
+{\r
+\r
+"HUB COMMERCIAL POWER FAIL\_FWD",\r
+\r
+"HUB COMMERCIAL POWER FAIL",\r
+\r
+"RBS COMMERCIAL POWER FAIL - Fixed\_FWD",\r
+\r
+"RBS COMMERCIAL POWER FAIL\_FWD",\r
+\r
+"RBS COMMERCIAL POWER FAIL - No Generator\_FWD",\r
+\r
+"RBS COMMERCIAL POWER FAIL - Portable\_FWD",\r
+\r
+"RBS COMMERCIAL POWER FAIL - Shared\_FWD",\r
+\r
+"RBS COMMERCIAL POWER FAIL - Yes\_FWD",\r
+\r
+"RBS COMMERCIAL POWER FAIL - YES\_FWD",\r
+\r
+"RBS COMMERCIAL POWER FAIL - Fixed",\r
+\r
+"RBS COMMERCIAL POWER FAIL - No Generator",\r
+\r
+"RBS COMMERCIAL POWER FAIL - Portable",\r
+\r
+"RBS COMMERCIAL POWER FAIL - Shared",\r
+\r
+"RBS COMMERCIAL POWER FAIL - YES",\r
+\r
+"RBS COMMERCIAL POWER FAIL - Yes",\r
+\r
+"RBS COMMERCIAL POWER FAIL",\r
+\r
+"HUB COMMERCIAL POWER FAIL - Fixed",\r
+\r
+"HUB COMMERCIAL POWER FAIL - No Generator",\r
+\r
+"HUB COMMERCIAL POWER FAIL - Portable",\r
+\r
+"HUB COMMERCIAL POWER FAIL - Shared",\r
+\r
+"HUB COMMERCIAL POWER FAIL - Fixed\_FWD",\r
+\r
+"HUB COMMERCIAL POWER FAIL - No Generator\_FWD",\r
+\r
+"HUB COMMERCIAL POWER FAIL - Portable\_FWD",\r
+\r
+"HUB COMMERCIAL POWER FAIL - Shared\_FWD",\r
+\r
+};\r
+\r
+}\r
+\r
+ \r
+\r
+** Filter Builder**\r
+\r
+ MR server-side filtering allows a consumer to filter the stream of\r
+messages returned from the GET call. The following link provide details\r
+of building some of the filter to illustrate Filter Builder. It is not\r
+meant to cover and provide examples of every filter\r
+\r
+.. |image1| image:: images/image1.png\r
+.. |image2| image:: images/image2.png\r
+\r