1 ============================================
\r
2 Message Router (MR) API Guide
\r
3 ============================================
\r
8 DMaaP Message Router utilizes an HTTP REST API to service all Publish
\r
9 and Consume transactions. HTTP and REST standards are followed so
\r
10 clients as varied as CURL, Java applications and even Web Browsers will
\r
11 work to interact with Message Router.Message Router uses AAF for user's
\r
12 authentication and authorization.
\r
14 General HTTP Requirements
\r
15 =========================
\r
17 A DMaaP Message Router transactions consists of 4 distinct segments,
\r
18 HTTP URL, HTTP Header, HTTP Body (POST) and HTTP Response. The general
\r
19 considerations for each segment are as follows and are required for each
\r
20 of the specific transactions described in this section.
\r
25 http[s]://Username:Password@serverBaseURL{/routing}{resourcePath}
\r
27 \95 The Username:Password utilizes HTTP Basic Authentication and HTTPS/TLS
\r
28 to securely transmit the authorization and authentication credentials
\r
29 that AAF needs to validate the client's access to the requested
\r
32 \95 The serverBaseURL points to DMaaP Message Router host/port that will
\r
33 service the request. Optionally DME2 service end points for Message
\r
36 \95 The resourcePath specifies the specific service, or Topic, that the
\r
37 client is attempting to reach
\r
42 Specifies HTTP Headers, such as Content-Type, that define the parameters
\r
43 of the HTTP Transaction
\r
48 The HTTP Body contains the topic content when Publishing or Consuming.
\r
49 The Body may contain topic messages in several formats (like below) but
\r
50 it must be noted, that, except in very specific circumstances, messages
\r
51 are not inspected for content.
\r
54 +-------------------------+----------------------------------------------------------------------------------------------------------------+
\r
55 | Content-Type | Description |
\r
56 +=========================+================================================================================================================+
\r
57 | text/plain | Each line in the POST body is treated as a separate message. No partition key is specified, and therefore no |
\r
58 | | order is guaranteed. This format is mainly for test, as messages are highly likely to be re-ordered when |
\r
59 | | delivered through the Kafka cluster. |
\r
60 +-------------------------+----------------------------------------------------------------------------------------------------------------+
\r
61 | application/json | The payload maybe a single JSON object or a JSON array of JSON objects. Each object is handled as an individual|
\r
62 | | message..Note that use of this format may result in equivalent but altered JSON objects sent to consumers. |
\r
63 | | That's because MR uses a standard JSON parser to read each object into memory before pushing the object to the |
\r
64 | | Kafka system. At that point, the JSON object is re-written from the in-memory object. This can result in |
\r
65 | | re-ordered fields or changes in whitespace. If you want to preseve JSON objects exactly, |
\r
66 | | use application/cambria. Recommended to follow the JSON format after validating the |
\r
67 | | message in https://jsonformatter.curiousconcept.com/ |
\r
68 +-------------------------+----------------------------------------------------------------------------------------------------------------+
\r
73 DME2 Service endpoints:
\r
74 =======================
\r
76 Message Router supports DME2 clients. That is , Client application may
\r
77 use DME2Client and DME2 service address to call the MessageRouter
\r
80 Example DME2 service address:
\r
82 Please use DMaap Message Router Run Book for complete details
\r
84 TEST: http://hostname/events?version=XXX&envContext=XXX&partner=XX
\r
86 PROD: https://hostname/events?version=XXX &envContext=XXX&partner=XX
\r
88 The values of version/envContext/routerOffer may change based upon the
\r
91 The specific details for each API are described as below
\r
101 Publishes data to Kafka server on the topic mentioned in the URL.
\r
102 Messages will be in the request body
\r
104 The MessageRouter service has no requirements on what publishers can put
\r
105 onto a topic. The messages are opaque to the service and are treated as
\r
106 raw bytes. In general, passing JSON messages is preferred, but this is
\r
107 due to higher-level features and related systems, not the MessageRouter
\r
108 broker itself. The only constraint placed on messages is their on their
\r
109 size
\96 messages must be under the maximum size, which is currently
\r
110 configured at 1 MB.
\r
115 POST http(s)://{HOST:PORT}/events/{topicname}
\r
120 +--------------------------+---------------------------------+------------------+------------+-----------+-------------+--------------------------------+---------------------------------------------------+
\r
121 | Name | Description | Param Type | Data type | Max Len | Req
\92d | Format | Valid/Example Values |
\r
122 +==========================+=================================+==================+============+===========+=============+================================+===================================================+
\r
123 | Topicname | topic name to be posted | Path | String | 40 | Y | <app namespace>.<topicname> | Org.onap.crm.empdetails |
\r
124 +--------------------------+---------------------------------+------------------+------------+-----------+-------------+--------------------------------+---------------------------------------------------+
\r
125 | `content-type | To specify type of message | Header | String | 20 | N | | application/json text/plain application/cambria |
\r
127 | | content(json,text or cambria) | | | | | | |
\r
128 +--------------------------+---------------------------------+------------------+------------+-----------+-------------+--------------------------------+---------------------------------------------------+
\r
129 | Username | userid | Header | String | | N | Basic Authentication Header | |
\r
130 +--------------------------+---------------------------------+------------------+------------+-----------+-------------+--------------------------------+---------------------------------------------------+
\r
131 | Password | | Header | String | | N | Basic Authentication Header | |
\r
132 +--------------------------+---------------------------------+------------------+------------+-----------+-------------+--------------------------------+---------------------------------------------------+
\r
133 | partitionKey | | QueryParam | String | | N | String value | ?partitionKey=123 |
\r
134 +--------------------------+---------------------------------+------------------+------------+-----------+-------------+--------------------------------+---------------------------------------------------+
\r
143 Publishers/user should have access on the topics. The user (id) and
\r
144 permissions details needs to be in AAF.
\r
146 Response Parameters:
\r
147 ====================
\r
149 +------------------+------------------------+------------+--------------+------------------------------+
\r
150 | Name | Description | Type | Format | Valid/Example Values |
\r
151 +==================+========================+============+==============+==============================+
\r
152 | httpStatusCode | | | | 200, 201 etc. |
\r
153 +------------------+------------------------+------------+--------------+------------------------------+
\r
154 | mrErrorCode | Numeric error code | | | 200, 201 etc. |
\r
155 +------------------+------------------------+------------+--------------+------------------------------+
\r
156 | errorMessage | | | | SUCCESS, or error message. |
\r
157 +------------------+------------------------+------------+--------------+------------------------------+
\r
158 | helpURL | helpurl | | | |
\r
159 +------------------+------------------------+------------+--------------+------------------------------+
\r
160 | transactionid | transaction-id value | | | |
\r
161 +------------------+------------------------+------------+--------------+------------------------------+
\r
165 Response /Error Codes:
\r
166 =====================
\r
168 +---------------------------+------------------------------------+
\r
169 | Response statusCode | Response statusMessage |
\r
170 +===========================+====================================+
\r
171 | 200-299 | Success |
\r
172 +---------------------------+------------------------------------+
\r
173 | 400-499 | the client request has a problem |
\r
174 +---------------------------+------------------------------------+
\r
175 | 500-599 | the DMaaP service has a problem |
\r
176 +---------------------------+------------------------------------+
\r
178 +------------------------+----------------+---------------------------------+---------------------------------------------------------------------------------------------------------+
\r
179 | Error code | HTTPCode | Description | Issue Reason |
\r
180 +========================+================+=================================+=========================================================================================================+
\r
181 | DMaaP\_MR\_ERR\_3001 | 413 | Request Entity too large | Message size exceeds the batch limit <limit>.Reduce the batch size and try again |
\r
182 +------------------------+----------------+---------------------------------+---------------------------------------------------------------------------------------------------------+
\r
183 | DMaaP\_MR\_ERR\_3002 | 500 | Internal Server Error | Unable to publish messages. Please contact administrator |
\r
184 +------------------------+----------------+---------------------------------+---------------------------------------------------------------------------------------------------------+
\r
185 | DMaaP\_MR\_ERR\_3003 | 400 | Bad Request | Incorrect Batching format. Please correct the batching format and try again |
\r
186 +------------------------+----------------+---------------------------------+---------------------------------------------------------------------------------------------------------+
\r
187 | DMaaP\_MR\_ERR\_3004 | 413 | Request Entity too large | Message size exceeds the message size limit <limit>.Reduce the message size and try again |
\r
188 +------------------------+----------------+---------------------------------+---------------------------------------------------------------------------------------------------------+
\r
189 | DMaaP\_MR\_ERR\_3005 | 400 | Bad Request | Incorrect JSON object. Please correct the JSON format and try again |
\r
190 +------------------------+----------------+---------------------------------+---------------------------------------------------------------------------------------------------------+
\r
191 | DMaaP\_MR\_ERR\_3006 | 504 | Network Connect Timeout Error | Connection to the DMaaP MR was timed out.Please try again |
\r
192 +------------------------+----------------+---------------------------------+---------------------------------------------------------------------------------------------------------+
\r
193 | DMaaP\_MR\_ERR\_3007 | 500 | Internal Server Error | Failed to publish messages to topic <topicName>. Successfully published <count > number of messages. |
\r
194 +------------------------+----------------+---------------------------------+---------------------------------------------------------------------------------------------------------+
\r
195 | | 503 | Service Unavailable | Service Unavailable |
\r
196 +------------------------+----------------+---------------------------------+---------------------------------------------------------------------------------------------------------+
\r
203 +-----------------------------------------------------------------------------------+
\r
206 | *Payload-* *{"message":"message description"}* *Content-Type: application/json* |
\r
210 | curl -u XXXX@abc.com:X -H 'Content-Type:text/plain' -X POST -d @sampleMsg.txt |
\r
214 | "serverTimeMs": 19" |
\r
217 +-----------------------------------------------------------------------------------+
\r
222 +---------------------------------------------------------------------+
\r
223 | HTTP/1.1 200 OK |
\r
225 | Server: Apache-Coyote/1.1 |
\r
227 | transactionId: 28-12-2015::08:18:50:682::<IP>::28122015552391 |
\r
229 | Content-Type: application/json |
\r
231 | Content-Length: 42 |
\r
233 | Date: Mon, 28 Dec 2015 13:18:50 GMT |
\r
234 +---------------------------------------------------------------------+
\r
243 To subscribe to a MessageRouter topic, a subscriber issues a GET to the RESTful HTTP endpoint for events.
\r
250 GET http(s)://{HOST:PORT}}/events/{topicname}/{consumegroup}/{consumerid}?{timeout=x}
\r
252 Request Parameters:
\r
255 +-------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-------------------------------------------------+
\r
256 | Name | Description | Param Type | data type | MaxLen | Req
\92d | Format | Valid/Example Values |
\r
258 +-------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-------------------------------------------------+
\r
259 | Topicname | topic name to be posted | Path | String | 40 | Y | namespace. | |
\r
260 | | | | | | | String | |
\r
261 +-------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-------------------------------------------------+
\r
262 |Consumergroup| A name that uniquely identifies | Path | String | | | | |
\r
263 | | your subscriber's | | | | Y | | CG1 |
\r
264 +-------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-------------------------------------------------+
\r
265 | consumerId | Within your subscriber's group, | | | | | | |
\r
266 | | a name that uniquely identifies | Path | String | | y | | C1 |
\r
267 | | your subscriber's process | | | | | | |
\r
268 +-------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-------------------------------------------------+
\r
269 | content-type| To specify type of message | | | | | |aplication/json |
\r
270 | | content(json,text or cambria) | Header | String | 20 | N | | |
\r
272 +-------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-------------------------------------------------+
\r
273 |Username | userid | Header | String | 1 | N | | |
\r
274 +-------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-------------------------------------------------+
\r
275 | Password | | Header | String | 1 | N | | |
\r
276 +-------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-------------------------------------------------+
\r
280 Subscribers /user should have access on the topics. The user () and
\r
281 permissions details needs to be in AAF.
\r
287 A few consumer client app team who does continuous polling (by
\r
288 multiple rest calls) complained that they receive "503 consumer lock
\r
289 exception". The messages will not be lost from the topic when this
\r
290 exception occurs and it will be still in the topic. This happens when
\r
291 the concurrent rest call requests are made between several nodes in a
\r
292 cluster. Consumer lock is done at zookeeper level to ensure the offset
\r
293 for each consumer group is maintained to avoid losing the message in
\r
294 multiple concurrent calls of same consumer Group. We are looking
\r
295 otherways to resolve this issue. But for these continuous polling
\r
296 scenario, the suggested workaround is to ensure the request is made to
\r
297 only one node of the cluster. Session stickiness in DME2 is example of
\r
298 handling this. If this exception occurs , waiting for few minutes with
\r
299 out making api calls will release the lock in zookeeper.**
\r
301 Response Parameters:
\r
302 ===================
\r
304 +------------------+--------------------------------+------------+--------------+-----------------------------------------------------------+
\r
305 | Name | Description | Type | Format | Valid/Example Values |
\r
306 +==================+================================+============+==============+===========================================================+
\r
307 | httpStatusCode | | | | 200, 201 etc. |
\r
308 +------------------+--------------------------------+------------+--------------+-----------------------------------------------------------+
\r
309 | mrErrorCode | Numeric error code | | | 200, 201 etc. |
\r
310 +------------------+--------------------------------+------------+--------------+-----------------------------------------------------------+
\r
311 | errorMessage | | | | SUCCESS, or error message. |
\r
312 +------------------+--------------------------------+------------+--------------+-----------------------------------------------------------+
\r
313 | helpURL | helpurl | | | |
\r
314 +------------------+--------------------------------+------------+--------------+-----------------------------------------------------------+
\r
315 | tranactionid | transaction-id value | | | 28-12-2015::08:18:50:682::135.25.227.66::28122015552391 |
\r
316 +------------------+--------------------------------+------------+--------------+-----------------------------------------------------------+
\r
317 | ResponseBody | Messages consumed from topic | Json | Json | |
\r
318 +------------------+--------------------------------+------------+--------------+-----------------------------------------------------------+
\r
322 +---------------------------+------------------------------------+
\r
323 | Response statusCode | Response statusMessage |
\r
324 +===========================+====================================+
\r
325 | 200-299 | Success |
\r
326 +---------------------------+------------------------------------+
\r
327 | 400-499 | the client request has a problem |
\r
328 +---------------------------+------------------------------------+
\r
329 | 500-599 | the DMaaP service has a problem |
\r
330 +---------------------------+------------------------------------+
\r
334 +-------------------------+-----------------+----------------------------+----------------------------------------------------------------------------------------------------+
\r
335 | Error code | HTTP Code | Description |Issue reason |
\r
336 +-------------------------+-----------------+----------------------------+----------------------------------------------------------------------------------------------------+
\r
337 | DMaaP\_MR\_ERR\_3008 | 413 | Request Entity too large | Message size exceeds the batch limit <limit>.Reduce the batch size and try again | +-------------------------+-----------------+----------------------------+----------------------------------------------------------------------------------------------------+
\r
338 | DMaaP\_MR\_ERR\_3009 | 500 | Internal Server Error | Unable to publish messages. Please contact administartor | +-------------------------+-----------------+----------------------------+----------------------------------------------------------------------------------------------------+
\r
339 | DMaaP\_MR\_ERR\_3010 | 400 | Bad Request | Incorrect Batching format. Please correct the batching format and try again | +-------------------------+-----------------+----------------------------+----------------------------------------------------------------------------------------------------+
\r
340 | DMaaP\_MR\_ERR\_3011 | 413 | Request Entity too large | Message size exceeds the message size limit <limit>.Reduce the message size and try again | +-------------------------+-----------------+----------------------------+----------------------------------------------------------------------------------------------------+
\r
341 | DMaaP\_MR\_ERR\_5012 | 429 | Too many requests | This client is making too many requests. Please use a long poll setting to decrease the number of | | | | | requests that result in empty responses. |
\r
342 +-------------------------+-----------------+----------------------------+----------------------------------------------------------------------------------------------------+
\r
343 | | 503 | Service Unavailable | Service Unavailable | +-------------------------+-----------------+----------------------------+----------------------------------------------------------------------------------------------------+
\r
Code Review / dmaap / messagerouter / messageservice.git / blob