From: sg481n Date: Sun, 24 Sep 2017 17:32:56 +0000 (+0000) Subject: Release MR API documents X-Git-Tag: v1.0.1~37 X-Git-Url: https://gerrit.onap.org/r/gitweb?a=commitdiff_plain;h=57fd07eeb7921ec6648a933cdf01efa2d64147ad;p=dmaap%2Fmessagerouter%2Fmessageservice.git Release MR API documents Issue-id: DMAAP-69 Change-Id: I84bff1eb7d2e793eccbadf4ec401ee08347c134f Signed-off-by: sg481n --- diff --git a/docs/index.rst b/docs/index.rst new file mode 100644 index 0000000..be660db --- /dev/null +++ b/docs/index.rst @@ -0,0 +1,15 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. Copyright © 2017 AT&T Intellectual Property. All rights reserved. + +Message Router (MR) +==================== + + + +.. toctree:: + :maxdepth: 2 + + message-router/message-router + + diff --git a/docs/message-router/message-router.rst b/docs/message-router/message-router.rst new file mode 100644 index 0000000..cbb755f --- /dev/null +++ b/docs/message-router/message-router.rst @@ -0,0 +1,348 @@ +============================================ +Message Router (MR) API Guide +============================================ + +HTTP Service APIs: +================== + +DMaaP Message Router utilizes an HTTP REST API to service all Publish +and Consume transactions. HTTP and REST standards are followed so +clients as varied as CURL, Java applications and even Web Browsers will +work to interact with Message Router.Message Router uses AAF for user's +authentication and authorization. + +General HTTP Requirements +========================= + +A DMaaP Message Router transactions consists of 4 distinct segments, +HTTP URL, HTTP Header, HTTP Body (POST) and HTTP Response. The general +considerations for each segment are as follows and are required for each +of the specific transactions described in this section. + +HTTP URL +======== + +http[s]://Username:Password@serverBaseURL{/routing}{resourcePath} + +• The Username:Password utilizes HTTP Basic Authentication and HTTPS/TLS +to securely transmit the authorization and authentication credentials +that AAF needs to validate the client's access to the requested +resource. + +• The serverBaseURL points to DMaaP Message Router host/port that will +service the request. Optionally DME2 service end points for Message +Router can be used. + +• The resourcePath specifies the specific service, or Topic, that the +client is attempting to reach + +HTTP Header +=========== + +Specifies HTTP Headers, such as Content-Type, that define the parameters +of the HTTP Transaction + +HTTP Body +========= + +The HTTP Body contains the topic content when Publishing or Consuming. +The Body may contain topic messages in several formats (like below) but +it must be noted, that, except in very specific circumstances, messages +are not inspected for content. + + ++-------------------------+----------------------------------------------------------------------------------------------------------------+ +| Content-Type | Description | ++=========================+================================================================================================================+ +| text/plain | Each line in the POST body is treated as a separate message. No partition key is specified, and therefore no | +| | order is guaranteed. This format is mainly for test, as messages are highly likely to be re-ordered when | +| | delivered through the Kafka cluster. | ++-------------------------+----------------------------------------------------------------------------------------------------------------+ +| application/json | The payload maybe a single JSON object or a JSON array of JSON objects. Each object is handled as an individual| +| | message..Note that use of this format may result in equivalent but altered JSON objects sent to consumers. | +| | That's because MR uses a standard JSON parser to read each object into memory before pushing the object to the | +| | Kafka system. At that point, the JSON object is re-written from the in-memory object. This can result in | +| | re-ordered fields or changes in whitespace. If you want to preseve JSON objects exactly, | +| | use application/cambria. Recommended to follow the JSON format after validating the | +| | message in https://jsonformatter.curiousconcept.com/ | ++-------------------------+----------------------------------------------------------------------------------------------------------------+ + + + + +DME2 Service endpoints: +======================= + +Message Router supports DME2 clients. That is , Client application may +use DME2Client and DME2 service address to call the MessageRouter +service. + +Example DME2 service address: + +Please use DMaap Message Router Run Book for complete details + +TEST: http://hostname/events?version=XXX&envContext=XXX&partner=XX + +PROD: https://hostname/events?version=XXX &envContext=XXX&partner=XX + +The values of version/envContext/routerOffer may change based upon the +environment. + +The specific details for each API are described as below + + + +Publishers +========== + +Description: +=========== + +Publishes data to Kafka server on the topic mentioned in the URL. +Messages will be in the request body + +The MessageRouter service has no requirements on what publishers can put +onto a topic. The messages are opaque to the service and are treated as +raw bytes. In general, passing JSON messages is preferred, but this is +due to higher-level features and related systems, not the MessageRouter +broker itself. The only constraint placed on messages is their on their +size – messages must be under the maximum size, which is currently +configured at 1 MB. + +Request URL: +=========== + +POST http(s)://{HOST:PORT}/events/{topicname} + +Request Parameters +================== + ++--------------------------+---------------------------------+------------------+------------+-----------+-------------+--------------------------------+---------------------------------------------------+ +| Name | Description | Param Type | Data type | Max Len | Req’d | Format | Valid/Example Values | ++==========================+=================================+==================+============+===========+=============+================================+===================================================+ +| Topicname | topic name to be posted | Path | String | 40 | Y | . | Org.onap.crm.empdetails | ++--------------------------+---------------------------------+------------------+------------+-----------+-------------+--------------------------------+---------------------------------------------------+ +| `content-type | To specify type of message | Header | String | 20 | N | | application/json text/plain application/cambria | +| | | | | | | | | +| | content(json,text or cambria) | | | | | | | ++--------------------------+---------------------------------+------------------+------------+-----------+-------------+--------------------------------+---------------------------------------------------+ +| Username | userid | Header | String | | N | Basic Authentication Header | | ++--------------------------+---------------------------------+------------------+------------+-----------+-------------+--------------------------------+---------------------------------------------------+ +| Password | | Header | String | | N | Basic Authentication Header | | ++--------------------------+---------------------------------+------------------+------------+-----------+-------------+--------------------------------+---------------------------------------------------+ +| partitionKey | | QueryParam | String | | N | String value | ?partitionKey=123 | ++--------------------------+---------------------------------+------------------+------------+-----------+-------------+--------------------------------+---------------------------------------------------+ + + + + + +Note: +----- + +Publishers/user should have access on the topics. The user (id) and +permissions details needs to be in AAF. + +Response Parameters: +==================== + ++------------------+------------------------+------------+--------------+------------------------------+ +| Name | Description | Type | Format | Valid/Example Values | ++==================+========================+============+==============+==============================+ +| httpStatusCode | | | | 200, 201 etc. | ++------------------+------------------------+------------+--------------+------------------------------+ +| mrErrorCode | Numeric error code | | | 200, 201 etc. | ++------------------+------------------------+------------+--------------+------------------------------+ +| errorMessage | | | | SUCCESS, or error message. | ++------------------+------------------------+------------+--------------+------------------------------+ +| helpURL | helpurl | | | | ++------------------+------------------------+------------+--------------+------------------------------+ +| transactionid | transaction-id value | | | | ++------------------+------------------------+------------+--------------+------------------------------+ + + + +Response /Error Codes: +===================== + ++---------------------------+------------------------------------+ +| Response statusCode | Response statusMessage | ++===========================+====================================+ +| 200-299 | Success | ++---------------------------+------------------------------------+ +| 400-499 | the client request has a problem | ++---------------------------+------------------------------------+ +| 500-599 | the DMaaP service has a problem | ++---------------------------+------------------------------------+ + ++------------------------+----------------+---------------------------------+---------------------------------------------------------------------------------------------------------+ +| Error code | HTTPCode | Description | Issue Reason | ++========================+================+=================================+=========================================================================================================+ +| DMaaP\_MR\_ERR\_3001 | 413 | Request Entity too large | Message size exceeds the batch limit .Reduce the batch size and try again | ++------------------------+----------------+---------------------------------+---------------------------------------------------------------------------------------------------------+ +| DMaaP\_MR\_ERR\_3002 | 500 | Internal Server Error | Unable to publish messages. Please contact administrator | ++------------------------+----------------+---------------------------------+---------------------------------------------------------------------------------------------------------+ +| DMaaP\_MR\_ERR\_3003 | 400 | Bad Request | Incorrect Batching format. Please correct the batching format and try again | ++------------------------+----------------+---------------------------------+---------------------------------------------------------------------------------------------------------+ +| DMaaP\_MR\_ERR\_3004 | 413 | Request Entity too large | Message size exceeds the message size limit .Reduce the message size and try again | ++------------------------+----------------+---------------------------------+---------------------------------------------------------------------------------------------------------+ +| DMaaP\_MR\_ERR\_3005 | 400 | Bad Request | Incorrect JSON object. Please correct the JSON format and try again | ++------------------------+----------------+---------------------------------+---------------------------------------------------------------------------------------------------------+ +| DMaaP\_MR\_ERR\_3006 | 504 | Network Connect Timeout Error | Connection to the DMaaP MR was timed out.Please try again | ++------------------------+----------------+---------------------------------+---------------------------------------------------------------------------------------------------------+ +| DMaaP\_MR\_ERR\_3007 | 500 | Internal Server Error | Failed to publish messages to topic . Successfully published number of messages. | ++------------------------+----------------+---------------------------------+---------------------------------------------------------------------------------------------------------+ +| | 503 | Service Unavailable | Service Unavailable | ++------------------------+----------------+---------------------------------+---------------------------------------------------------------------------------------------------------+ + + + +Sample Request: +============== + ++-----------------------------------------------------------------------------------+ +| POST | +| | +| *Payload-* *{"message":"message description"}* *Content-Type: application/json* | +| | +| Example: | +| | +| curl -u XXXX@abc.com:X -H 'Content-Type:text/plain' -X POST -d @sampleMsg.txt | +| | +| { "count": 1, | +| | +| "serverTimeMs": 19" | +| | +| } | ++-----------------------------------------------------------------------------------+ + +Sample Response: +=============== + ++---------------------------------------------------------------------+ +| HTTP/1.1 200 OK | +| | +| Server: Apache-Coyote/1.1 | +| | +| transactionId: 28-12-2015::08:18:50:682::::28122015552391 | +| | +| Content-Type: application/json | +| | +| Content-Length: 42 | +| | +| Date: Mon, 28 Dec 2015 13:18:50 GMT | ++---------------------------------------------------------------------+ + + + +Subscribers: +=========== + +Description: + + To subscribe to a MessageRouter topic, a subscriber issues a GET to the RESTful HTTP endpoint for events. + + + +Request URL: +============ + +GET http(s)://{HOST:PORT}}/events/{topicname}/{consumegroup}/{consumerid}?{timeout=x} + +Request Parameters: +================== + ++-------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-------------------------------------------------+ +| Name | Description | Param Type | data type | MaxLen | Req’d | Format | Valid/Example Values | +| | | | | | | | | ++-------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-------------------------------------------------+ +| Topicname | topic name to be posted | Path | String | 40 | Y | namespace. | | +| | | | | | | String | | ++-------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-------------------------------------------------+ +|Consumergroup| A name that uniquely identifies | Path | String | | | | | +| | your subscriber's | | | | Y | | CG1 | ++-------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-------------------------------------------------+ +| consumerId | Within your subscriber's group, | | | | | | | +| | a name that uniquely identifies | Path | String | | y | | C1 | +| | your subscriber's process | | | | | | | ++-------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-------------------------------------------------+ +| content-type| To specify type of message | | | | | |aplication/json | +| | content(json,text or cambria) | Header | String | 20 | N | | | +| | | | | | | | | ++-------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-------------------------------------------------+ +|Username | userid | Header | String | 1 | N | | | ++-------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-------------------------------------------------+ +| Password | | Header | String | 1 | N | | | ++-------------+---------------------------------+------------------+------------+--------------+-------------+-------------+-------------------------------------------------+ + +Note1: +====== +Subscribers /user should have access on the topics. The user () and +permissions details needs to be in AAF. + +Note 2: +======= + + +A few consumer client app team who does continuous polling (by +multiple rest calls) complained that they receive "503 consumer lock +exception". The messages will not be lost from the topic when this +exception occurs and it will be still in the topic. This happens when +the concurrent rest call requests are made between several nodes in a +cluster. Consumer lock is done at zookeeper level to ensure the offset +for each consumer group is maintained to avoid losing the message in +multiple concurrent calls of same consumer Group. We are looking +otherways to resolve this issue. But for these continuous polling +scenario, the suggested workaround is to ensure the request is made to +only one node of the cluster. Session stickiness in DME2 is example of +handling this. If this exception occurs , waiting for few minutes with +out making api calls will release the lock in zookeeper.** + +Response Parameters: +=================== + ++------------------+--------------------------------+------------+--------------+-----------------------------------------------------------+ +| Name | Description | Type | Format | Valid/Example Values | ++==================+================================+============+==============+===========================================================+ +| httpStatusCode | | | | 200, 201 etc. | ++------------------+--------------------------------+------------+--------------+-----------------------------------------------------------+ +| mrErrorCode | Numeric error code | | | 200, 201 etc. | ++------------------+--------------------------------+------------+--------------+-----------------------------------------------------------+ +| errorMessage | | | | SUCCESS, or error message. | ++------------------+--------------------------------+------------+--------------+-----------------------------------------------------------+ +| helpURL | helpurl | | | | ++------------------+--------------------------------+------------+--------------+-----------------------------------------------------------+ +| tranactionid | transaction-id value | | | 28-12-2015::08:18:50:682::135.25.227.66::28122015552391 | ++------------------+--------------------------------+------------+--------------+-----------------------------------------------------------+ +| ResponseBody | Messages consumed from topic | Json | Json | | ++------------------+--------------------------------+------------+--------------+-----------------------------------------------------------+ + + + ++---------------------------+------------------------------------+ +| Response statusCode | Response statusMessage | ++===========================+====================================+ +| 200-299 | Success | ++---------------------------+------------------------------------+ +| 400-499 | the client request has a problem | ++---------------------------+------------------------------------+ +| 500-599 | the DMaaP service has a problem | ++---------------------------+------------------------------------+ + + + ++-------------------------+-----------------+----------------------------+----------------------------------------------------------------------------------------------------+ +| Error code | HTTP Code | Description |Issue reason | ++-------------------------+-----------------+----------------------------+----------------------------------------------------------------------------------------------------+ +| DMaaP\_MR\_ERR\_3008 | 413 | Request Entity too large | Message size exceeds the batch limit .Reduce the batch size and try again | +-------------------------+-----------------+----------------------------+----------------------------------------------------------------------------------------------------+ +| DMaaP\_MR\_ERR\_3009 | 500 | Internal Server Error | Unable to publish messages. Please contact administartor | +-------------------------+-----------------+----------------------------+----------------------------------------------------------------------------------------------------+ +| DMaaP\_MR\_ERR\_3010 | 400 | Bad Request | Incorrect Batching format. Please correct the batching format and try again | +-------------------------+-----------------+----------------------------+----------------------------------------------------------------------------------------------------+ +| DMaaP\_MR\_ERR\_3011 | 413 | Request Entity too large | Message size exceeds the message size limit .Reduce the message size and try again | +-------------------------+-----------------+----------------------------+----------------------------------------------------------------------------------------------------+ +| DMaaP\_MR\_ERR\_5012 | 429 | Too many requests | This client is making too many requests. Please use a long poll setting to decrease the number of | | | | | requests that result in empty responses. | ++-------------------------+-----------------+----------------------------+----------------------------------------------------------------------------------------------------+ +| | 503 | Service Unavailable | Service Unavailable | +-------------------------+-----------------+----------------------------+----------------------------------------------------------------------------------------------------+ + + + + +