Improve the performance of resoures microservice 32/74232/3 1.4.0
authorKajur, Harish (vk250x) <vk250x@att.com>
Tue, 4 Dec 2018 19:06:14 +0000 (14:06 -0500)
committerKajur, Harish (vk250x) <vk250x@att.com>
Wed, 5 Dec 2018 14:38:36 +0000 (09:38 -0500)
commit8fb7aa6480d4d7becbbab5fcfabd6af3f57e7d74
treee6cb6aa0e6839100a06a787c8ff3034aac11dbfb
parent7bc1735a516e56d837aa5af3d4849cdcb7af26c7
Improve the performance of resoures microservice

Currently the findSubGraph is being invoked
and causing additional calls to the database by backing
a tree backed vertex and slowing the performance of GET ALLs
Original intention of both of them was to pre fetch depth 0, 1, and 2
at once so we don't have to get them at each depth but since
that was not done right there is extra amount of time

After the aai-uri migration to ensure all vertexes
have the aai-uri and also to make sure they are unique across graph
we don't need to traverse a vertex to find the parents to build aai-uri
it was done previously when there was no aai-uri to derive the uri of a
given vertex and its not necessary anymore so there would be performance
improvements of vserver when there are a lot of relationships

Currently the edge labels are retrieved for each vertex a
and b and the performance of the GET with relationships will be
propotional to the GET request and how many cousin edges it has
and the more cousin edges there are the more slower the response time
will be as for each cousin vertex its trying to get the edge in between
so the code is modified to only go to the database for the edge label
when there are multiple edge labels (cousin edges) between a and b
If there are only one edge label and its a cousin vertex then we
can use the edge rule to be able to figure out the edge label

Improve PUT on the cloud region by modifying the getEdgesBetween
method which currently was retrieving all the parent
child edges between a and b and then only using the first edge
The traversal itself was too complicated and was costly in terms of
database retrieval and calls and optimized the code to utilize the
edge information so we can modify the query to db at runtime
based on the information provided so when a 10000 vservers under a
tenant adding a new vserver under tenant would be slow because of the
old query performance as it was taking some time there but with this
optimization, its only spending at most a millisecond or 2 in that
method

Also noticed that when a PUT operation takes place, the method
calls the related objects to create a dmaap event which was in turn
calling the findParents and actually utilizing the parents to create the
dmaap event and would spend quite a lot of time here because of the
expensive call of the findParents in this case we need those vertexes
So optimized the code so based on a given vertex, we have the aai-uri
and the newly added metadata uriTemplate to break the aai-uri into
its parent aai-uri and grand parent aai-uri and so forth
With this breakdown, we can get the list of aai-uris which are parents,
grandparents and then use the aai-uri to look them up which is O(1)
lookup time due to the fact they are unique indexes
The time it takes when doing a traversal to find the parents is
propotional to the number of edges but this will be optimistic

Another area which was improved was the json path execution of the
edge rules so when the edge rules get loaded into memory it creates a
document object, it utilizes the jsonpath to query information about
the edge rules but the only thing here is each time it gets called
the query gets invoked and uses jsonpath to retrieve the edge rules when
we can cached them based on the filter so that the user executed and if
the filter is the same as before, the expected edge rules will return
the same
Too much time was spent making the query and retrieving and building the
edgerules
So a cache is a perfect way to optimize this part

Issue-ID: AAI-1987
Change-Id: Ieb8237de3fd31136ceac14bf4a8216a7ab3b7179
Signed-off-by: Kajur, Harish (vk250x) <vk250x@att.com>
34 files changed:
aai-core/pom.xml.versionsBackup [new file with mode: 0644]
aai-core/src/main/java/org/onap/aai/config/IntrospectionConfig.java
aai-core/src/main/java/org/onap/aai/introspection/Introspector.java
aai-core/src/main/java/org/onap/aai/introspection/JSONStrategy.java
aai-core/src/main/java/org/onap/aai/introspection/Loader.java
aai-core/src/main/java/org/onap/aai/introspection/MoxyLoader.java
aai-core/src/main/java/org/onap/aai/introspection/MoxyStrategy.java
aai-core/src/main/java/org/onap/aai/rest/db/HttpEntry.java
aai-core/src/main/java/org/onap/aai/serialization/db/DBSerializer.java
aai-core/src/main/java/org/onap/aai/serialization/engines/query/GraphTraversalQueryEngine.java
aai-core/src/main/java/org/onap/aai/serialization/engines/query/QueryEngine.java
aai-core/src/test/java/org/onap/aai/HttpTestUtil.java
aai-core/src/test/java/org/onap/aai/parsers/uri/URIToObjectTest.java
aai-core/src/test/java/org/onap/aai/rest/EdgeNotValidAnymoreTest.java [new file with mode: 0644]
aai-core/src/test/java/org/onap/aai/rest/EntitlementTest.java [new file with mode: 0644]
aai-core/src/test/java/org/onap/aai/rest/ModelElementTest.java [new file with mode: 0644]
aai-core/src/test/java/org/onap/aai/rest/PserverDuplicateTest.java [new file with mode: 0644]
aai-core/src/test/java/org/onap/aai/rest/VipAddressListTest.java [new file with mode: 0644]
aai-core/src/test/java/org/onap/aai/serialization/db/DbSerializerTest.java
aai-core/src/test/java/org/onap/aai/serialization/engines/query/GraphTraversalQueryEngineTest.java
aai-core/src/test/resources/payloads/resource/cloud-region-with-vlan.json [new file with mode: 0644]
aai-core/src/test/resources/payloads/resource/cloud-region.json [new file with mode: 0644]
aai-core/src/test/resources/payloads/resource/entitlement.json [new file with mode: 0644]
aai-core/src/test/resources/payloads/resource/model-element.json [new file with mode: 0644]
aai-core/src/test/resources/payloads/resource/model.json [new file with mode: 0644]
aai-core/src/test/resources/payloads/resource/vip-ipv4-address-list.json [new file with mode: 0644]
aai-core/src/test/resources/payloads/resource/vnf.json [new file with mode: 0644]
aai-schema-ingest/src/main/java/org/onap/aai/edges/EdgeIngestor.java
aai-schema-ingest/src/main/java/org/onap/aai/edges/EdgeRule.java
aai-schema-ingest/src/main/java/org/onap/aai/edges/EdgeRuleQuery.java
aai-schema-ingest/src/main/java/org/onap/aai/edges/SchemaFilter.java [new file with mode: 0644]
aai-schema-ingest/src/main/java/org/onap/aai/edges/exceptions/AmbiguousRuleChoiceException.java
aai-schema-ingest/src/main/java/org/onap/aai/edges/exceptions/EdgeRuleNotFoundException.java
aai-schema/src/main/resources/onap/oxm/v15/aai_oxm_v15.xml