1 ## MUSIC - Multi-site State Coordination Service
3 To truly achieve 5 9s of availability on 3 9s or lower software and infrastructure in a cost-effective manner, ONAP components need to work in a reliable, active-active manner across multiple sites (platform-maturity resiliency level 3). A fundamental aspect of this is state management across geo-distributed sites in a reliable, scalable, highly available and efficient manner. This is an important and challenging problem because of three fundamental reasons:
5 * Current solutions for state-management of ONAP components like MariaDB clustering, that work very effectively within a site, may not scale across geo-distributed sites (e.g., Beijing, Amsterdam and Irvine) or allow partitioned operation (thereby compromising availability). This is mainly because WAN latencies are much higher across sites and frequent network partitions can occur.
7 * ONAP components often have a diverse range of requirements in terms of state replication. While some components need to synchronously manage state across replicas, others may tolerate asynchronous replication. This diversity needs to be leveraged to provide better performance and higher availability across sites.
9 * ONAP components often need to partition state across different replicas, perform consistent operations on them and ensure that on failover, the new owner has access to the latest state. The distributed protocols to achieve such consistent ownership is complex and replete with corners cases, especially in the face of network partitions. Currently, each component is building its own handcrafted solution which is wasteful and worse, can be erroneous.
11 In this project, we identify common state management concerns across ONAP components and provide a multi-site state coordination/management service (MUSIC) with a rich suite of recipes that each ONAP component can simply configure and use for their state-management needs.
15 [Local Installation](#local-install)
17 [Multi-site Installation](#ms-install)
19 [Muting MUSIC jersey output](#jersey-mute)
21 [Authentications/AAF Setup](#auth-setup)
26 MUSIC is to be installed in a single Dir.
29 #The main MUSIC dir should be
31 # These also need to be set up
35 When installing Cassandra it should also be installed here.
37 /opt/app/music/apache-cassandra-n.n.n
39 Its suggested you create links from install dirs to a common name ie:
41 ln -s /opt/app/music/apache-cassandra-n.n.n cassandra
44 Cassandra has data dirs.
46 # For cassandra it should be (This is the default)
47 /opt/app/music/cassandra/data
50 If you are using a VM make sure it has at least 8 GB of RAM (It may work with 4 GB, but with 2 GB it
53 <a name="local-install">
61 - Create MUSIC Install dir /opt/app/music
62 - Open /etc/hosts as sudo and enter the name of the vm alongside localhost in the line for 127.0.0.1. E.g. 127.0.0.1 localhost music-1. Some of the apt-get installation seem to require this.
63 - Ensure you have java jdk 8 or above working on your machine.
64 - Download apache Apache Cassandra 3.2, install into /opt/app/music and follow these instructions http://cassandra.apache.org/doc/latest/getting_started/installing.html till and including Step
65 - By the end of this you should have Cassandra working.
66 - Create a music.properties file and place it in /opt/app/music/etc/. Here is a sample of the file:
71 my.public.ip=localhost
72 all.public.ips=localhost
73 #######################################
74 # Optional current values are defaults
75 #######################################
76 #cassandra.host=localhost
79 #music.rest.ip=localhost
80 #lock.lease.period=6000
81 cassandra.user=cassandra
82 cassandra.password=cassandra
83 aaf.endpoint.url=http://aafendpoint/proxy/authz/nss/
86 - Make a dir /opt/app/music/logs MUSIC dir with MUSIC logs will be created in this dir after MUSIC starts.
87 - Authentications/AAF Setup For Authentication setup.
88 - Start MUSIC using the jar built in distribution/music folder
94 Extra Cassandra information for Authentication:
96 To create first user in Cassandra
98 Edit conf/Cassandra.yaml file
101 authenticator: PasswordAuthenticator
102 authorizer: CassandraAuthorizer
105 Login to cqlsh with default credentials
108 cqlsh -u cassandra -p cassandra
111 To change default user create new user with the following command.
114 CREATE USER new_user WITH PASSWORD ‘new_password’ SUPERUSER;
116 Change password for default user ‘Cassandra’ so that no one will be able to login
119 ALTER USER cassandra WITH PASSWORD ‘SomeLongRandomStringNoOneWillThinkOf’;
121 Provide the new user credentials to Music. Update music.properties file and uncomment or add the following:
124 cassandra.user=<new_user>
125 cassandra.password=<new_password>
128 To access keyspace through cqlsh, login with credentials that are passed to MUSIC while creating the keyspace.
132 <a name ="ms-install">
134 ## Multi-site Installation
138 Follow the instructions for local MUSIC installation on all the machines/VMs/hosts (referred to as a node) on which you want MUSIC installed. However, Cassandra and Zookeeper needs to be configured to run as multi-node installations (instructions below) before running them.
141 In the cassandra.yaml file which is present in the cassa_install/conf directory in each node, set the following parameters:
144 cluster_name: ‘name of cluster’
147 class_name: org.apache.cassandra.locator.SimpleSeedProvider
149 seeds: "<public ip of first seed>, <public ip of second seed>, etc"
150 listen_address: private ip of VM
151 broadcast_address: public ip of VM
152 endpoint_snitch: GossipingPropertyFileSnitch
153 rpc_address: <private ip>
154 phi_convict_threshold: 12
155 authenticator: PasswordAuthenticator
156 authorizer: CassandraAuthorizer
160 In the cassandra-rackdc.properties file, assign data center and rack names as if required.
162 Once this is done on all three nodes, you can run cassandra on each of the nodes through the cassandra bin folder with this command
166 In the cassandra bin folder, if you run the following it will tell you the state of the cluster.
170 To access cassandra, one any of the nodes you can run the following and then perform CQL queries.
175 To create first user in Cassandra
178 Login to cqlsh with default credentials
181 cqlsh -u cassandra -p cassandra
184 To change default user create new user with the following command.
187 CREATE USER new_user WITH PASSWORD ‘new_password’ SUPERUSER;
189 Change password for default user ‘Cassandra’ so that no one will be able to login
192 ALTER USER cassandra WITH PASSWORD ‘SomeLongRandomStringNoOneWillThinkOf’;
194 Provide the new user credentials to Music. Update music.properties file and uncomment or add the following:
197 cassandra.user=<new_user>
198 cassandra.password=<new_password>
201 To access keyspace through cqlsh, login with credentials that are passed to MUSIC while creating the keyspace.
205 Create a music.properties file and place it in /opt/app/music/etc at each node. Here is a sample of the file: If this location is to be changed please update the file project.properties in the src/main/resources directory before compiling MUSIC to a war.
209 my.public.ip=public IP of node 0
210 #For each node, a separate file needs to be created with its own id (between 0 and the number of nodes) and with information about its own public ip.
211 all.public.ips=public IP of node 0:public IP of node 1:public IP of node 2
212 #######################################
213 # Optional current values are defaults
214 #######################################
215 #cassandra.host=localhost
218 #music.rest.ip=localhost
219 #lock.lease.period=6000
220 cassandra.user=cassandra
221 cassandra.password=cassandra
222 aaf.endpoint.url=http://aafendpoint/proxy/authz/nss/
225 - Download the latest Apache Tomcat and install it using these instructions http://tecadmin.net/install-tomcat-9-on-ubuntu/ (this is for version 9).
226 - Build the MUSIC.war and place it within the webapps folder of the tomcat installation.
227 - Start tomcat and you should now have MUSIC running.
229 - For Logging create a dir /opt/app/music/logs. When MUSIC/Tomcat starts a dir opt/app/music/logsMUSIC with various logs will be created.
230 The application log is music.log and the others are debug.log and error.log.
232 <a name="jersey-mute">
234 ### Muting MUSIC jersey output
236 The jersey package that MUSIC uses to parse REST calls prints out the entire header and json body by
237 default. To mute it (if it exists), remove the following lines from the web.xml in the WEB_INF foler:
241 <param-name>com.sun.jersey.spi.container.ContainerResponseFilters</param-name>
242 <param-value>com.sun.jersey.api.container.filter.LoggingFilter</param-value>
246 <a name ="auth-setup">
248 ### Authentication Setup
250 MUSIC has been enhanced to support applications which are already authenticated using AAF and applications which are not authenticated using AAF.
251 If an application has already been using AAF, it should have required namespace, mechId and password. Non AAF applications will be provided with a random unique id. MUSIC will tag the keyspace with the UID and store internally and any calls to use/modify the keyspace will require the UID.
252 All the required params should be sent as headers.
254 - MUSIC has been enhanced to support applications which are already authenticated using AAF and applications which are not authenticated using AAF.
255 - If an application has already been using AAF, it should have required namespace, mechId and password.
256 - Non AAF applications (AID) will be provided with a random unique id. MUSIC will tag the keyspace with the UID and store internally and any calls to use/modify the keyspace will require the UID.
258 All the required params should be sent as headers.
260 Admin needs to create the following keyspace and table via cslsh.
262 In the cassandra bin dir run ./cqlsh -u <user> -p <password> and log in to db then:
264 If you want to save the following in a file you can then run ./cqlsh -f <file.cql>
267 //Create Admin Keyspace
269 CREATE KEYSPACE admin
271 'class' : 'SimpleStrategy',
272 'replication_factor': 1
274 AND DURABLE_WRITES = true;
276 CREATE TABLE admin.keyspace_master (
279 application_name text,
289 //Create Admin Keyspace
291 CREATE KEYSPACE admin
293 'class' : 'NetworkTopologyStrategy',
298 AND DURABLE_WRITES = true;
300 CREATE TABLE admin.keyspace_master (
303 application_name text,
313 - For AAF applications all the 3 of the following headers are mandatory.
317 - For Non AAF applications if the header <b>aid</b> is not provided MUSIC creates new random unique UUID and returns to caller.
318 - Caller application then need to save the UUID and need to pass the UUID to further modify/access the keyspace.
319 - Make sure steps to set up Cassandra for authentication above in the cassandra.yaml were done.
323 The following API is used to onboard the applications.
324 /MUSIC/rest/v2/admin/onboardAppWithMusic
325 This is a POST call and requires the following JSON:
328 "appname": "<the Namespace for aaf or the Identifier for the specific app using AID access>",
329 "userId" : "<userid>",