Home

Welcome to the DSDummies wiki!

For this weather application we are using the following technology stack:

1. ReactJS (Frontend)
2. NodeJS (Backend - AuthService)
3. Java Springboot (Backend - CacheService)
4. Python (Backend - Data Ingestor)
5. MongoDB (Database)

Data Flow Diagram :

Napkin Diagram:

User Napkin Diagram:

Steps to install :

Docker

We just need to run the following script to run the entire application :

• Create a new directory in your local machine
• Open a terminal in the folder created above and run "git clone https://github.com/airavata-courses/DSDummies.git"
• Type cd DSDummies (only folder in the directory)
• Run the individual commands as listed or run the script inside the DSDummies directory.

Directly execute script:

• chmod 755 start.sh

• ./start.sh

Alternatively, if not using script or it doesn't runs for you, the following commands would work for deploying the application on docker as well:

• cd Client ( You need to go into the "Client" directory )

  • make build

• cd ../Auth. ( You need to go into the "Auth" directory )

  • make build

• cd ../cache. ( You need to go into the "cache" directory )

  • make build

• cd .. ( You need to go into the "Data\Ingestor" directory )

  • cd Data\ Ingestor
  • make build

• cd ..

• make run-dev (From root directory)

1. This shall spawn all the containers and then you would be able to open localhost://3000
2. This shall open up a login page, if you're a first time user , then click on register.
3. Once you register, go to login and put in your credentials.
4. It shall open a US map with drop down boxes for data and time as well as specifying video option.
5. Put in the radar station name, date and time, choose if you want to select a video or not, then click on getplot.
6. This will fetch you the result from the aws and show a plotted graph of the received data.

Note : Step 6 doesn't works yet on Docker, but the complete functionality works end to end perfectly on local, we're in the process of fixing it, but until that time, for grading , please refer to the local setup.

UPDATE: Complete dockerization of the application works and the above mentioned steps should be able to deploy our code on the docker and run in it. This is fixed by Issue #50 commit

Project 3 details

It is deployed on Jetstream. The link to access it :

                                       http://149.165.156.154:30500/

We have deployed our microservices with Kubernetes on jetstream, with each service running multiple instances. It includes both Merra and existing implementation of the Nexrad dataset.

We have used kubeadm to deploy it on Jetstream forming a k8 cluster, the database connection doesn't work on the exosphere, so we are going with deploying it on atmosphere with the same code base . We have used Openstack to spawn instances of the Master and worker nodes on the exosphere.

The branch for these changes is : develop-MERRA2-visualization

We have also implemented the RabbitMq changes on our system but we are facing issues to deploy it on jet stream. The branch for rabbitmq changes is: develop-rabbitmq To test the messaging queue, local testing can be done. For testing locally, all the services have to be up.

Navigate to each of the folders and run the commands in order:

Frontend (reactjs):

• npm install
• npm start

Api Gateway (nodejs):

• npm install
• node app.js

Auth Service (nodejs)

• npm install
• npm start

Cache Service (Springboot):

• Run as java application using eclipse

Data Ingestor (python):

• cd Data\Ingestor/
• conda env create -f environment.yml (setting up virtual env)
• conda activate nexrad-env (activating env)
• pip install -r requirements.txt
• conda run -n nexrad-env --no-capture-output python3 main.py (starting flask server)

Completed items:

• Merra data visualisation
• CI/CD [Refer branches: Auth-ci-pipeline, cache-ci-pipeline, client-ci-pipeline, data-ingestor-ci-pipeline]
• Jetstream deployment [Refer link: http://149.165.156.154:30500/]
• Implemented rabbitmq