Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

☂️ Shut down production service #1550

Closed
18 tasks done
Mr0grog opened this issue Jun 9, 2023 · 2 comments · Fixed by #1588
Closed
18 tasks done

☂️ Shut down production service #1550

Mr0grog opened this issue Jun 9, 2023 · 2 comments · Fixed by #1588
Assignees
@Mr0grog
Labels
ops P1

Comments

@Mr0grog
Copy link
Collaborator

Mr0grog commented Jun 9, 2023 •
edited
Loading

On June 15, 2023, well be shutting down the production service, but will be keeping the source code available here, as well as the archived data at archives.getmyvax.org and a placeholder page at getmyvax.org.

What needs to be done to wrap things up?
@Mr0grog Mr0grog self-assigned this Jun 9, 2023
@Mr0grog Mr0grog pinned this issue Jun 9, 2023
Mr0grog added a commit that referenced this issue Jun 13, 2023
@Mr0grog
Archive a clean copy of Terraform configs
3ccf5cb 
For the rest of this week, I'll be planning and implementing a shutdown of our production services, which means making changes to Terraform that someone forking and re-deploying won't want to use. So others have our normal production configurations as a guide, I've made a copy of all the current files in `terraform/deprecated/`.

This is part of #1550.
This was referenced Jun 13, 2023
Merged
Merged
Mr0grog added a commit that referenced this issue Jun 13, 2023
@Mr0grog
Stop loaders before 2023-06-16
da9c14a 
Schedule all loaders so they automatically stop running as of 11pm UTC (4pm Pacific) on June 15.

Part of #1550.
@Mr0grog Mr0grog mentioned this issue Jun 13, 2023
Merged
Mr0grog added a commit that referenced this issue Jun 13, 2023
@Mr0grog
Stop loaders before 2023-06-16 (#1566)
71f9ceb 
Schedule all loaders so they automatically stop running as of 11pm UTC (4pm Pacific) on June 15.

Part of #1550.
Mr0grog added a commit that referenced this issue Jun 15, 2023
@Mr0grog
Add archives documentation site
cece72c 
This adds source for a documentation site to go with archives.getmyvax.org in the `archives/` folder. It's build with MkDocs, which unfortunately makes things slightly messy because it's Python and the rest of this project is JS. But it works fairly well.

Part of #1550.
@Mr0grog Mr0grog mentioned this issue Jun 15, 2023
Merged
Mr0grog added a commit that referenced this issue Jun 16, 2023
@Mr0grog
Shutdown: Replace the API with a tombstone site
12195c7 
This updates DNS to send API server traffic to our new tombstone page (#1569). Once this is done and propagated, I'll start shutting down production services.

Part of #1550.
@Mr0grog Mr0grog mentioned this issue Jun 16, 2023
Merged
Mr0grog added a commit that referenced this issue Jun 16, 2023
@Mr0grog
Shutdown: Replace the API with a tombstone (#1571)
0abeb11 
This updates DNS to send API server traffic to our new tombstone page (#1569). Once this is done and propagated, I'll start shutting down production services.

Part of #1550.
Mr0grog added a commit that referenced this issue Jun 16, 2023
@Mr0grog
Shutdown: Replace API with tombstone, 2nd try
b288bee 
The previous change did not take when Terraform applied it. Hopefully changing the resource name (so we delete the previous record and create a new one) works.

Part of #1550.
@Mr0grog Mr0grog mentioned this issue Jun 16, 2023
Merged
Mr0grog added a commit that referenced this issue Jun 16, 2023
@Mr0grog
Shutdown: Replace API with tombstone, 2nd try (#1572)
95e8905 
The previous change (#1571) did not take when Terraform applied it, and lost the old A record from Terraform's state. I went ahead and deleted the old record in AWS's web console, and this should create a new CNAME.

Part of #1550.
Mr0grog added a commit that referenced this issue Jun 16, 2023
@Mr0grog
Shutdown: Replace API with tombstone, 3rd try
916820f 
Replace apex DNS record with A records for GitHub pages (docs: https://docs.github.com/en/pages/configuring-a-custom-domain-for-your-github-pages-site/managing-a-custom-domain-for-your-github-pages-site). This should fix things after #1571 and #1572.

Part of #1550.
@Mr0grog Mr0grog mentioned this issue Jun 16, 2023
Merged
Mr0grog added a commit that referenced this issue Jun 16, 2023
@Mr0grog
Shutdown: Replace API with tombstone, 3rd try (#1573)
8de25ed 
Replace apex DNS record with A records for GitHub pages (docs: https://docs.github.com/en/pages/configuring-a-custom-domain-for-your-github-pages-site/managing-a-custom-domain-for-your-github-pages-site). This should fix things after #1571 and #1572.

Part of #1550.
Mr0grog added a commit that referenced this issue Jun 16, 2023
@Mr0grog
Shutdown: remove all scheduled jobs
2903121 
The service is no longer operational, so we don't need any more scheduled jobs to run. Part of #1550.
@Mr0grog Mr0grog mentioned this issue Jun 16, 2023
Merged
Mr0grog added a commit that referenced this issue Jun 16, 2023
@Mr0grog
Shutdown: remove all scheduled jobs (#1575)
e907e1b 
The service is no longer operational, so we don't need any more scheduled jobs to run. Part of #1550.
@Mr0grog Mr0grog mentioned this issue Jun 16, 2023
Merged
Mr0grog added a commit that referenced this issue Jun 16, 2023
@Mr0grog
Shutdown: Remove CloudFront and DNS for API (#1576)
58046e7 
The `getmyvax.org` and `www.getmyvax.org` domains no longer point to CloudFront, so we can get rid of it and the old `ecs.` subdomain.

Part of #1550.
Mr0grog added a commit that referenced this issue Jun 16, 2023
@Mr0grog
Shutdown: Stop continuously deploying
d7b5f87 
There are no longer any API resources to deploy code to, so this removes the docker image publishing Terraform deploy steps from our CI workflow.

Part of #1550.
Mr0grog added a commit that referenced this issue Jun 16, 2023
@Mr0grog
Shutdown: Destroy API server
fefd85a 
The API server is no longer reachable, so it can safely no longer exist.

Part of #1550.
@Mr0grog Mr0grog mentioned this issue Jun 16, 2023
Merged
Mr0grog added a commit that referenced this issue Jun 16, 2023
@Mr0grog
Shutdown: Destroy API server (#1577)
9e0f010 
The API server is no longer reachable, so it can safely no longer exist. This also stops auto-deploying in our CI workflow and auto-publishing docker images to ECR.

Part of #1550.
@Mr0grog
Copy link
Collaborator Author

Mr0grog commented Jun 16, 2023

Dumped a final copy of the DB to PG SQL and SQLite and uploaded to S3. Still need to add info to the archives PR: #1570

@Mr0grog
Copy link
Collaborator Author

Mr0grog commented Jun 16, 2023

All the running services in AWS have been torn down, but there are some other lightweight things to get rid of:

  • ECS Cluster (nothing is running it now)
  • Actual database in RDS
  • Bastion
  • Subnets
  • Docker images in ECR

Then I need to clean up the Terraform files.

Finally, there is some more general docs and repo cleanup to do, including finishing out docs for the archives (#1570).

I might do a bit more today, but will probably leave most of that for next week.

Mr0grog added a commit that referenced this issue Jun 16, 2023
@Mr0grog
Shutdown: Add boastful stats to the tombstone
918aa59 
This adds some statistics about overall performance and results of UNIVAF to the tombstone page.

Part of #1550.
@Mr0grog Mr0grog mentioned this issue Jun 16, 2023
Merged
Mr0grog added a commit that referenced this issue Jun 16, 2023
@Mr0grog
Shutdown: Add boastful stats to the tombstone (#1578)
3a28a29 
This adds some statistics about overall performance and results of UNIVAF to the tombstone page.

Part of #1550.
Mr0grog added a commit that referenced this issue Jun 16, 2023
@Mr0grog
Add archives documentation site
cc8ba1b 
This adds source for a documentation site to go with archives.getmyvax.org in the `archives/` folder. It's build with MkDocs, which unfortunately makes things slightly messy because it's Python and the rest of this project is JS. But it works fairly well.

Part of #1550.
Mr0grog added a commit that referenced this issue Jun 16, 2023
@Mr0grog
Add docs site for archives.getmyvax.org (#1570)
31c2e0a 
We’ve needed to have some docs around https://archives.getmyvax.org for a while and never gotten around to it. Now that the API and its accompanying docs are disappearing, This adds some docs that live directly on `archives.getmyvax.org` (so whatever else happens to other sites, the docs stay alongside the archive data).

The docs page got quite long once I’d pasted in all the schemas and sources and ID systems and so on, so I split it up into a few pages and used MkDocs to generate a site from the markdown files.

All the source for the docs site is in the `archives` directory. There’s a workflow to build the site and upload it to the `/docs` folder in the `univaf-data-snapshots` S3 bucket where we serve the archives site from.

Part of #1550.
Mr0grog added a commit that referenced this issue Jun 19, 2023
@Mr0grog
Shutdown: remove ECS cluster
186d8be 
This drops the ECS cluster and ECR image repositories. Since the service has shut down for several days, it's clear we don't need these around to run any remaining tasks or code.

This is part of #1550.
@Mr0grog Mr0grog mentioned this issue Jun 19, 2023
Merged
Mr0grog added a commit that referenced this issue Jun 19, 2023
@Mr0grog
Shutdown: remove ECS cluster (#1581)
d611eeb 
This drops the ECS cluster and ECR image repositories. Since the service has been shut down for several days, it's clear we don't need these around to run any remaining tasks or code.

This is part of #1550.
Mr0grog added a commit that referenced this issue Jun 19, 2023
@Mr0grog
Shutdown: Drop RDS database
66e623b 
We've archived final dumps of the database in S3 at archives.getmyvax.org, and all the services and resources that touch the database been been removed. It's time to drop the database, too.

Part of #1550.
This was referenced Jun 19, 2023
Merged
Merged
Mr0grog added a commit that referenced this issue Jun 19, 2023
@Mr0grog
Shutdown: Drop ECR repositories, take 2
5de6176 
This is a follow-on to #1581 and #1583. The original deletion of our ECR repos didn't take because they weren't empty and we hadn't configured them with force_delete. #1583 should have configured them so deleting them now works.

Part of #1550.
@Mr0grog Mr0grog mentioned this issue Jun 19, 2023
Merged
Mr0grog added a commit that referenced this issue Jun 19, 2023
@Mr0grog
Shutdown: Drop ECR repositories, take 2 (#1584)
e30b276 
This is a follow-on to #1581 and #1583. The original deletion of our ECR repos didn't take because they weren't empty and we hadn't configured them with force_delete. #1583 should have configured them so deleting them now works in this commit.

Part of #1550.
Mr0grog added a commit that referenced this issue Jun 19, 2023
@Mr0grog
Shutdown: Drop RDS database
03eb4d1 
We've archived final dumps of the database in S3 at archives.getmyvax.org, and all the services and resources that touch the database been been removed. It's time to drop the database, too.

Part of #1550.
Mr0grog added a commit that referenced this issue Jun 19, 2023
@Mr0grog
Shutdown: Delete database and bastion (#1582)
d579a97 
We've archived final dumps of the database in S3 at archives.getmyvax.org, and all the services and resources that touch the database been been removed. It's time to drop the database, too. That leaves us without any reason for having the bastion server, so this also removes it.

Part of #1550.
@Mr0grog Mr0grog mentioned this issue Jun 19, 2023
Merged
Mr0grog added a commit that referenced this issue Jun 19, 2023
@Mr0grog
Shutdown: Clean up Terraform configuration (#1585)
cdca825 
This cleans up some of the lingering messy bits in Terraform since we've removed so many resources. It also drops the subnets and VPC while we're at it, since they are no longer needed.

Part of #1550.
@Mr0grog Mr0grog mentioned this issue Jun 19, 2023
Merged
Mr0grog added a commit that referenced this issue Jun 19, 2023
@Mr0grog
Cleanup: Use S3 bucket name vaiable (#1586)
9600399 
Follow-on to #1585. Not sure why we weren't using this variable before. But we're using it now!

Part of #1550.
@Mr0grog Mr0grog mentioned this issue Jun 19, 2023
Merged
Mr0grog added a commit that referenced this issue Jun 19, 2023
@Mr0grog
Shutdown: Add maintenance notice to README (#1588)
c7fd7c9 
Add a notice about the maintenance status of this project to the README. Fixes #1550.
@Mr0grog Mr0grog unpinned this issue Jun 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Mr0grog Mr0grog

Labels
ops P1
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Shutdown: Add maintenance notice to README usdigitalresponse/univaf
1 participant
@Mr0grog