Skip to content

Commit

Permalink
Migrate from Planetscale to Neon (Part 1) (#267)
Browse files Browse the repository at this point in the history
* Update drizzle

* Most part of it

* Add prepared statements names'

* Remove insertId

* captureDate is timestamp

* Add missing hashedPassword

* Fix CustomAdapter type

* better error

* Merge drizzle files

* Fix amountOfPeopleEnum

* WTF

* Nulls last

* Fix with translations

* type placeholders

* SKIP_ENV_VALIDATION

* remove random hola

* update migration ci

* Update readme

* more rm

* uppercasse

* cfjghvj

* argvfd

* ef
  • Loading branch information
mauriciabad authored Apr 7, 2024
1 parent c28c45b commit 9e4fd2d
Show file tree
Hide file tree
Showing 106 changed files with 2,400 additions and 28,015 deletions.
10 changes: 3 additions & 7 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -8,22 +8,17 @@ GOOGLE_CLIENT_SECRET=toBeChanged
BASE_URL=http://localhost:3000
NEXTAUTH_URL=http://localhost:3000

DATABASE_HOST=toBeChanged
DATABASE_NAME=toBeChanged

# Prod
# NEXT_PUBLIC_ENV_LABEL=prod
# DATABASE_USERNAME=toBeChanged
# DATABASE_PASSWORD=toBeChanged
# DATABASE_URL=toBeChanged
# AWS_ACCESS_KEY_ID=toBeChanged
# AWS_SECRET_ACCESS_KEY=toBeChanged
# NEXT_PUBLIC_AWS_BUCKET_REGION=toBeChanged
# NEXT_PUBLIC_AWS_BUCKET_NAME=toBeChanged

# Stage
NEXT_PUBLIC_ENV_LABEL=stage
DATABASE_USERNAME=toBeChanged
DATABASE_PASSWORD=toBeChanged
DATABASE_URL=toBeChanged
AWS_ACCESS_KEY_ID=toBeChanged
AWS_SECRET_ACCESS_KEY=toBeChanged
NEXT_PUBLIC_AWS_BUCKET_REGION=toBeChanged
Expand All @@ -32,3 +27,4 @@ NEXT_PUBLIC_AWS_BUCKET_NAME=toBeChanged
# Local
# NEXT_PUBLIC_ENV_LABEL=local
# USE_LOCAL_DB=true
# DATABASE_URL=toBeChanged
9 changes: 2 additions & 7 deletions .github/workflows/comment-if-db-migrations.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,7 @@ jobs:
> [!NOTE]
> This PR introduces changes to the database.
**Remember to:**
**Remember to migrate the DB**
1. Run the migrations in [stage database branch](https://app.planetscale.com/begursecret/descobreix-begur-app/stage) with:
```bash
pnpm db:push
```
1. Create a [deploy request in Planetscale](https://app.planetscale.com/begursecret/descobreix-begur-app/deploy-requests/)
1. Merge the deploy request from Planetscale just before merging the PR.
For now it is not possible to run the migrations automatically, so you need to do it manually.
comment_tag: execution
24 changes: 22 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,10 @@ Discover incredible places and explore Begur like never before.
#### Using local database
By default, the app will connect to stage database in PlanetScale, but you can run it locally too.
> [!CAUTION]
> This guide is outdated! The migration to Neon was very fast and I didn't have time to standarize the procedure.

By default, the app will connect to stage database in Neon, but you can run it locally too.

1. Install [docker](https://www.docker.com/products/docker-desktop/).
1. Run the database **in another console**:
Expand Down Expand Up @@ -79,6 +82,9 @@ By default, the app will connect to stage database in PlanetScale, but you can r

#### How to deploy changes in the database

> [!CAUTION]
> This guide is outdated! The migration to Neon was very fast and I didn't have time to standarize the procedure.
1. Make the changes in your code, testing them with the local database.
1. Run `pnpm db:generate` to generate the migrations.
1. Run `pnpm db:push` to upload the migrations to PlanetScale stage branch.
Expand All @@ -88,13 +94,27 @@ By default, the app will connect to stage database in PlanetScale, but you can r
1. Right before merging the PR, go to PlanetScale dashboard and create a deploy preview and deploy it.
1. Merge the PR in GitHub.
##### New procedure random thoughts
1. Run `pnpm db:generate`, to generate the new sql file.
2. DON'T run `pnpm db:push`, because it won't work. Instead, go to the Neon website, and in a SQL console paste the generated SQL.
Besides that. These are the manual changes I did to the migration files:
1. Add `CREATE EXTENSION IF NOT EXISTS postgis;` to the top
2. Remove the double quotes (`"`) from `Geometry(Point, 25831)` and `Geometry(MultiLineString, 25831)`.
Something else:
The script `db:local:run-db` is not done, it is commented out.
#### Access to other services
Ask @mauriciabad for access.
These are the services we use related to the code:
- [PlanetScale](https://app.planetscale.com/begursecret/descobreix-begur-app)
- [Neon](https://console.neon.tech/app/projects/spring-brook-65910694)
- [AWS S3](https://aws.amazon.com/s3)
- [Vercel](https://vercel.com/mauriciabad/begursecret)
- [GitHub](https://github.com/mauriciabad/begursecret)
Expand Down
31 changes: 7 additions & 24 deletions drizzle.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,33 +5,16 @@ import path from 'node:path'
dotenv.config({ path: path.resolve(process.cwd(), '.env.local') })
dotenv.config({ path: path.resolve(process.cwd(), '.env') })

if (
process.env.USE_LOCAL_DB !== 'true' &&
process.env.SKIP_ENV_VALIDATION !== 'true'
) {
if (!process.env.DATABASE_HOST)
throw new Error('Missing environment variable DATABASE_HOST')
if (!process.env.DATABASE_NAME)
throw new Error('Missing environment variable DATABASE_NAME')
if (!process.env.DATABASE_USERNAME)
throw new Error('Missing environment variable DATABASE_USERNAME')
if (!process.env.DATABASE_PASSWORD)
throw new Error('Missing environment variable DATABASE_PASSWORD')
if (process.env.SKIP_ENV_VALIDATION !== 'true') {
if (!process.env.DATABASE_URL)
throw new Error('Missing environment variable DATABASE_URL')
}

export default {
schema: './src/server/db/schema/*',
out: './drizzle',
driver: 'mysql2',
dbCredentials:
process.env.USE_LOCAL_DB === 'true'
? {
host: '127.0.0.1',
user: 'root',
password: 'unsafePaswordOnlyForLocalhost',
database: 'descobreix-begur-app',
}
: {
uri: `mysql://${process.env.DATABASE_USERNAME}:${process.env.DATABASE_PASSWORD}@${process.env.DATABASE_HOST}/${process.env.DATABASE_NAME}?ssl={"rejectUnauthorized":true}`,
},
driver: 'pg',
dbCredentials: {
connectionString: process.env.DATABASE_URL!,
},
} satisfies Config
43 changes: 0 additions & 43 deletions drizzle/0000_mute_harrier.sql

This file was deleted.

217 changes: 217 additions & 0 deletions drizzle/0000_previous_ozymandias.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,217 @@
CREATE EXTENSION IF NOT EXISTS postgis;
--> statement-breakpoint
DO $$
BEGIN
CREATE TYPE "allowedAccess" AS ENUM(
'public', 'permissive', 'customers', 'permit', 'private', 'mixed'
);
EXCEPTION WHEN duplicate_object THEN null;
END
$$;
--> statement-breakpoint
DO $$
BEGIN
CREATE TYPE "amountOfPeople" AS ENUM(
'none', 'few', 'some', 'many', 'crowded'
);
EXCEPTION WHEN duplicate_object THEN null;
END
$$;
--> statement-breakpoint
DO $$
BEGIN
CREATE TYPE "difficulty" AS ENUM(
'accessible', 'normal', 'smallEffort', 'hard', 'dangerous'
);
EXCEPTION WHEN duplicate_object THEN null;
END
$$;
--> statement-breakpoint
DO $$
BEGIN
CREATE TYPE "groundType" AS ENUM(
'sand', 'pebbles', 'rocks', 'concrete', 'dirt', 'pavimented'
);
EXCEPTION WHEN duplicate_object THEN null;
END
$$;
--> statement-breakpoint
DO $$
BEGIN
CREATE TYPE "howNarrow" AS ENUM(
'extremlyNarrow', 'narrow', 'extraSpace', 'wide', 'veryWide'
);
EXCEPTION WHEN duplicate_object THEN null;
END
$$;
--> statement-breakpoint
DO $$
BEGIN
CREATE TYPE "placeToArriveFrom" AS ENUM(
'townCenter', 'parking', 'beach', 'road'
);
EXCEPTION WHEN duplicate_object THEN null;
END
$$;
--> statement-breakpoint
DO $$
BEGIN
CREATE TYPE "priceUnit" AS ENUM(
'eur', 'eur/minute', 'eur/hour', 'eur/day'
);
EXCEPTION WHEN duplicate_object THEN null;
END
$$;
--> statement-breakpoint
DO $$
BEGIN
CREATE TYPE "scubaDivingLevel" AS ENUM(
'discoverScubaDiving', 'openWater', 'advancedOpenWater', 'specialtyDiver', 'technicalDiver'
);
EXCEPTION WHEN duplicate_object THEN null;
END
$$;
--> statement-breakpoint
DO $$
BEGIN
CREATE TYPE "trainingLevel" AS ENUM(
'noTraining', 'amateur', 'entryLevel', 'advanced', 'professional', 'elite'
);
EXCEPTION WHEN duplicate_object THEN null;
END
$$;
--> statement-breakpoint
DO $$
BEGIN
CREATE TYPE "gender" AS ENUM('masculine', 'feminine');
EXCEPTION WHEN duplicate_object THEN null;
END
$$;
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "account" (
"userId" text NOT NULL, "type" text NOT NULL, "provider" text NOT NULL, "providerAccountId" text NOT NULL, "refresh_token" text, "access_token" text, "expires_at" integer, "token_type" text, "scope" text, "id_token" text, "session_state" text, CONSTRAINT "account_provider_providerAccountId_pk" PRIMARY KEY (
"provider", "providerAccountId"
)
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "session" (
"sessionToken" text PRIMARY KEY NOT NULL, "userId" text NOT NULL, "expires" timestamp NOT NULL
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "verificationToken" (
"identifier" text NOT NULL, "token" text NOT NULL, "expires" timestamp NOT NULL, CONSTRAINT "verificationToken_identifier_token_pk" PRIMARY KEY ("identifier", "token")
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "externalLink" (
"id" serial PRIMARY KEY NOT NULL, "placeId" integer, "routeId" integer, "isOfficialWebsite" boolean, "url" text NOT NULL, "title" text
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "externalLink_translation" (
"id" serial PRIMARY KEY NOT NULL, "externalLink_id" integer NOT NULL, "locale" varchar(10) NOT NULL, "url" text NOT NULL, "title" text
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "feature" (
"id" serial PRIMARY KEY NOT NULL, "amountOfPeople" "amountOfPeople", "difficulty" "difficulty", "groundType" "groundType", "hasBus" boolean, "hasParking" boolean, "parkingSpaces" integer, "hasToilet" boolean, "hasRestaurant" boolean, "hasDrinkingWater" boolean, "hasShower" boolean, "hasLifeguard" boolean, "hasLeisure" boolean, "isNudist" boolean, "hasUnofficialName" boolean, "hasInacurateLocation" boolean, "date" text, "isBoatOnly" boolean, "trainingLevel" "trainingLevel", "hasMissingInfo" boolean, "height" integer, "depth" integer, "depthMin" integer, "depthMax" integer, "scubaDivingLevel" "scubaDivingLevel", "notThereAnymore" boolean, "isOutOfTheMunicipality" boolean, "hasBench" boolean, "allowedAccess" "allowedAccess", "dimensions" text, "price" double precision, "priceUnit" "priceUnit", "isCovered" boolean, "duration" integer, "distance" integer, "slope" integer, "timeToArrive" integer, "placeToArriveFrom" "placeToArriveFrom", "isFreeWithLocalStamp" boolean, "howNarrow" "howNarrow", "difficultyNotes" text, "priceNotes" text, "allowedAccessNotes" text, "hasInacurateLocationNotes" text, "hasMissingInfoNotes" text, "notThereAnymoreNotes" text
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "feature_translation" (
"id" serial PRIMARY KEY NOT NULL, "feature_id" integer NOT NULL, "locale" varchar(10) NOT NULL, "difficultyNotes" text, "priceNotes" text, "allowedAccessNotes" text, "hasInacurateLocationNotes" text, "hasMissingInfoNotes" text, "notThereAnymoreNotes" text
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "image" (
"id" serial PRIMARY KEY NOT NULL, "key" varchar(1024) NOT NULL, "width" integer NOT NULL, "height" integer NOT NULL, "source" text, "captureDate" timestamp, "alt" text, "blurDataURL" text
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "placeCategory" (
"id" serial PRIMARY KEY NOT NULL, "icon" text NOT NULL, "color" text NOT NULL, "hasVisitMission" boolean DEFAULT true NOT NULL, "order" integer, "name" text NOT NULL, "namePlural" text NOT NULL, "nameGender" "gender"
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "placeCategory_translation" (
"id" serial PRIMARY KEY NOT NULL, "placeCategory_id" integer NOT NULL, "locale" varchar(10) NOT NULL, "name" text NOT NULL, "namePlural" text NOT NULL, "nameGender" "gender"
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "placeToPlaceCategory" (
"placeId" integer NOT NULL, "categoryId" integer NOT NULL, CONSTRAINT "placeToPlaceCategory_categoryId_placeId_pk" PRIMARY KEY ("categoryId", "placeId")
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "placeCategoryToPlaceCategoryGroup" (
"categoryGroupId" integer NOT NULL, "categoryId" integer NOT NULL, "highlight" boolean, CONSTRAINT "placeCategoryToPlaceCategoryGroup_categoryGroupId_categoryId_pk" PRIMARY KEY (
"categoryGroupId", "categoryId"
)
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "placeCategoryGroup" (
"id" serial PRIMARY KEY NOT NULL, "icon" text NOT NULL, "color" text NOT NULL, "order" integer, "name" text NOT NULL, "nameGender" "gender"
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "placeCategoryGroup_translation" (
"id" serial PRIMARY KEY NOT NULL, "placeCategoryGroup_id" integer NOT NULL, "locale" varchar(10) NOT NULL, "name" text NOT NULL, "nameGender" "gender"
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "placeListToPlace" (
"placeListId" integer NOT NULL, "placeId" integer NOT NULL, "addedAt" timestamp DEFAULT CURRENT_TIMESTAMP NOT NULL, CONSTRAINT "placeListToPlace_placeId_placeListId_pk" PRIMARY KEY ("placeId", "placeListId")
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "placeList" (
"id" serial PRIMARY KEY NOT NULL, "userId" varchar(255) NOT NULL
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "place" (
"id" serial PRIMARY KEY NOT NULL, "mainImageId" integer, "googleMapsId" text, "location" Geometry (Point, 25831) NOT NULL, "mainCategoryId" integer NOT NULL, "featuresId" integer NOT NULL, "verificationRequirementsId" integer, "importance" double precision, "name" text NOT NULL, "description" text, "content" text
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "place_translation" (
"id" serial PRIMARY KEY NOT NULL, "place_id" integer NOT NULL, "locale" varchar(10) NOT NULL, "name" text NOT NULL, "description" text, "content" text
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "routeCategory" (
"id" serial PRIMARY KEY NOT NULL, "icon" text NOT NULL, "color" text NOT NULL, "hasVisitMission" boolean DEFAULT true NOT NULL, "order" integer, "name" text NOT NULL, "namePlural" text NOT NULL, "nameGender" "gender"
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "routeCategory_translation" (
"id" serial PRIMARY KEY NOT NULL, "routeCategory_id" integer NOT NULL, "locale" varchar(10) NOT NULL, "name" text NOT NULL, "namePlural" text NOT NULL, "nameGender" "gender"
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "routeToRouteCategory" (
"routeId" integer NOT NULL, "categoryId" integer NOT NULL, CONSTRAINT "routeToRouteCategory_categoryId_routeId_pk" PRIMARY KEY ("categoryId", "routeId")
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "route" (
"id" serial PRIMARY KEY NOT NULL, "mainImageId" integer, "path" Geometry (MultiLineString, 25831) NOT NULL, "mainCategoryId" integer NOT NULL, "featuresId" integer NOT NULL, "verificationRequirementsId" integer, "importance" double precision, "name" text NOT NULL, "description" text, "content" text
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "routeToPlace" (
"routeId" integer NOT NULL, "placeId" integer NOT NULL, CONSTRAINT "routeToPlace_placeId_routeId_pk" PRIMARY KEY ("placeId", "routeId")
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "route_translation" (
"id" serial PRIMARY KEY NOT NULL, "route_id" integer NOT NULL, "locale" varchar(10) NOT NULL, "name" text NOT NULL, "description" text, "content" text
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "user" (
"id" varchar(255) PRIMARY KEY NOT NULL, "name" text, "hashedPassword" varchar(255), "email" text NOT NULL, "emailVerified" timestamp, "image" text, "role" text DEFAULT 'user' NOT NULL, "visitedPlaceListId" integer NOT NULL
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "verificationRequirement" (
"id" serial PRIMARY KEY NOT NULL, "isLocationRequired" boolean DEFAULT true NOT NULL, "maxLocationDistance" integer
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "verification" (
"id" serial PRIMARY KEY NOT NULL, "placeId" integer NOT NULL, "userId" varchar(255) NOT NULL, "validatedOn" timestamp DEFAULT CURRENT_TIMESTAMP NOT NULL, "deviceLocation" Geometry (Point, 25831), "deviceLocationAccuracy" integer
);
--> statement-breakpoint
DO $$
BEGIN
ALTER TABLE "account"
ADD CONSTRAINT "account_userId_user_id_fk" FOREIGN KEY ("userId") REFERENCES "user" ("id") ON DELETE cascade ON UPDATE no action;
EXCEPTION WHEN duplicate_object THEN null;
END
$$;
--> statement-breakpoint
DO $$
BEGIN
ALTER TABLE "session"
ADD CONSTRAINT "session_userId_user_id_fk" FOREIGN KEY ("userId") REFERENCES "user" ("id") ON DELETE cascade ON UPDATE no action;
EXCEPTION WHEN duplicate_object THEN null;
END
$$;
Loading

0 comments on commit 9e4fd2d

Please sign in to comment.