Skip to content

Commit

Permalink
update example with logs
Browse files Browse the repository at this point in the history
  • Loading branch information
ezekg committed Aug 16, 2024
1 parent 305344a commit c7e6460
Showing 1 changed file with 44 additions and 2 deletions.
46 changes: 44 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,13 +42,55 @@ $ gem install verbose_migrations
## Usage

```ruby
class SeedTagsFromPosts < ActiveRecord::Migration[7.1]
class SeedTagsFromPosts < ActiveRecord::Migration[7.2]
verbose!

def up
# ...
batch_count = 0
loop do
batch_count += 1
update_count = exec_update(<<~SQL.squish, batch_count:, batch_size: 10_000)
WITH batch AS (
SELECT id, unnest(tags) AS tag_name
FROM posts
WHERE tags IS NOT NULL
LIMIT :batch_size
), inserted_tags AS (
INSERT INTO tags (name)
SELECT DISTINCT tag_name
FROM batch
ON CONFLICT (name) DO NOTHING
RETURNING id, name
)
INSERT INTO post_tags (post_id, tag_id)
SELECT batch.id, tags.id
FROM batch
JOIN tags ON tags.name = batch.tag_name
/* batch=:batch_count */
SQL
break if update_count == 0
end
end
end

```

Before, you have a black box:

```
== 20240817010101 SeedTagsFromPosts: migrating ================================
== 20240817010101 SeedTagsFromPosts: migrated (0.0312s) =======================
```

After, you see progress:

```
== 20240817010101 SeedTagsFromPosts: migrating ================================
==> DEBUG -- : (0.2ms) WITH batch AS ( SELECT id, unnest(tags) AS tag_name FROM posts WHERE tags IS NOT NULL LIMIT 10000 ), inserted_tags AS ( INSERT INTO tags (name) SELECT DISTINCT tag_name FROM batch ON CONFLICT (name) DO NOTHING RETURNING id, name ) INSERT INTO post_tags (post_id, tag_id) SELECT batch.id, tags.id FROM batch JOIN tags ON tags.name = batch.tag_name /* batch=1 */
==> DEBUG -- : (0.2ms) WITH batch AS ( SELECT id, unnest(tags) AS tag_name FROM posts WHERE tags IS NOT NULL LIMIT 10000 ), inserted_tags AS ( INSERT INTO tags (name) SELECT DISTINCT tag_name FROM batch ON CONFLICT (name) DO NOTHING RETURNING id, name ) INSERT INTO post_tags (post_id, tag_id) SELECT batch.id, tags.id FROM batch JOIN tags ON tags.name = batch.tag_name /* batch=2 */
==> DEBUG -- : (0.2ms) WITH batch AS ( SELECT id, unnest(tags) AS tag_name FROM posts WHERE tags IS NOT NULL LIMIT 10000 ), inserted_tags AS ( INSERT INTO tags (name) SELECT DISTINCT tag_name FROM batch ON CONFLICT (name) DO NOTHING RETURNING id, name ) INSERT INTO post_tags (post_id, tag_id) SELECT batch.id, tags.id FROM batch JOIN tags ON tags.name = batch.tag_name /* batch=3 */
==> DEBUG -- : (0.1ms) ...
== 20240817010101 SeedTagsFromPosts: migrated (0.0312s) =======================
```

## Supported Rubies
Expand Down

0 comments on commit c7e6460

Please sign in to comment.