Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: use canbench from HEAD in CI #67

Merged
merged 2 commits into from
Sep 18, 2024
Merged

Conversation

ielashi
Copy link
Collaborator

@ielashi ielashi commented Sep 16, 2024

Problem

The CI is downloading a version of canbench from crates.io, which means the benchmarks do not capture any performance changes from the current PR.

Solution

Update the CI script to build canbench from HEAD.

Note the regression in the btreemap example due to the change in pricing of stable memory operations in later versions of canbench.

@ielashi ielashi changed the title fix: use canbench from HEAD fix: use canbench from HEAD in CI Sep 16, 2024
Copy link

github-actions bot commented Sep 16, 2024

canbench 🏋 (dir: examples/btreemap_vs_hashmap)

Significant performance change detected! ⚠️

examples/btreemap_vs_hashmap/canbench_results.yml is up to date ✅


---------------------------------------------------

Benchmark: insert_users
  total:
    instructions: 2.57 B (no change)
    heap_increase: 871 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: remove_users
  total:
    instructions: 2.07 B (no change)
    heap_increase: 0 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: pre_upgrade_bench
  total:
    instructions: 743.98 M (regressed by 2.04%)
    heap_increase: 519 pages (no change)
    stable_memory_increase: 184 pages (no change)

  serialize_state (scope):
    instructions: 729.04 M (-0.01%) (change within noise threshold)
    heap_increase: 519 pages (no change)
    stable_memory_increase: 0 pages (no change)

  writing_to_stable_memory (scope):
    instructions: 14.93 M (regressed by 2950599.60%)
    heap_increase: 0 pages (no change)
    stable_memory_increase: 184 pages (no change)

---------------------------------------------------

Copy link

github-actions bot commented Sep 16, 2024

canbench 🏋 (dir: examples/fibonacci)

No significant performance changes detected ✅

examples/fibonacci/canbench_results.yml is up to date ✅


---------------------------------------------------

Benchmark: fibonacci_20
  total:
    instructions: 2224 (no change)
    heap_increase: 0 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: fibonacci_45
  total:
    instructions: 3011 (no change)
    heap_increase: 0 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

@ielashi ielashi marked this pull request as ready for review September 16, 2024 12:04
@ielashi ielashi requested a review from a team as a code owner September 16, 2024 12:04
@ielashi ielashi merged commit ae93b79 into main Sep 18, 2024
12 checks passed
@ielashi ielashi deleted the ielashi/fix_canbench_version branch September 18, 2024 09:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants