Skip to content

Commit

Permalink
experiment: emulate stable memory in WASIMode and WasmMode (to suppor…
Browse files Browse the repository at this point in the history
…t testing of stable data structures on non-IC targets) (#4256)

This PR exploits wasmtime's support for multiple memory and bulk memory instructions to emulate the IC stable memory API (i.e. ExperimentalStableMemory.mo and Regions.mo).

The basic idea is to introduce an extra Wasm memory and emulate the low-level  `ic0.stable64_grow/size/read/write` operations, using bulk MemoryCopy instructions for data transfer.

The goal is to allow users to easily unit-test stable data structures without having to resort to the full IC environment.
For example, the third-party StableBTree implementation went to a lot of trouble and was less efficient as a result, just to support testing outside the IC.

@rvanasa would this work with your VSCode unit testing support? It does require enabling some additional features on wasmtime (no idea about compatibility with wasmer, if we still use that).

The current implementation only adds a 32-bit memory, but it should be possible to add a 64-bit one instead. I just want to get this working first. 32-bit might be all we can support in browsers anyway (not sure).

- [x] enable wasmtime flags in base repo so tests compile (this is one reason CI is failing). Alternatively, only add the stable memory if it's used.
- [ ] PR to mops to enable additional wasmtime flags
  • Loading branch information
crusso authored Nov 11, 2023
1 parent b3c1234 commit 956de8c
Show file tree
Hide file tree
Showing 39 changed files with 1,102 additions and 175 deletions.
8 changes: 8 additions & 0 deletions Changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,14 @@

* motoko (`moc`)

* Code compiled for targets WASI (`-wasi-system-api`) and pure Wasm (`-no-system-api`) can now
use up to 4GB of (efficiently emulated) stable memory, enabling more offline testing of, for example,
stable data structures built using libraries `Regions.mo` and `ExperimentalStableMemory.mo`.
Note that any Wasm engine (such as `wasmtime`), used to execute such binaries, must support and enable
Wasm features multi-memory and bulk-memory (as well as the standard NaN canonicalization) (#4256).

* bugfix: fully implement `Region.loadXXX/storeXXX` for `Int8`, `Int16` and `Float` (#4270).

* BREAKING CHANGE (Minor): values of type `Principal` are now constrained to contain
at most 29 bytes, matching the IC's notion of principal (#4268).

Expand Down
Loading

0 comments on commit 956de8c

Please sign in to comment.