Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Light Client States Updates #111

Merged
merged 52 commits into from
May 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
a351312
pipeline for merkle proof validation
nomaxg Apr 2, 2024
f1d03a4
verify stub proof in replay.go
nomaxg Apr 2, 2024
3cd4b80
add espresso crypto
nomaxg Apr 2, 2024
5c940f3
update raw.s
nomaxg Apr 2, 2024
dc24eb5
add todo
nomaxg Apr 2, 2024
cdbaa51
fix wasm build
nomaxg Apr 2, 2024
d6d2fcb
add some mock merkle logic
nomaxg Apr 2, 2024
a23e3c4
use mock data
nomaxg Apr 2, 2024
1da99db
more mock validation code
nomaxg Apr 5, 2024
a327c87
more mock code
nomaxg Apr 16, 2024
415ec62
merge
nomaxg Apr 16, 2024
4053075
Fix the e2e test
ImJeremyHe Apr 17, 2024
a0332f1
serialize proof variables correctly
nomaxg Apr 17, 2024
d56f060
update espresso-go version
nomaxg Apr 18, 2024
cbf1880
wasm compatibility
nomaxg Apr 18, 2024
2160d86
remove stale file, lint fix
nomaxg Apr 18, 2024
5ac43b3
merge e2e fixes
nomaxg Apr 18, 2024
e30ea2a
update to recent espresso go version
nomaxg Apr 18, 2024
958a068
refactor block merkle verification
nomaxg Apr 22, 2024
1f59045
work on E2E
nomaxg Apr 23, 2024
0c3f87a
progress on merkle tree verification
nomaxg Apr 25, 2024
0b2b876
add test vectors, fetch proofs at the correct indicies
nomaxg Apr 25, 2024
c0e755c
cleanup, remove outfiles, fetch merkle proof at the correct justified…
nomaxg Apr 29, 2024
7aead9e
add logic to compare circuit field
nomaxg Apr 30, 2024
971c9cb
check that circuit representation equals u256 representation in the test
nomaxg Apr 30, 2024
4271370
send byte slice to wasm instead of byte array which lives on the stack
nomaxg May 3, 2024
5d9554d
cleanup, remove debug statements, add documentation
nomaxg May 3, 2024
f76dbb7
remove error logs
nomaxg May 3, 2024
58055d6
fix build
nomaxg May 3, 2024
d9275d4
lint fix
nomaxg May 3, 2024
383725d
progress on e2e tests, comment out vid stuff for now because of wasm …
nomaxg May 6, 2024
ef03d22
update espresso-sequencer-go tag, lint ci fix
nomaxg May 6, 2024
e8a1d79
cleanup, add issues for unresolved problems
nomaxg May 7, 2024
13cec14
remove block that spams transactions
nomaxg May 7, 2024
13dd1a3
comment out unused txns
nomaxg May 7, 2024
a65bdb8
Allow dead code in validation
ImJeremyHe May 8, 2024
7718708
Specify foundry version
ImJeremyHe May 8, 2024
a7d9d43
Add random_get stub in wasi_snapshot_preview1
ImJeremyHe May 8, 2024
ca5a362
Add stubs required in wasi_snapshot_preview1
ImJeremyHe May 8, 2024
8be940c
Fix `make lint`
sveitser May 8, 2024
a95173d
Enable prover service to talk to L1 on host
sveitser May 8, 2024
3bf48c9
Add builder timeout
sveitser May 8, 2024
dd885d1
Add missing env var, more logging in e2e test
sveitser May 8, 2024
8d165f1
Revert "Fix `make lint`"
sveitser May 8, 2024
dd2b2a0
Fix go linting
sveitser May 8, 2024
0b43062
Parse espresso messages
ImJeremyHe May 8, 2024
8732617
Add timeout limit
ImJeremyHe May 9, 2024
9aa3e0d
Fix the e2e test
ImJeremyHe May 9, 2024
47b6bb2
Add timeout limit
ImJeremyHe May 9, 2024
10d3214
Run the e2e test and switch test in 1 thread
ImJeremyHe May 9, 2024
b199e74
Update the e2e workflow
ImJeremyHe May 9, 2024
4500909
Increase time limit
ImJeremyHe May 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/arbitrator-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,8 @@ jobs:

- name: Install Foundry
uses: foundry-rs/foundry-toolchain@v1
with:
version: nightly-f625d0fa7c51e65b4bf1e8f7931cd1c6e2e285e9

- name: Cache cbrotli
uses: actions/cache@v3
Expand Down
5 changes: 4 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,8 @@ jobs:

- name: Install Foundry
uses: foundry-rs/foundry-toolchain@v1
with:
version: nightly-f625d0fa7c51e65b4bf1e8f7931cd1c6e2e285e9

- name: Cache Build Products
uses: actions/cache@v3
Expand Down Expand Up @@ -126,8 +128,9 @@ jobs:
run: make -j build-node-deps

- name: Lint
uses: golangci/golangci-lint-action@v3
uses: golangci/golangci-lint-action@v5
with:
annotations: false
version: latest
skip-pkg-cache: true
- name: Custom Lint
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,8 @@ jobs:

- name: Install Foundry
uses: foundry-rs/foundry-toolchain@v1
with:
version: nightly-f625d0fa7c51e65b4bf1e8f7931cd1c6e2e285e9

- name: Cache Rust Build Products
uses: actions/cache@v3
Expand Down
11 changes: 9 additions & 2 deletions .github/workflows/espresso-e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,8 @@ jobs:

- name: Install Foundry
uses: foundry-rs/foundry-toolchain@v1
with:
version: nightly-f625d0fa7c51e65b4bf1e8f7931cd1c6e2e285e9

- name: Cache Build Products
uses: actions/cache@v3
Expand Down Expand Up @@ -130,7 +132,12 @@ jobs:
with:
version: '2.22.0'

- name: Run test
- name: Run E2E test
run: |
packages=`go list ./... | grep system_tests`
gotestsum --format short-verbose --packages="$packages" --rerun-fails=1 -- -v -timeout 25m -p 1 ./... -run 'TestEspressoE2E'

- name: Run switch test
run: |
packages=`go list ./... | grep system_tests`
gotestsum --format short-verbose --packages="$packages" --rerun-fails=1 -- -v -timeout 35m ./... -run 'TestEspressoE2E|TestEspressoSwitch'
gotestsum --format short-verbose --packages="$packages" --rerun-fails=1 -- -v -timeout 15m -p 1 ./... -run 'TestEspressoSwitch'
8 changes: 4 additions & 4 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ RUN apt-get install -y clang=1:14.0-55.7~deb12u1 lld=1:14.0-55.7~deb12u1
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain 1.70.0 --target x86_64-unknown-linux-gnu wasm32-unknown-unknown wasm32-wasi
COPY ./Makefile ./
COPY arbitrator/arbutil arbitrator/arbutil
COPY arbitrator/vid-helper arbitrator/vid-helper
COPY arbitrator/espresso-crypto-helper arbitrator/espresso-crypto-helper
COPY config/vid_srs.json config/vid_srs.json
COPY arbitrator/wasm-libraries arbitrator/wasm-libraries
COPY --from=brotli-wasm-export / target/
Expand All @@ -60,7 +60,7 @@ FROM wasm-base as wasm-bin-builder
RUN curl -L https://golang.org/dl/go1.20.linux-`dpkg --print-architecture`.tar.gz | tar -C /usr/local -xzf -
COPY ./Makefile ./go.mod ./go.sum ./
COPY ./arbcompress ./arbcompress
COPY ./arbvid ./arbvid
COPY ./espressocrypto ./espressocrypto
COPY ./arbos ./arbos
COPY ./arbstate ./arbstate
COPY ./arbutil ./arbutil
Expand Down Expand Up @@ -96,7 +96,7 @@ COPY ./Makefile ./
COPY arbitrator/arbutil arbitrator/arbutil
COPY arbitrator/prover arbitrator/prover
COPY arbitrator/jit arbitrator/jit
COPY arbitrator/vid-helper arbitrator/vid-helper
COPY arbitrator/espresso-crypto-helper arbitrator/espresso-crypto-helper
COPY config/vid_srs.json config/vid_srs.json
RUN NITRO_BUILD_IGNORE_TIMESTAMPS=1 make build-prover-header

Expand All @@ -113,7 +113,7 @@ COPY arbitrator/Cargo.* arbitrator/
COPY arbitrator/arbutil arbitrator/arbutil
COPY arbitrator/prover/Cargo.toml arbitrator/prover/
COPY arbitrator/jit/Cargo.toml arbitrator/jit/
COPY arbitrator/vid-helper arbitrator/vid-helper
COPY arbitrator/espresso-crypto-helper arbitrator/espresso-crypto-helper
COPY config/vid_srs.json config/vid_srs.json
RUN mkdir arbitrator/prover/src arbitrator/jit/src && \
echo "fn test() {}" > arbitrator/jit/src/lib.rs && \
Expand Down
8 changes: 4 additions & 4 deletions Dockerfile.espresso
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ RUN apt-get install -y clang=1:14.0-55.7~deb12u1 lld=1:14.0-55.7~deb12u1
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain 1.70.0 --target x86_64-unknown-linux-gnu wasm32-unknown-unknown wasm32-wasi
COPY ./Makefile ./
COPY arbitrator/arbutil arbitrator/arbutil
COPY arbitrator/vid-helper arbitrator/vid-helper
COPY arbitrator/espresso-crypto-helper arbitrator/espresso-crypto-helper
COPY config/vid_srs.json config/vid_srs.json
COPY arbitrator/wasm-libraries arbitrator/wasm-libraries
COPY --from=brotli-wasm-export / target/
Expand All @@ -64,7 +64,7 @@ FROM wasm-base as wasm-bin-builder
RUN curl -L https://golang.org/dl/go1.20.linux-`dpkg --print-architecture`.tar.gz | tar -C /usr/local -xzf -
COPY ./Makefile ./go.mod ./go.sum ./
COPY ./arbcompress ./arbcompress
COPY ./arbvid ./arbvid
COPY ./espressocrypto ./espressocrypto
COPY ./arbos ./arbos
COPY ./arbstate ./arbstate
COPY ./arbutil ./arbutil
Expand Down Expand Up @@ -100,7 +100,7 @@ COPY ./Makefile ./
COPY arbitrator/arbutil arbitrator/arbutil
COPY arbitrator/prover arbitrator/prover
COPY arbitrator/jit arbitrator/jit
COPY arbitrator/vid-helper arbitrator/vid-helper
COPY arbitrator/espresso-crypto-helper arbitrator/espresso-crypto-helper
COPY config/vid_srs.json config/vid_srs.json
RUN NITRO_BUILD_IGNORE_TIMESTAMPS=1 make build-prover-header

Expand All @@ -117,7 +117,7 @@ COPY arbitrator/Cargo.* arbitrator/
COPY arbitrator/arbutil arbitrator/arbutil
COPY arbitrator/prover/Cargo.toml arbitrator/prover/
COPY arbitrator/jit/Cargo.toml arbitrator/jit/
COPY arbitrator/vid-helper arbitrator/vid-helper
COPY arbitrator/espresso-crypto-helper arbitrator/espresso-crypto-helper
COPY config/vid_srs.json config/vid_srs.json
RUN mkdir arbitrator/prover/src arbitrator/jit/src && \
echo "fn test() {}" > arbitrator/jit/src/lib.rs && \
Expand Down
12 changes: 6 additions & 6 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -44,13 +44,13 @@ color_reset = "\e[0;0m"

done = "%bdone!%b\n" $(color_pink) $(color_reset)

replay_deps=arbos wavmio arbstate arbcompress arbvid solgen/go/node-interfacegen blsSignatures cmd/replay
replay_deps=arbos wavmio arbstate arbcompress espressocrypto solgen/go/node-interfacegen blsSignatures cmd/replay

replay_wasm=$(output_root)/machines/latest/replay.wasm

arbitrator_generated_header=$(output_root)/include/arbitrator.h
arbitrator_wasm_libs_nogo=$(output_root)/machines/latest/wasi_stub.wasm $(output_root)/machines/latest/host_io.wasm $(output_root)/machines/latest/soft-float.wasm
arbitrator_wasm_libs=$(arbitrator_wasm_libs_nogo) $(patsubst %,$(output_root)/machines/latest/%.wasm, go_stub brotli vid)
arbitrator_wasm_libs=$(arbitrator_wasm_libs_nogo) $(patsubst %,$(output_root)/machines/latest/%.wasm, go_stub brotli espresso_crypto)
arbitrator_prover_lib=$(output_root)/lib/libprover.a
arbitrator_prover_bin=$(output_root)/bin/prover
arbitrator_jit=$(output_root)/bin/jit
Expand Down Expand Up @@ -282,13 +282,13 @@ $(output_root)/machines/latest/brotli.wasm: $(DEP_PREDICATE) $(wildcard arbitrat
cargo build --manifest-path arbitrator/wasm-libraries/Cargo.toml --release --target wasm32-wasi --package brotli
install arbitrator/wasm-libraries/target/wasm32-wasi/release/brotli.wasm $@

$(output_root)/machines/latest/vid.wasm: $(DEP_PREDICATE) $(wildcard arbitrator/wasm-libraries/vid/src/*)
$(output_root)/machines/latest/espresso_crypto.wasm: $(DEP_PREDICATE) $(wildcard arbitrator/wasm-libraries/espresso-crypto/src/*)
mkdir -p $(output_root)/machines/latest
cargo build --manifest-path arbitrator/wasm-libraries/Cargo.toml --release --target wasm32-wasi --package vid
install arbitrator/wasm-libraries/target/wasm32-wasi/release/vid.wasm $@
cargo build --manifest-path arbitrator/wasm-libraries/Cargo.toml --release --target wasm32-wasi --package espresso-crypto
install arbitrator/wasm-libraries/target/wasm32-wasi/release/espresso_crypto.wasm $@

$(output_root)/machines/latest/machine.wavm.br: $(DEP_PREDICATE) $(arbitrator_prover_bin) $(arbitrator_wasm_libs) $(replay_wasm)
$(arbitrator_prover_bin) $(replay_wasm) --generate-binaries $(output_root)/machines/latest -l $(output_root)/machines/latest/soft-float.wasm -l $(output_root)/machines/latest/wasi_stub.wasm -l $(output_root)/machines/latest/go_stub.wasm -l $(output_root)/machines/latest/host_io.wasm -l $(output_root)/machines/latest/brotli.wasm -l $(output_root)/machines/latest/vid.wasm
$(arbitrator_prover_bin) $(replay_wasm) --generate-binaries $(output_root)/machines/latest -l $(output_root)/machines/latest/soft-float.wasm -l $(output_root)/machines/latest/wasi_stub.wasm -l $(output_root)/machines/latest/go_stub.wasm -l $(output_root)/machines/latest/host_io.wasm -l $(output_root)/machines/latest/brotli.wasm -l $(output_root)/machines/latest/espresso_crypto.wasm

$(arbitrator_cases)/%.wasm: $(arbitrator_cases)/%.wat
wat2wasm $< -o $@
Expand Down
Loading
Loading