Include chunked phase2 script #439

emmorais · 2021-11-04T17:57:06Z

Ongoing work. Currently getting the following error:

Running `/media/eduardo/DATA/eduardo/Aleo/code/aleo-setup/target/release/setup2 new --curve-type bw6 --chunk-size 131072 --batch-size 131072 --contribution-mode full --challenge-fname challenge --challenge-hash-fname challenge.verified.hash --phase1-fname ../../phase1-cli/scripts/combined --phase1-powers 18 --num-validators 1 --num-epochs 1`
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: ZexeSerializationError(IoError(Custom { kind: Other, error: "Invalid field element" }))', setup2/src/cli/new.rs:200:6
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Pratyush · 2021-11-29T19:55:25Z

setup2/scripts/phase2_chunked.sh

+cargo $CARGO_VER build --release --bin setup2
+
+phase2_new="cargo run --release --features cli  -- new --curve-type $CURVE --chunk-size $CHUNK_SIZE --batch-size $BATCH --contribution-mode full"
+phase2_chunked="cargo run --release --bin setup2 --features cli  -- --curve-type $CURVE --chunk-size $CHUNK_SIZE --batch-size $BATCH --contribution-mode full --proving-system $PROVING_SYSTEM"


This should come with an argument for the binary, right? That is, one of new, contribute, etc?

Looks like we're still missing the seed and the proving_system in the related struct (edit: and possibly more judging by the rest of the script, and assuming it is correct).

The command is actually specified lower in the script.

@emmorais are these commands currently complete?

Pratyush · 2021-11-29T19:55:34Z

setup2/scripts/phase2_chunked.sh

+phase2_1="cargo run --release --bin setup2 --features cli  -- --curve-type $CURVE --batch-size $BATCH --contribution-mode chunked --chunk-size $CHUNK_SIZE --seed seed1 --proving-system $PROVING_SYSTEM"
+phase2_2="cargo run --release --bin setup2 --features cli  -- --curve-type $CURVE --batch-size $BATCH --contribution-mode chunked --chunk-size $CHUNK_SIZE --seed seed2 --proving-system $PROVING_SYSTEM"


Same for these two?

Pratyush · 2022-01-04T23:28:46Z

phase1-cli/src/bin/prepare_phase2.rs

-        UseCompression::Yes,
-        CheckForCorrectness::Full,
+        UseCompression::No,
+        CheckForCorrectness::No,


if we've already checked the output of Phase1, then we don't need to check for correctness, which makes the preparation step faster. This should be re-enabled if we did not check that the output of Phase 1 is in the appropriate prime-order group.

Pratyush · 2022-01-04T23:30:17Z

phase2/src/polynomial.rs

+    if input.len() > 10 {
+        let mut input = input
+            .par_iter()
+            .map(|(coeff, lag)| {
+                let ind = match *lag {
                    Index::Public(i) => i,
                    Index::Private(i) => num_inputs + i,
                };
-                sum += coeffs[ind].mul(coeff).into_projective();
-                sum
-            },
-        )
-        .reduce(|| C::Projective::zero(), |a, b| a + b)
+                (coeff, ind)
+            })
+            .collect::<Vec<_>>();
+        input.sort_unstable_by(|a, b| a.1.cmp(&b.1));
+        let input = input
+            .into_par_iter()
+            .map(|(coeff, _)| coeff.to_repr())
+            .collect::<Vec<_>>();
+        snarkvm_algorithms::msm::variable_base::VariableBaseMSM::multi_scalar_mul(coeffs, &input)


The dot product can be computed quickly using a variable base MSM. We do this if input.len() is large enough.

Pratyush · 2022-01-04T23:31:05Z

setup-utils/src/helpers.rs

    let mut points: Vec<_> = cfg_iter!(bases)
-        .map(|base| base.into_projective().mul(*coeff))
+        .map(|base| base.mul_bits(BitIteratorBE::new(coeff)))


This change ensures that we use mixed addition, which makes the multiplication slightly faster.

Pratyush · 2022-01-04T23:32:30Z

setup1-contributor/src/utils.rs

    let address = Address::try_from(private_key)?.to_string();

    let message = format!("{} {}", method.to_lowercase(), path.to_lowercase());
-    let signature = hex::encode(&view_key.sign(message.as_bytes(), rng)?.to_bytes_le()?);
+    let signature = hex::encode(&private_key.sign(message.as_bytes(), rng)?.to_bytes_le()?);


Q: is this change warranted? Not sure if this breaks things (did the address scheme change since the code was written?)

cc @howardwu

Pratyush · 2022-01-04T23:32:38Z

setup1-contributor/src/utils.rs

+    let private_key = PrivateKey::<Testnet2>::from_str(signing_key)?;
+    let signature = hex::encode(&private_key.sign(message.as_bytes(), rng)?.to_bytes_le()?);


Pratyush · 2022-01-04T23:35:46Z

Update: in the latest commit, the error that Eduardo encountered is no longer present, so the new subcommand runs to completion. The remaining work involves implementing the rest of the commands specified in the script.

Pratyush · 2022-01-06T01:28:10Z

In the latest commit, most tests are passing, except some tests in setup1-verifier. These tests are failing due to an inability to deserialize a PrivateKey from the hardcoded ViewKey bytes. We can no longer use the ViewKey because in the latest snarkvm, we use PrivateKeys to sign (whereas earlier we signed with ViewKeys).

To get the tests passing, I can sample a new PrivateKey and hardcode this in the tests, but overall I'm not sure if the new behaviour (i.e. switching away from ViewKey signing) is wanted.

niklaslong · 2022-01-11T20:42:16Z

This may need a rebase after #448 is merged.

Pratyush · 2022-01-18T19:09:39Z

Merged phase2 into this.

emmorais and others added 4 commits November 4, 2021 14:48

Include chunked phase2 script

b894388

Update script and include NewOpts fields

8f4d840

Include all missing parameters into NewOpt

128d72c

Fix curve-kind

82ba6db

Pratyush reviewed Nov 29, 2021

View reviewed changes

Better logging and perf improvements

f7cd635

Pratyush force-pushed the phase2_chunked_script branch from 774524c to f7cd635 Compare December 21, 2021 22:20

Pratyush added 6 commits December 21, 2021 22:45

Optimizatin

44d92ec

Updates

29a0f40

Upgrade to latest snarkvm where appropriate

13bc54d

Update Cargo.lock

003705b

Working new command in setup2

f639d42

Fix compilatin for setup1-contributor

3509c96

Pratyush reviewed Jan 4, 2022

View reviewed changes

Pratyush added 3 commits January 5, 2022 21:53

Completely remove old snarkvm

f7766f4

Update setup1-verifier

0f22458

Tweaks

2ac6f63

Pratyush added 2 commits January 14, 2022 00:04

Progress

3d0f600

Merge

86e3754

Pratyush added 2 commits January 29, 2022 20:58

Implement some more commands

81ab4f5

Implement combine command

b0715cc

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Include chunked phase2 script #439

Include chunked phase2 script #439

emmorais commented Nov 4, 2021 •

edited

Loading

Pratyush Nov 29, 2021

niklaslong Dec 6, 2021 •

edited

Loading

niklaslong Dec 6, 2021

niklaslong Dec 6, 2021

Pratyush Nov 29, 2021

niklaslong Dec 6, 2021

Pratyush Jan 4, 2022

Pratyush Jan 4, 2022

Pratyush Jan 4, 2022

Pratyush Jan 4, 2022

Pratyush Jan 4, 2022

Pratyush Jan 4, 2022

Pratyush commented Jan 4, 2022

Pratyush commented Jan 6, 2022

niklaslong commented Jan 11, 2022

Pratyush commented Jan 18, 2022

		phase2_1="cargo run --release --bin setup2 --features cli -- --curve-type $CURVE --batch-size $BATCH --contribution-mode chunked --chunk-size $CHUNK_SIZE --seed seed1 --proving-system $PROVING_SYSTEM"
		phase2_2="cargo run --release --bin setup2 --features cli -- --curve-type $CURVE --batch-size $BATCH --contribution-mode chunked --chunk-size $CHUNK_SIZE --seed seed2 --proving-system $PROVING_SYSTEM"

		let private_key = PrivateKey::<Testnet2>::from_str(signing_key)?;
		let signature = hex::encode(&private_key.sign(message.as_bytes(), rng)?.to_bytes_le()?);

Include chunked phase2 script #439

Are you sure you want to change the base?

Include chunked phase2 script #439

Conversation

emmorais commented Nov 4, 2021 • edited Loading

Choose a reason for hiding this comment

niklaslong Dec 6, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Pratyush commented Jan 4, 2022

Pratyush commented Jan 6, 2022

niklaslong commented Jan 11, 2022

Pratyush commented Jan 18, 2022

emmorais commented Nov 4, 2021 •

edited

Loading

niklaslong Dec 6, 2021 •

edited

Loading