Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discv5 support #6151

Closed
wants to merge 211 commits into from
Closed
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
211 commits
Select commit Hold shift + click to select a range
88eca09
Merge v4 and v5 streams and add downgrade logic to discv4
emhane Jan 19, 2024
c99982c
fixup! Merge v4 and v5 streams and add downgrade logic to discv4
emhane Jan 22, 2024
186bac3
Add combined handler for discv5 and discv4
emhane Jan 22, 2024
ee5433d
Merge branch 'main' into emhane/discv5-support
emhane Jan 23, 2024
ec2cb1a
Simplify streams
emhane Jan 24, 2024
dbaf224
Avoid heap allocation
emhane Jan 24, 2024
2ff13bd
Move discv5 out of discv4 data struct
emhane Jan 24, 2024
e0b4472
Shrink smallvec to max 2x 64 bytes hashes
emhane Jan 24, 2024
f21f5a5
Fix lint
emhane Jan 24, 2024
9eb8ef6
Plug discv5 into newtrok
emhane Jan 25, 2024
d545be5
Box update stream
emhane Jan 25, 2024
2af5dcb
Add docs
emhane Jan 25, 2024
48f0119
Plug in discv5 messages
emhane Jan 25, 2024
77d5b88
Implement HandleDiscovery for discv5
emhane Jan 26, 2024
ac81277
Fix docs
emhane Jan 26, 2024
8c1b07e
Fix docs and fetaure gates for discv5
emhane Jan 26, 2024
11ad046
Merge branch 'main' into emhane/discv5-support
emhane Jan 26, 2024
c1f80b0
Fix lint
emhane Jan 26, 2024
9c9b9b2
Merge branch 'emhane/discv5-support' of github.com:paradigmxyz/reth i…
emhane Jan 26, 2024
b1b9b33
Fix docs
emhane Jan 26, 2024
55ae4fe
Feature gate discv4 test
emhane Jan 26, 2024
137ccb6
Fix lint
emhane Jan 26, 2024
77dad1b
Fix disable discovery for tests
emhane Jan 26, 2024
ac2a2ca
Fix readme
emhane Jan 26, 2024
c4741f1
Fix readme
emhane Jan 26, 2024
64559d7
Make node record generic in dns to allow inserting into discv5 routin…
emhane Jan 27, 2024
578b4d5
Merge branch 'emhane/discv5-support' of github.com:paradigmxyz/reth i…
emhane Jan 27, 2024
e3f38e7
Make node record generic in dns to allow inserting into discv5 routin…
emhane Jan 27, 2024
36c47a6
Receive full enr in Discovery type
emhane Jan 28, 2024
68514a4
Reset original discv4 public api
emhane Jan 31, 2024
5f438ef
Fix dns to be compatible for discv5
emhane Jan 31, 2024
cf41a0e
Fix plug in dns into discv5
emhane Jan 31, 2024
7b6c2e7
Fix lint
emhane Jan 31, 2024
178e852
Lint fixes
emhane Feb 1, 2024
2d6cc07
Fix deps
emhane Feb 1, 2024
7c83bfe
Merge branch 'emhane/discv5-support'
emhane Feb 1, 2024
ed5518b
Merge branch 'main' into emhane/discv5-support
emhane Feb 4, 2024
d7b5d4a
Checkout reth binary from main
emhane Feb 4, 2024
3fdaa91
Merge branch 'emhane/discv5-support' of github.com:paradigmxyz/reth i…
emhane Feb 4, 2024
a765c4d
Fix network crate deps
emhane Feb 4, 2024
8eecf48
Fix lint
emhane Feb 4, 2024
a7f45a8
Merge branch 'main' into emhane/discv5-support
emhane Feb 29, 2024
607f7ed
Fix merge conflicts with main
emhane Feb 29, 2024
56032d7
Fix lint
emhane Feb 29, 2024
2629782
Fix doctest
emhane Feb 29, 2024
7f1ccd2
Merge branch 'main' into emhane/discv5-support
emhane Mar 9, 2024
b6ec00c
Fix merge conflicts
emhane Mar 9, 2024
4c83b24
Expose API of discv4 and discv5 for tests
emhane Mar 9, 2024
48870f2
Clear nodes from discv4 that connect on discv5 later
emhane Mar 9, 2024
21bb035
Add test for discv5 with discv4 downgrade
emhane Mar 9, 2024
c79aaf4
Move notification of discv5 kbuckets update a layer lower into Merged…
emhane Mar 9, 2024
8bf41a7
Add todo to address conversion between discv5 and discv4 node ids
emhane Mar 9, 2024
e4354d0
Add test for converting discv4 node id to discv5 node id
emhane Mar 14, 2024
ea1b2c2
Derive as ref for node key
emhane Mar 14, 2024
e9c14ba
Use discv5 master branch to extract public key from enr without cloni…
emhane Mar 14, 2024
0674335
fixup! Derive as ref for node key
emhane Mar 14, 2024
05e63c3
Remove nodes connected over discv5 from pending discv4 requests
emhane Mar 14, 2024
504c394
Drop incoming discv4 requests from senders connected over discv5
emhane Mar 14, 2024
e4a63b2
Add helpful debug info
emhane Mar 14, 2024
bc3bce4
Fix bug discv4 not notified of updated discv5 kbuckets on manually ad…
emhane Mar 14, 2024
a3434c0
Only emit log message of discv4 kbuckets state on lookup interval tick
emhane Mar 14, 2024
e30ca24
Generalise node record with fork id and filter out discv5 enrs withou…
emhane Mar 14, 2024
d61173e
Clean up crypto
emhane Mar 14, 2024
be27d99
Fix lint
emhane Mar 14, 2024
77e945c
fixup! Clean up crypto
emhane Mar 14, 2024
c3ca494
Merge branch 'emhane/discv5-support'
emhane Mar 15, 2024
a7f4095
Move interface between v5 and v4 node id to own file
emhane Mar 15, 2024
26a5edd
Fix docs
emhane Mar 15, 2024
5844ee2
Fix magic numbers in crypto
emhane Mar 15, 2024
68c3d94
Remove debug code
emhane Mar 15, 2024
2fded4c
Disable checking fork id on discv5 enr
emhane Mar 15, 2024
8d41f4f
Notify discv4 of likely kbucket updates
emhane Mar 15, 2024
ffa99d1
Fix imports test
emhane Mar 15, 2024
cb51b76
Properly remove peer from discv4
emhane Mar 15, 2024
0c51381
fixup! Notify discv4 of likely kbucket updates
emhane Mar 15, 2024
8575e5f
Remove unnecessary lock on discv5
emhane Mar 16, 2024
6ae4903
Allow git for discv5
emhane Mar 16, 2024
a29bd1a
Move minimal disocvery IF to common crate
emhane Mar 16, 2024
e5f896c
Add discv5 only IF and add error handling
emhane Mar 16, 2024
6a595b3
Update readme
emhane Mar 16, 2024
5cb1a9c
fixup! Add discv5 only IF and add error handling
emhane Mar 16, 2024
5a5bb20
Enable discv5 only
emhane Mar 16, 2024
d130909
Allow ci to pass on --all-features flag set
emhane Mar 16, 2024
1e3f810
Merge branch 'main' into emhane/discv5-support
emhane Mar 16, 2024
cf44158
Fix Cargo.lock
emhane Mar 16, 2024
5589e13
Export discovery type aliases
emhane Mar 16, 2024
afd349b
fixup! Export discovery type aliases
emhane Mar 17, 2024
24de4fd
Temp disable all features in ci
emhane Mar 17, 2024
18facf1
fixup! Temp disable all features in ci
emhane Mar 17, 2024
9bb1a9f
fixup! Temp disable all features in ci
emhane Mar 17, 2024
8a984fe
Fix feature logic wrt de morgans law
emhane Mar 17, 2024
a785e17
Permanent fix for CI lint with mutually exclusive features
emhane Mar 17, 2024
89b32f5
Improve error handling
emhane Mar 17, 2024
e015205
Plug in discv5 hive test
emhane Mar 17, 2024
22c3cfd
fixup! Plug in discv5 hive test
emhane Mar 17, 2024
796ddb4
fixup! Improve error handling
emhane Mar 17, 2024
be0f975
Allow disable discovery for tests
emhane Mar 17, 2024
50cf5d6
Fix lint
emhane Mar 17, 2024
5de989c
Fix make file for new features
emhane Mar 18, 2024
2eb0411
Fix docs
emhane Mar 18, 2024
a2c7f9a
fixup! Allow disable discovery for tests
emhane Mar 18, 2024
0646475
Make conversion from discv5 less restrictive, allow omit tcp
emhane Mar 18, 2024
8e2e7f2
Re-scope dns instantiation
emhane Mar 18, 2024
ed10662
Stream enr that doesn't decode into node record, as on main
emhane Mar 18, 2024
e9a828f
Syntax fix
emhane Mar 18, 2024
066e53b
Allow discv5 nodes without tcp port to be added to discovered nodes
emhane Mar 18, 2024
4f07579
Revise todo
emhane Mar 18, 2024
ab953f0
Reorder impl bodies within file
emhane Mar 18, 2024
8bb71c1
Fix docs
emhane Mar 18, 2024
7da82d1
Fix lint
emhane Mar 18, 2024
428d817
Temp configure hive tests to run in pr ci
emhane Mar 19, 2024
0af2ce7
Merge remote-tracking branch 'remotes/origin/emhane/discv5-support' i…
emhane Mar 19, 2024
5ba3005
Revert commit 428d817c3
emhane Mar 19, 2024
48daeda
Bootstrap discv5
emhane Mar 19, 2024
84c0130
Fix lint
emhane Mar 19, 2024
acaceab
Fix lint
emhane Mar 19, 2024
f0e5f1f
Merge remote-tracking branch 'remotes/origin/emhane/discv5-support' i…
emhane Mar 19, 2024
ae59958
Use ip mode to decide if a peer is contactable from local node
emhane Mar 19, 2024
0292f36
Add todo
emhane Mar 19, 2024
8048311
Fix module docs
emhane Mar 19, 2024
d60d8c9
Add tcp port to local enr
emhane Mar 20, 2024
ffa62a2
Remove optimism feature from discv5 crate
emhane Mar 20, 2024
0a15365
fixup! Add tcp port to local enr
emhane Mar 20, 2024
bef703f
Fix conversion into node record wrt ip mode
emhane Mar 20, 2024
32ec97a
fixup! Remove optimism feature from discv5 crate
emhane Mar 20, 2024
b901f32
Allow adding serialized bootnodes
emhane Mar 20, 2024
1c3fb13
Allow adding other fields to local enr via config
emhane Mar 20, 2024
eb0ad69
Manually init first lookup
emhane Mar 20, 2024
65ed334
Make fork id optional
emhane Mar 21, 2024
8329ad8
Add enodes as bootnodes and make self-lookup query periodic
emhane Mar 21, 2024
2ff5d61
Order module prelude
emhane Mar 21, 2024
adb3f56
Add tracing and re-order module prelude
emhane Mar 21, 2024
8a9ba33
Add todo
emhane Mar 21, 2024
aa8d675
Make errors more helpful
emhane Mar 21, 2024
7966e4f
Fix fs
emhane Mar 21, 2024
b4d96b6
Rescope start functions
emhane Mar 21, 2024
e9d36ce
Fix logs
emhane Mar 21, 2024
69184a7
Test discv5 reth types in discv5 crate
emhane Mar 21, 2024
498f20a
Fix merge conflicts
emhane Mar 21, 2024
501cc52
Parse enode into sigp/discv5 compat multiaddress
emhane Mar 21, 2024
6efb42f
fixup! Parse enode into sigp/discv5 compat multiaddress
emhane Mar 21, 2024
c2d0bf3
Update hardcoded bootnodes in tests
emhane Mar 22, 2024
e708350
Fix target tracing
emhane Mar 22, 2024
dd594e9
Add tracing, checkpoint swarm meets discovery
emhane Mar 22, 2024
d4388a6
Constraint peer lookup to relevant peers with generic filter
emhane Mar 22, 2024
9034fad
Make discv5 generic over filter in network
emhane Mar 22, 2024
f78b51d
fixup! Make discv5 generic over filter in network
emhane Mar 22, 2024
bcb15e3
Set default filter
emhane Mar 22, 2024
e59d0e7
fixup! Make discv5 generic over filter in network
emhane Mar 22, 2024
828b6ac
Set default filter as default generic
emhane Mar 22, 2024
38ea824
Fix clone discv5 handle
emhane Mar 23, 2024
a8b0bd1
Fix get latest fork id
emhane Mar 23, 2024
749beb2
Make default lookup filter stricter
emhane Mar 23, 2024
812de70
Add bootstrap tracing
emhane Mar 23, 2024
49effe7
fixup! Fix clone discv5 handle
emhane Mar 23, 2024
75d12df
fixup! Make default lookup filter stricter
emhane Mar 23, 2024
044a180
fixup! Make default lookup filter stricter
emhane Mar 23, 2024
598b4eb
Fix constructor of fork filter
emhane Mar 23, 2024
27dacce
Fix downgrade feature
emhane Mar 23, 2024
0b545cd
Fix lint
emhane Mar 23, 2024
7593a33
Advertise fork hash not fork id
emhane Mar 23, 2024
8509b74
Debug v4 downgrade test
emhane Mar 23, 2024
86bbb00
Reset filtering to less restrictive filter matching only on chain not…
emhane Mar 23, 2024
3365ee0
Enforce filtering discovered peers
emhane Mar 23, 2024
3fd9eab
Avoid double rlp decoding
emhane Mar 23, 2024
f7cbbd7
fixup! Reset filtering to less restrictive filter matching only on ch…
emhane Mar 23, 2024
d4dac01
fixup! Reset filtering to less restrictive filter matching only on ch…
emhane Mar 23, 2024
5b4a682
fixup! Reset filtering to less restrictive filter matching only on ch…
emhane Mar 23, 2024
9228b22
Fix funky refactor
emhane Mar 23, 2024
1f8d9fd
fixup! Fix funky refactor
emhane Mar 23, 2024
4c45861
Verify advertised mempool socket against mempool socket
emhane Mar 23, 2024
34b2fb1
fixup! Verify advertised mempool socket against mempool socket
emhane Mar 23, 2024
e278977
Warn on advertised mempool socket overwrite
emhane Mar 23, 2024
e8cb761
Add metrics
emhane Mar 23, 2024
340713f
Remove uneccessary import of workspace dep via discv5 crate
emhane Mar 23, 2024
ce6ef76
Clean up type system
emhane Mar 23, 2024
7d192cb
Add fix lint to make file
emhane Mar 23, 2024
5146cc5
Remove discv5 feature
emhane Mar 23, 2024
04ca057
fixup! Remove discv5 feature
emhane Mar 24, 2024
3c6b289
fixup! Remove discv5 feature
emhane Mar 24, 2024
a1eb822
fixup! Remove discv5 feature
emhane Mar 24, 2024
96463ee
Add filter MustNotIncludeChains
emhane Mar 24, 2024
04cd8aa
Add discv5 cli args
emhane Mar 26, 2024
b7e4fd3
Add test for filter
emhane Mar 26, 2024
59550fb
Fix ignore message
emhane Mar 26, 2024
5ef97f4
Fix network args and make config in manager
emhane Mar 26, 2024
ecaece6
Fix naming
emhane Mar 26, 2024
bee9d62
Disable same filter used during lookup queries as on result passed to…
emhane Mar 26, 2024
d55fda2
Temp hardcode filter to exlcude eth2 nodes
emhane Mar 26, 2024
3ceb8e3
fixup! Fix network args and make config in manager
emhane Mar 26, 2024
440db80
fix lint
emhane Mar 26, 2024
9930360
Use discv5 default port and set tcp port
emhane Mar 26, 2024
989c129
Temp use other ports to allow testing
emhane Mar 27, 2024
9bc1c2b
fixup! Temp use other ports to allow testing
emhane Mar 27, 2024
12e29a5
fixup! Temp use other ports to allow testing
emhane Mar 27, 2024
fe45c19
fixup! Temp use other ports to allow testing
emhane Mar 27, 2024
b68b3f6
Temp disable discv4 for p2p command test
emhane Mar 27, 2024
b32df13
Add sepolia bootstrap nodes
emhane Mar 27, 2024
3d49986
Bump zstd
emhane Mar 27, 2024
5c11bdc
Remove use of local method
emhane Mar 27, 2024
a97b0ba
Show disconnected bootnodes
emhane Mar 27, 2024
e644ea2
Use op mainnet bootstrap nodes for base mainnet
emhane Mar 27, 2024
b558124
Use op sepolia bootnodes for base sepolia
emhane Mar 27, 2024
c628cc4
add recover_signer_unchecked_with_buffer to TransactionSignedNoHash
joshieDo Mar 28, 2024
409c5eb
clippy
joshieDo Mar 28, 2024
211da78
clear buffer before use
joshieDo Mar 28, 2024
39f0827
rename to encode_and_recover_unchecked
joshieDo Mar 28, 2024
36cbac5
encode TxDeposit on encode_and_recover_unchecked
joshieDo Mar 28, 2024
9f6ba86
Merge joshie/fix-optimism-sender-recovery
emhane Mar 30, 2024
1da55aa
Find contacts in all subtrees
emhane Mar 31, 2024
a8ce615
Merge branch 'emhane/discv5-support' of github.com:paradigmxyz/reth i…
emhane Mar 31, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 75 additions & 29 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 4 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ members = [
"crates/metrics/metrics-derive/",
"crates/net/common/",
"crates/net/discv4/",
"crates/net/discv5/",
"crates/net/dns/",
"crates/net/downloaders/",
"crates/net/ecies/",
Expand Down Expand Up @@ -114,6 +115,7 @@ reth-config = { path = "crates/config" }
reth-consensus-common = { path = "crates/consensus/common" }
reth-db = { path = "crates/storage/db" }
reth-discv4 = { path = "crates/net/discv4" }
reth-discv5 = { path = "crates/net/discv5" }
reth-dns-discovery = { path = "crates/net/dns" }
reth-node-builder = { path = "crates/node-builder" }
reth-node-core = { path = "crates/node-core" }
Expand Down Expand Up @@ -181,8 +183,7 @@ ethers-core = { version = "2.0", default-features = false }
ethers-providers = { version = "2.0", default-features = false }
ethers-signers = { version = "2.0", default-features = false }
ethers-middleware = { version = "2.0", default-features = false }

discv5 = { git = "https://github.com/sigp/discv5", rev = "f289bbd4c57d499bb1bdb393af3c249600a1c662" }
discv5 = { version = "0.3" }
igd = { git = "https://github.com/stevefan1999-personal/rust-igd", rev = "c2d1f83eb1612a462962453cb0703bc93258b173" }

# js
Expand Down Expand Up @@ -213,6 +214,7 @@ once_cell = "1.17"
syn = "2.0"
ahash = "0.8.6"
nybbles = "0.1"
smallvec = "1.13.1"

# proc-macros
proc-macro2 = "1.0"
Expand Down
3 changes: 3 additions & 0 deletions crates/net/discv4/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,9 @@ parking_lot.workspace = true
rand = { workspace = true, optional = true }
generic-array = "0.14"
serde = { workspace = true, optional = true }
itertools.workspace = true
smallvec.workspace = true
derive_more.workspace = true

[dev-dependencies]
rand.workspace = true
Expand Down
8 changes: 7 additions & 1 deletion crates/net/discv4/src/error.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
//! Error types that can occur in this crate.

use tokio::sync::{mpsc::error::SendError, oneshot::error::RecvError};
use tokio::sync::{mpsc::error::SendError, oneshot::error::RecvError, watch};

/// Error thrown when decoding a UDP packet.
#[derive(Debug, thiserror::Error)]
Expand Down Expand Up @@ -36,6 +36,12 @@ pub enum Discv4Error {
/// Failed to receive a command response
#[error(transparent)]
Receive(#[from] RecvError),
/// Failed to receive state change update.
#[error("failed to receive update on state change of primary kbuckets")]
MirrorUpdateFailed(watch::error::RecvError),
/// Decoding a packet received over the network failed.
#[error(transparent)]
DecodePacketError(#[from] DecodePacketError),
}

impl<T> From<SendError<T>> for Discv4Error {
Expand Down
Loading