Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use somltcp #208

Open
ibigbug opened this issue Dec 7, 2023 · 36 comments
Open

Use somltcp #208

ibigbug opened this issue Dec 7, 2023 · 36 comments

Comments

@ibigbug
Copy link
Member

ibigbug commented Dec 7, 2023

No description provided.

This was referenced Dec 26, 2023
@cavivie
Copy link

cavivie commented Apr 10, 2024

@ibigbug I think I might be of some help here, as I recently stabilized the netstack-smoltcp crate.

@ibigbug
Copy link
Member Author

ibigbug commented Apr 10, 2024

yeah that'd be great. I think @VendettaReborn has been playing with it a bit lately, https://github.com/Watfaq/netstack-smoltcp/blob/main/examples/proxy.rs

and it seems it's feasible to use tun crate for the device setup and use smoltcp's stack to deal with layer 4 logic. so we can replace lwip.

is that something similar to what in your mind?

@cavivie
Copy link

cavivie commented Apr 10, 2024

yeah that'd be great. I think @VendettaReborn has been playing with it a bit lately, https://github.com/Watfaq/netstack-smoltcp/blob/main/examples/proxy.rs

and it seems it's feasible to use tun crate for the device setup and use smoltcp's stack to deal with layer 4 logic. so we can replace lwip.

is that something similar to what in your mind?

Yes, you are right. I would like to clarify why this is layer 4 and not layer 3. Currently netstack based on smoltcp can work in most scenarios of layer 3.

@cavivie
Copy link

cavivie commented Apr 10, 2024

Well, I misunderstood you. It seems that you mean tcp/udp.

@ibigbug
Copy link
Member Author

ibigbug commented Apr 10, 2024

yeah that's waht i meant

@ibigbug
Copy link
Member Author

ibigbug commented Apr 12, 2024

@cavivie were you planning to creat a PR for this?

@cavivie
Copy link

cavivie commented Apr 12, 2024

I'd love to, but before that, can you add more about tun in ss, including unresolved problems and known problems and goals.

@ibigbug
Copy link
Member Author

ibigbug commented Apr 12, 2024

I think the main goal is to replace the lwip w smoltcp, for performance boost, see https://github.com/Watfaq/netstack-smoltcp/blob/main/benchmark.md

And adopting pure rust impls over foreign C bindings.

This is only for the tun input, not much todo with ss.

We already use smoltcp in the user space wirefuard, it consolidates the deps which is another good reason too.

And thanks for you work.

@cavivie
Copy link

cavivie commented Apr 12, 2024

Thank you for your description. I think I will take time to integrate it recently.

@VendettaReborn
Copy link
Contributor

@cavivie @ibigbug i added some soem auto detect interface feature for rust-tun, and here is the example code: https://github.com/VendettaReborn/netstack-smoltcp/blob/f89b9e9b2e56a6abf1b2be9290e4cf63c7be3e11/examples/forward.rs#L142 . Basically, it's just a code version of route commands. And i've tested it on linux,macos & windows.

Shall we extract the logic of auto-detect interface, auto setup routes into a independent crate? just like sing-tun?

@ibigbug
Copy link
Member Author

ibigbug commented Apr 16, 2024

cc @Itsusinn

@Itsusinn
Copy link
Member

Itsusinn commented Apr 16, 2024

Shall we extract the logic of auto-detect interface, auto setup routes into a independent crate? just like sing-tun?

i would like call it watfaq-tun : )

It will combine both https://crates.io/crates/net-route and https://crates.io/crates/tun

@VendettaReborn
Copy link
Contributor

VendettaReborn commented May 14, 2024

any updates? cc @cavivie

@Itsusinn

This comment was marked as outdated.

@cavivie
Copy link

cavivie commented May 14, 2024

Ah, ah, first of all, I'm sorry for the reason why I've been busy recently. I want to disassemble the current steps here:
1.

Shall we extract the logic of auto-detect interface, auto setup routes into an independent crate? just like sing-tun?

Are there any disassembled independent clates here? I want to do some work based on this.

BTW, netstack-smoltcp may have some things that need to be changed. Can I think of anything else you need to add besides some refactoring of the udp part?

@VendettaReborn
Copy link
Contributor

VendettaReborn commented May 14, 2024

hh, it's ok, and hope you can rejoin it whenever you're ready.

for the first point, for example, the route's auto-configuration can be done via programming, here is a WIP version: (https://github.com/VendettaReborn/netstack-smoltcp/blob/auto-detect-interface/src/utils.rs), it can works in netstack-smoltcp's example, but need some extra work before adding it into clash-rs.

What's your idea?

@ibigbug
Copy link
Member Author

ibigbug commented May 14, 2024

i would like call it watfaq-tun : )

Or wontun 😄

@VendettaReborn
Copy link
Contributor

i've done some extra jobs on it:

  1. forward_lwip && forward_smoltcp. I compared smoltcp with lwip on their basic proxy mechanism, and found that the netstack-smoltcp may not work well on ipv6 (lwip is good)
  2. some work on adding ipv6 addr for tun device, which is missed in rust-tun
  3. watfaq-tun. code equivalent for ip route & 'ip rule' commands. But with supports of ipv6. Now works on linux&macos.

and watfaq-tun may need some reviews and comments. We can make it more robust and user-friendly before putting into usage in clash-rs.

cc @ibigbug @cavivie @Itsusinn

@ibigbug
Copy link
Member Author

ibigbug commented May 17, 2024

Awesome work! Thanks! Will take a look

@ibigbug
Copy link
Member Author

ibigbug commented May 20, 2024

@VendettaReborn i think we can start integrating your tun with #396 and improve it progressively, thoughts?

@Itsusinn
Copy link
Member

watfaq-tun. code equivalent for ip route & 'ip rule' commands. But with supports of ipv6. Now works on linux&macos.

I dont think we need net_route crate(at least on linux). It doesn't meet my requirement of impl auto-route(eg. no fwmark support).

@VendettaReborn
Copy link
Contributor

of course, global routing on linux is easy, your impl of fwmark is surely sufficient, but what about macos and windows? I don't expect most of our users to use linux.

@Itsusinn
Copy link
Member

of course, global routing on linux is easy, your impl of fwmark is surely sufficient, but what about macos and windows? I don't expect most of our users to use linux.

I could write my route impl on windows. But indeed i cant do the same for macos xD. So i would like take net-route as a platform-specific dep.

@cavivie
Copy link

cavivie commented May 25, 2024

forward_lwip && forward_smoltcp. I compared smoltcp with lwip on their basic proxy mechanism, and found that the netstack-smoltcp may not work well on ipv6 (lwip is good)

I think we may be further along, since ipv6 local routing is supported after this MR, I have successfully tested ipv6 ping in netstack-smoltcp using the latest main branch after this MR: smoltcp-rs/smoltcp#900

@cavivie
Copy link

cavivie commented May 25, 2024

For now, this means we can further work on IPv6 code. Nonetheless, this also seems to mean that the support of the ipv6 part of the smoltcp crate is not perfect enough. We may need to test more to find the problem. I am not sure what specific aspects we need to test for IPv6. Can you give me some advice?

@VendettaReborn
Copy link
Contributor

VendettaReborn commented May 25, 2024

for the outbound handlers, we have the docker test suites, which will cover tcp&udp. For tun, we haven't figure out some good ways to test it on both ipv4 & ipv6. here is a brief thought: #362 (comment)

@cavivie thoughts?

@cavivie
Copy link

cavivie commented May 25, 2024

Here is a pic that some of simple tcp tests shared. rust-tun does not currently support ipv6, I want to do something about rust-tun's shortcomings in ipv6 in the near future to avoid us do some hacky things about ipv6 for tun device at outside of rust-tun crate. What do you think?

image

@cavivie
Copy link

cavivie commented May 25, 2024

for the outbound handlers, we have the docker test suites, which will cover tcp&udp. For tun, we haven't figure out some good ways to test it on both ipv4 & ipv6. here is a brief thought: #362 (comment)

@cavivie thoughts?

Absolutely agree, usually routing is definitely an important point that hinders testing.

@VendettaReborn
Copy link
Contributor

yeah, i agree. rust-tun hasn't supported ipv6 yet, we can extend the ability and some other feature(like routing rules setup) in a separate crate.

Here is a pic that some of simple tcp tests shared. rust-tun does not currently support ipv6, I want to do something about rust-tun's shortcomings in ipv6 in the near future to avoid us do some hacky things about ipv6 for tun device at outside of rust-tun crate. What do you think?

image

@cavivie
Copy link

cavivie commented May 25, 2024

a separate crate

What I mean is that we may need to use the rust-tun patch first, and then merge some patches about ipv6 into rust-tun upstream if possible.

@VendettaReborn
Copy link
Contributor

VendettaReborn commented May 25, 2024

oh, i see, that's better.

@cavivie
Copy link

cavivie commented May 25, 2024

To add tun feature, we want rust-tun to behave like:

  • Dual stack is enabled by default, that is, ipv4/ipv6 addresses are bound to the tun device at the same time, and then a method is provided to disable ipv6 or ipv4;
  • IPv4 is enabled by default, IPv6 can be optionally added
  • IPv6 is enabled by default, IPv4 can be optionally added

Obviously, I think the first two are more reliable, and in particular, I think the second point is more needed. What do you think?

@VendettaReborn
Copy link
Contributor

that would be great

@Itsusinn Itsusinn mentioned this issue Jun 12, 2024
3 tasks
@cavivie
Copy link

cavivie commented Aug 13, 2024

After some time testing tun2 and various netstacks, I found that in terms of throughput:

  • Throughput (nix write): lwip > smoltcp > ipstack
  • Throughput (win write): lwip > smoltcp > ipstack
  • Throughput (nix read): smoltcp = lwip > ipstack
  • Throughput (win read): smoltcp > lwip > ipstack

In terms of stability, ipstack currently performs the worst, and even the throughput is unacceptable on win, about 0.2 to 0.3 Gbps., while the VPN scenario focuses more on netstack's read performance, followed by write performance, and finally stability.

The above data is only a personal test on a specific device. Later, I will publish a test routine for you to test and compare.

@cavivie
Copy link

cavivie commented Aug 13, 2024

@ibigbug
Copy link
Member Author

ibigbug commented Aug 14, 2024

Thanks for sharing.

To me I think it's still worth adopting smoltcp even with similar bar of performance to lwip for simplicity and pure rust impl.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants