Sunshine + moonlight ALMOST make for a viable VNC replacement

[shodanx2]

Hi,

So I've been using sunshine + moonlight as a more pleasant VNC-like experience for various admin tasks.

Here are things that could help improve the user experience for this case

Most important
Sunshine installs as a system service
Sunshine runs as administrator to be able to click yes on UAC prompts
Moonlight allows connection using hostname/ip and password instead of the PIN mechanism
Moonlight client able to work as a resizeable window

medium important
Moonlight client able to choose which monitor is visible (live without needing reconnect preferably)
Systray server control app (context menu)

• double click icon opens browser to https://localhost:47990
• icon reflects server status (available, connected, error, paused, stopped)
• start/restart/stop/uninstall service
• change the password or enter client PIN -> https://localhost:47990/password
• change which display (until multi display streaming or client choosing display is an option)
• view server text console output -> https://localhost:47990/console (doesn't exist yet)
• disable remote control input (viewer only mode)
• view/kill/kick/ban clients
• "reverse connection" (server connects to a waiting client, allows connecting to NAT blocked servers without having to explain port sharing, this is a rarely used advanced function, but when you need it, you need it)

low important
Sunshine unattended installer flag including setting the password
Moonlight client installs unattended (might already be the case)
One click connecting to known client
Client auto-reconnects on disconnects
Client remembers password
Ability to create a connection like *.vnc files

now some references

VNC server control dialog

vnc server property page

Example of a *.vnc connection file

[ReenigneArcher]

Moonlight allows connection using hostname/ip and password instead of the PIN mechanism
Moonlight client able to work as a resizeable window
Moonlight client able to choose which monitor is visible (live without needing reconnect preferably)
Moonlight client installs unattended (might already be the case)

Moonlight is a separate project and is an open source implementation of GeForce Experience (GFE). Keeping that in mind, they are unlikely to implement protocols that are not allowed in GFE. There may be some exceptions such as allowing custom family of ports, but even this is only implemented in Moonlight QT, and not any other clients as far as I know.

Moonlight client installs unattended (might already be the case)

There's many options for this. If Android use ADB. If PC clients there are tons of options, too many to probably list here.

Sunshine installs as a system service

See #197

Sunshine runs as administrator to be able to click yes on UAC prompts

If running as a service, this is already the case. If not, right click, "run as admin". There are cases where users don't want Sunshine to run with elevated permissions, so this won't be added. You can also create a shortcut, where the shortcut launches as admin.

Systray server control app (context menu)

Feel free to open a PR. Remember Sunshine is cross platform (Linux, macOS, Windows) so anything added should be functional on all 3. From my experience system tray icons in Linux can be quite a pain depending on the desktop environment being used.

view server text console output

First step would probably be writing terminal output to a set of rotating log files. Again, PRs are welcome.

reverse connection

Pretty sure this one isn't possible with moonlight clients. Sorry.

Lot's of good ideas in your list. Thanks for the feedback.

[mirh]

Pretty sure this one isn't possible with moonlight clients. Sorry.

Meaning that's just some of the many limits of the original protocol, that it's not really something for the endpoints to handle, or what else?

[shodanx2]

I have been playing with video remote control apps for a while as part of various project and moonlight/sunshine have blown every other alternative out of the water in terms of performance (latency most of all, but also image quality and connection stability)

I tried ffmpeg on which this project appears to be based, but it seems there is no way to access the system that sunshine uses from the command line. Especially the framebuffer capture with gdi or directshow is very slow and cpu intensive.

I thought this was because of the nvidia hardware I was using, but in my latest test I used a 2018 toshiba laptop with intel graphic and it worked flawlessly.

My hope is to someday see a purpose made fork that keeps the underlying technology but just puts a different user interface skin on top.

VNC isn't the only use case, there is also another software called spacedesk which is just like sunshine except it creates an indirect display when a client connects and then serves this memory only display to the client. Result is I have a computer with only an ethernet and power and five monitors somewhere else on the network get their video feed over the network. Unfortunately they use software encoded MJPEG so bandwidth usage is very high (400mbps) and so is cpu usage (easily 20% of a ryzen 2700x). I have seen discussion somewhere about using amyuni indirect display driver to allow sunshine to create arbitrary display and stream them over network.

[ehfd]

Hi! @shodanx2 !
We have created a remote desktop protocol to replace noVNC and Apache Guacamole using WebRTC and GStreamer instead of FFmpeg.
https://github.com/selkies-project/selkies-gstreamer

We also love that Sunshine has opened up a valid open-source replacement for VNC and RDP, and we fully support the project with our hearts.
However, this project fills the void that Sunshine has; whereas it is not adequate for unprivileged containers (Games on Whales require privileged containers); situations that require web-based interfaces; and where only a single HTTP port works instead of multiple TCP ports (TURN servers however, are required for traversing NAT). We have created an implementation which does not require uinput.

We hope that you take a look at our project as well, even if our project does need a bit of refinement and documentation compared to Sunshine.
We would, perhaps, like to work with Sunshine in the future to leverage both of our strengths. After all, they both use RTSP and the video codecs under the hood.

Unprivileged container implementations based on Selkies-GStreamer adequate for Kubernetes:
https://github.com/ehfd/docker-nvidia-glx-desktop
https://github.com/ehfd/docker-nvidia-egl-desktop

[shodanx2]

Thank you @ehfd for bringing this to my attention.

However, for a VNC replacement the setup requirement are currently too onerous for casual user.
But I'm going to try it out of curiosity !

Sunshine runs as administrator to be able to click yes on UAC prompts

If running as a service, this is already the case. If not, right click, "run as admin". There are cases where users don't want Sunshine to run > with elevated permissions, so this won't be added. You can also create a shortcut, where the shortcut launches as admin.

@ReenigneArcher

I tried this weekend using sunshine 0.14, running sunshine as administrator and it could not see the UAC prompts (image just froze)

Systray server control app (context menu)

Feel free to open a PR. Remember Sunshine is cross platform (Linux, macOS, Windows) so anything added should be functional on all 3. From my experience system tray icons in Linux can be quite a pain depending on the desktop environment being used.

Perhaps an independent helper application could alleviate the need for cross platform support.
I can make a simple .net systray app that can change icon based on server status, and a context menu when you right click it that allows you to do the basic management functions from the systray.

I had a look at #197 but I saw nothing regarding the installation of a service when I installed 0.14 this weekend. Maybe I missed it ? I see this issue is closed but so far can't figure out how it works.

[ehfd]

However, for a VNC replacement the setup requirement are currently too onerous for casual user.

Our documentation is not finished, so we hope that out application becomes easy as VNC when it is completed.

Mostly ready for use.