LizardByte
Split-screen to multiple monitors #56
Replies: 0 comments 28 replies
-
|
I love this idea, the aspect ratio on the receiving end could be wonky. This touches on a lot of thing I would like sunshine to do. The easiest way to do what you ask is to send the full output to two moonlight clients So how would that work, does sunshine maintain multiple independent TCP video streams, or does the sunshine client send the video as a single UDP broadcast or multicast over the network ? How does multiple moonlight clients send controller input back to sunshine ? Maybe multiple client don't bother at all with controllers ? As for each player seeing only their screen, maybe moonlight client could have a blackout option where the server or client can specify area coordinates that are to be kept black ? What about live-cropping the incoming video so it stretches to more of the video (losing some of the aspect ratio and some pixels like "pan and scan" or overscan video ?) And the holy grail of this is, one gaming PC, streaming two full monitors (windows indirect displays, no physical monitors required) to two clients. One pc two gamers situation, this is certainly the future ! |
Beta Was this translation helpful? Give feedback.
-
|
I usually play splitscreen on the ultrawide - aspect ratio is not an issue in modern games. Moonlight could implement this by having an option to only show part of the multicast stream. When multicasting (i only tried this with parsec) input isn't a problem - controllers can be connected from multiple clients and works just fine. It would probably just work if moonlight implemented a left/right 50% solution, actually. Having this serverside would require two sessions, two times the encryption, etc. |
Beta Was this translation helpful? Give feedback.
-
|
moonlight-stream/moonlight-android#1104 Posted in the moonlight-github instead. |
Beta Was this translation helpful? Give feedback.
-
|
If you can already make the game work in 3840x1080, for now you could try with just plain ffmpeg like this ffmpeg -f gdigrab -framerate 30 -video_size 1920x1080 -offset_x 0 -offset_y 0 -i desktop -vcodec hevc_nvenc -f mpegts srt://:6666?mode=listener On the receiving computers ffplay -fs -f mpegts srt://yourserveriphere:6666?mode=caller However latency is probably too high, you need to reduce the buffers and use a directshow grabber instead of gdi grabber but I can't figure out how sunshine sets up ffmpeg to do this |
Beta Was this translation helpful? Give feedback.
-
|
pairing what you posted, with this: https://sunshinestream.readthedocs.io/en/latest/about/advanced_usage.html#channels - might be a solution. |
Beta Was this translation helpful? Give feedback.
-
|
it's bedtime in my end - it might be possible to just setup two channels, kill ffmpeg and start it again manually as you described. |
Beta Was this translation helpful? Give feedback.
-
But are you, actually? Not sure why it should be up to the streaming server to handle this. |
Beta Was this translation helpful? Give feedback.
-
|
I did not know about nucleus, thank you. Nucleus solves a different problem though. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, but.. I still have to understand what you are trying to propose here. |
Beta Was this translation helpful? Give feedback.
-
|
You would want sunshine to treat each splits of the screen as a different screen Then have moonlight pick which screen to display Then maybe install the amyuni indirect display driver ( https://www.amyuni.com/forum/viewtopic.php?t=3030 ) Maybe it could help to use fancyzone to subdivide the monitor into two monitor at the location of the in game split This is a narrow use case but there is a broader use case here, multi monitor streaming I have not tried it yet, but I suspect it's hard to run more than one instance of sunshine at a time on the same computer. Imagine a simple setup with 3 physical display. In Sunshine you can choose one display to stream. In moonlight you choose one computer to stream from, as far as I know you can't change the streaming source monitor from moonlight In an ideal world, Sunshine would let you, choose which of one or multiple monitor you want to stream. Sunshine should understand "sub" monitors created by fancy zones, either as independant monitor, or as "logical monitors" In the backend, it's not clear how to do this. Do you screenscrab every requested monitor at once, use one hardware accelerated encoder to compress the entire thing and then server the stream to the network and let the moonlight client crop only what they want to see ? Or, do you multiple screengrab process, each grabbing one monitor or sub monitor at a time, each using a separate hardware encoder process and moonlight only has to choose which monitor it wants to see ? Here for example my setup So the main monitor is connected directly with a hdmi cable, If Sunshine could do multiple monitors, I could replace spacedesk with sunshine And the OP could use the same thing to broadcast his split screen game to two separate monitors on the network |
Beta Was this translation helpful? Give feedback.
-
|
Good god, this is all over the place.. You basically want instead any-to-any monitor capabilities (including the host knowing and caring about the client topology).
And this is not even the craziest part. Faking monitors to Windows is a herculean task (yes took me three days to collect all that info). |
Beta Was this translation helpful? Give feedback.
-
|
It might just be a stupid midnight thought. We were playing Gears5 on an ultrawide monitor, we had full 16:9 aspect ratio for each splitscreen player. I ended up bying two copies of Gears, but i still think some split-screen games could benefit from that ability to have a TV for each player in seperate rooms. |
Beta Was this translation helpful? Give feedback.
-
|
Hey, is there any update on this? I am trying to multicast my screen, and your comments are the only things that remotely work, though it seems some of the command line options have changed in the latest ffmpeg. |
Beta Was this translation helpful? Give feedback.
-
|
I can tell you that in the sunshine documentation, when it talks about "multicast" it means multiple concurrent identical streams. But there is no way to do an "open loop" multicast (meaning, no bitrate scaling or any communication from a receiving client). It could be done, it's a core ffmpeg feature, but these controls are not broken out in sunshine. |
Beta Was this translation helpful? Give feedback.
-
Sunshine/Moonlight is not for watching screens. It's for streaming AND controlling them. The difficult part to solve would be sending input to separate applications. |
Beta Was this translation helpful? Give feedback.
-
|
Certainly that would be great to have multiple client controlling multiple different applications. The simple case to imagine is a dual monitor setup. As far as I know moonlight can't stream two monitors with a single client. And already there's two different use cases, You could also have one laptop plus a tablet, each a separate moonlight client. Only the laptop cares about sending input back. Most base operating system do not have a concept of focusing more than one application, so trying to get multiple clients interacting with multiple applications on the same host, is simply too science-fiction for the current state of computing. Even having two mouse connected to one computer is basically Star Trek level technology. I think X11 has support for it, but wayland had a feature regression on that one. In my earlier case I was hoping for multicast, in which there is no client proper, just a transmission to the network with no designated receivers. The ffmpeg commands above permit this but not with the low latency of sunshine because the filter stack is too primitive compared with sunshine. Essentially there might be a stub moonlight client just triggering the sunshine server to transmit toward 239.0.0.1, then any number of receiver could display the signal. In my case I was using that to mirror my screens into the bedroom and kitchen, so I could keep an eye on things or keep watching my video when I was cooking and stuff. As far as I'm concerned, moonlight is for conveniently streaming video with extreme low latency over the network. To and from any monitor, for any purpose and to stream user input controls in the other direction when possible. I've seen many other post that sunshine isn't for this or that, but Sunshine is VNC RDP SPICE pcanywhere bomgar teamviewer logmein chromecast and any number of app that all fundamentally do the same thing. Sunshine is just the best version of all of these and open source. It's so close to being everything one could want from any of these applications. Ah, just had an idea for multiple client controlling multiple independent applications, |
Beta Was this translation helpful? Give feedback.
-
|
@shodanx2 I'm kinda torn on it. Moonlight IS a GameStreaming tool, meaning, interacting remotely with games, that's how it started. And Sunshine was born as a host for it. But I do agree that it has potential outside of it, and it would be hypocrite of me, if I didn't agree, as I'm rarely using Moonlight/Sunshine for gaming, but using it daily as a productivity tool. And I also want extra flexibility on clients, streams and other aspects of Sunshine.
It's not, but I do understand why it feels like this, and I kinda use it like this too. A great open-source VNC project is RustDeck (if you don't know, maybe give it a try), but at the same time, it wasn't meeting my needs too, so here I am. But some feature requests, sound to me are better suited as new moonlight clients, or even a potential new wave of clients that could be it's own thing. For example:
That would be a good proposition to a standalone "Something Multicast Client", maybe Moonlight-Multicast, maybe an enterely unrelated project, that supports connecting to Sunshine instance and other moonlight-protocol server instances, and multicast it using adequate protocols for multicasting with low bandwidth. But I don't think anything has to be done on Sunshine, or even Moonlight, for it to exist. It feels like a separated entity to me.
I am currently using a setup where I primarily use my Sunshine host to work, where I created a virtual extended monitor, that I use Moonlight to see in another device, so that I can have multiple monitors working, and I'm using it as a productivity tool, not gaming. So, using a tool to create the virtual monitor, and adding another display, is kinda possible, but not possible to cast 2 monitors, into 2 other devices yet. With that said, I think devices like Apple Vision Pro may benefit of multi-screen streams too, so not only one to many clients, but many streams to a single client, may be useful, again, as a productivity tool. So while I think some quality of life features may land on Moonlight, I have a feeling that productivity tools, maybe would be better with a new client for productivity that's not Moonlight, or at least, Moonlight having a new "Productivity Mode" where it's explicit to the users, that's not for gaming, and Sunshine may adapt to include support for it, but I don't know if the maintainers have interest on it, and someone has to start the new clients or proposing how a "Productivity Mode" would make sense to Moonlight. |
Beta Was this translation helpful? Give feedback.
-
|
Multiple applications was never a thing.One application.Half the screen to each client.One client sends to controller 1And another client sends to controller 224. maj 2024 09.33 skrev shodanx2 ***@***.***>:
@ReenigneArcher
Certainly that would be great to have multiple client controlling multiple different applications.
But that is a whole other layer of complexity.
Just streaming multiple different monitor at once would be great.
The simple case to imagine is a dual monitor setup.
As far as I know moonlight can't stream two monitors with a single client.
And already there's two different use cases,
You could also have one laptop plus a tablet, each a separate moonlight client. Only the laptop cares about sending input back.
Most base operating system do not have a concept of focusing more than one application, so trying to get multiple clients interacting with multiple applications on the same host, is simply too science-fiction for the current state of computing.
Even having two mouse connected to one computer is basically Star Trek level technology. I think X11 has support for it, but wayland had a feature regression on that one.
In my earlier case I was hoping for multicast, in which there is no client proper, just a transmission to the network with no designated receivers. The ffmpeg commands above permit this but not with the low latency of sunshine because the filter stack is too primitive compared with sunshine.
Essentially there might be a stub moonlight client just triggering the sunshine server to transmit toward 239.0.0.1, then any number of receiver could display the signal.
In my case I was using that to mirror my screens into the bedroom and kitchen, so I could keep an eye on things or keep watching my video when I was cooking and stuff.
As far as I'm concerned, moonlight is for conveniently streaming video with extreme low latency over the network. To and from any monitor, for any purpose and to stream user input controls in the other direction when possible.
I've seen many other post that sunshine isn't for this or that, but Sunshine is VNC RDP SPICE pcanywhere bomgar teamviewer logmein chromecast and any number of app that all fundamentally do the same thing.
Sunshine is just the best version of all of these and open source. It's so close to being everything one could want from any of these applications.
Ah, just had an idea for multiple client controlling multiple independent applications,
On windows with https://github.com/stascorp/rdpwrap, I believe you could put each client into it's own "terminal service" so they'd essentially have each their own desktop, focus and mouse cursor.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
Multiple separate inputs to a single computer is pretty much a cakewalk on linux AFAIK
But if this was request, then the title has nothing to do with anything. |
Beta Was this translation helpful? Give feedback.
-
|
While there are this open-source project that does it, it by no means looks like a cakewalk endeavour to me, but I'd say that I think in that department, a possible interesting feature, to vote for support in Sunshine, would be instead of sending the input to the device, be able to assign a controller to an output channel, that other application could manually register to consume on the host and use as it pleases. What I mean by that, this way, splitscreenme-nucleus that has the purpose of allowing for splitscreen on games that don't support it, could then listen to a "Sunshine-controller" from each connected client and use it accordingly. So Sunshine could launch it, provide the controller channel, and it would then do it's thing behind. But if Sunshine ever creates a feature like that, splitscreenme-nucleus, or another similar tool, would still need to add support to this new "Sunshine-controller interface", whatever that may look like. I don't know, at the moment, how much of an endeavour that would be to start at this point, but I think it's an interesting feature to allow for some external integration.
The game already supports local split-screen on a single PC? Or we are talking about a game that don't support it? From your previous message, my understanding is that it doesn't support. If that's the case, you NEED 2 applications running, and for Windows, splitscreenme-nucleus purpose is to solve that, idk if there's something for linux. If you use Windows, at the moment, you could use VirtualHere to connect your controllers, then configure the controllers in nucleus, launch your game and then connect with moonlight clients to see the split-screen. But if the game, already supports local split-screen, what I understand from your request, would be that each Moonlight Client, be recognized as a different controller input, and to stream only portions of the screen for each, right? The game already supports local split-screen, or not? They are entirely different requests. |
Beta Was this translation helpful? Give feedback.
-
|
The idea was to cast a split screen game to two separate clients.I might have been unclear 24. maj 2024 15.22 skrev mirh ***@***.***>:
Multiple separate inputs to a single computer is pretty much a cakewalk on linux AFAIK
And even on windows.. you don't really need mices, as I already hinted above: https://github.com/SplitScreen-Me/splitscreenme-nucleus
Multiple applications was never a thing.One application.Half the screen to each client.One client sends to controller 1And another client sends to controller
But if this was request, then the title has nothing to do with anything.
You are just asking for multiple clients to be able to connect to the host.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
The game supports splitscreen.I was only talking about local wide splitscreen streamed to two separate tv's.Modern split screen games support 2x 16:9 left/right 24. maj 2024 19.01 skrev Vithorio Polten ***@***.***>:
While there are this open-source project that does it, it by no means looks like a cakewalk endeavour to me, but I'd say that I think in that department, a possible interesting feature, to vote for support in Sunshine, would be instead of sending the input to the device, be able to assign a controller to an output channel, that other application could manually register to consume on the host and use as it pleases.
What I mean by that, this way, splitscreenme-nucleus that has the purpose of allowing for splitscreen on games that don't support it, could then listen to a "Sunshine-controller" from each connected client and use it accordingly. So Sunshine could launch it, provide the controller channel, and it would then do it's thing behind.
But if Sunshine ever creates a feature like that, splitscreenme-nucleus, or another similar tool, would still need to add support to this new "Sunshine-controller interface", whatever that may look like. I don't know, at the moment, how much of an endeavour that would be to start at this point, but I think it's an interesting feature to allow for some external integration.
@lurendrejer
Right now, we are often "forced" to by two copies of the same game to play local multiplayer.
Multiple applications was never a thing.One application.Half the screen to each client.One client sends to controller 1And another client sends to controller 2
The game already supports local split-screen on a single PC? Or we are talking about a game that don't support it? From your previous message, my understanding is that it doesn't support. If that's the case, you NEED 2 applications running, and for Windows, splitscreenme-nucleus purpose is to solve that, idk if there's something for linux. If you use Windows, at the moment, you could use VirtualHere to connect your controllers, then configure the controllers in nucleus, launch your game and then connect with moonlight clients to see the split-screen.
But if the game, already supports local split-screen, what I understand from your request, would be that each Moonlight Client, be recognized as a different controller input, right?
The game already supports local split-screen, or not? They are entirely different requests.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
Ok, got it. I added that to my research backlog. |
Beta Was this translation helpful? Give feedback.
-
|
@ReenigneArcher I'm thinking that maybe we could split this split-screen discussion, into new smaller discussions, I'd propose the following discussion categories, and redirect people to each one for further comments, what do you think?
|
Beta Was this translation helpful? Give feedback.
-
|
Integration with nucleus seems like a killer feature, but putting aside this high-hanging fruit I'd say it's:
|
Beta Was this translation helpful? Give feedback.
-
|
Yes it should be split For this discussion, split monitor gaming, I think it should happen in the moonlight client. The client would allow the user to set named crop zone (main, player 1, player 2 etc..) and have a keyboard shortcut to switch between the zones. I don't think the sunshine server needs to do anything to accomodate this. As for multi monitor, it is already almost there. There is a workaround that appears to work but I have not yet tried it. Running multiple sunshine instances. For IP multicast, if the sunshine could be convinced to send the stream to 239.0.0.1 and just maintain a constant bitrate, that should be enough to allow multiple client to just playback the stream directly. Maybe ffplay could playback the stream itself. |
Beta Was this translation helpful? Give feedback.
-
Hi, was just playing a split-screen game and thought to myself:
Would it be viable for Sunshine to present two 'outputs' ?
If one television could connect to the left or right side of the stream, split-screen multiplayer could be routed to different rooms.
Right now, we are often "forced" to by two copies of the same game to play local multiplayer.
Beta Was this translation helpful? Give feedback.
All reactions