Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Revisit use of multiple subscription sockets/threads #194

Open
dwsutherland opened this issue Mar 26, 2021 · 1 comment · May be fixed by #574
Open

Revisit use of multiple subscription sockets/threads #194

dwsutherland opened this issue Mar 26, 2021 · 1 comment · May be fixed by #574
Assignees
@dwsutherland
Labels
efficiency
Milestone
1.6.0

Comments

@dwsutherland
Copy link
Member

dwsutherland commented Mar 26, 2021 •
edited
Loading

One socket may have many outgoing and many incoming connections.

Found here:
https://zguide.zeromq.org/docs/chapter2/#Plugging-Sockets-into-the-Topology

And further down:

We said that ZeroMQ does I/O in a background thread. One I/O thread (for all sockets) is sufficient for all but the most extreme applications. When you create a new context, it starts with one I/O thread.

Can we reduce the need for multi-threaded UIS by using one subscription socket with multiple connections? (to every publisher/Scheduler)
@dwsutherland dwsutherland self-assigned this Mar 26, 2021
@oliver-sanders oliver-sanders added this to the cylc-uiserver 1.0.0 milestone Aug 2, 2021
This was referenced Mar 26, 2024
Open
Closed
oliver-sanders added a commit to oliver-sanders/cylc-uiserver that referenced this issue Mar 28, 2024
@oliver-sanders
data store: remove threading
4a19241 
* Closes cylc#194
* Run subscribers via asyncio rather than the ThreadPoolExecutor.
* The only non-blocking operations in the code being called were:
  - time.sleep  (has an async variant)
  - asyncio.sleep (async)
  - self.socket.recv_multipart (async)
* Refactored the code so that the underlying async functions could be
  called via asyncio.
oliver-sanders added a commit to oliver-sanders/cylc-uiserver that referenced this issue Mar 28, 2024
@oliver-sanders
data store: remove threading
2a0a092 
* Closes cylc#194
* Run subscribers via asyncio rather than the ThreadPoolExecutor.
* The only non-blocking operations in the code being called were:
  - time.sleep  (has an async variant)
  - asyncio.sleep (async)
  - self.socket.recv_multipart (async)
* Refactored the code so that the underlying async functions could be
  called via asyncio.
@oliver-sanders oliver-sanders linked a pull request Mar 28, 2024 that will close this issue
Draft
8 tasks
@oliver-sanders
Copy link
Member

I've had a crack at removing the thread pool and running the subscribers via asyncio, seems to work: #574

oliver-sanders added a commit to oliver-sanders/cylc-uiserver that referenced this issue Apr 22, 2024
@oliver-sanders
data store: remove threading
1970ef9 
* Closes cylc#194
* Run subscribers via asyncio rather than the ThreadPoolExecutor.
* The only non-blocking operations in the code being called were:
  - time.sleep  (has an async variant)
  - asyncio.sleep (async)
  - self.socket.recv_multipart (async)
* Refactored the code so that the underlying async functions could be
  called via asyncio.
oliver-sanders added a commit to oliver-sanders/cylc-uiserver that referenced this issue Apr 22, 2024
@oliver-sanders
data store: remove threading
c730b51 
* Closes cylc#194
* Run subscribers via asyncio rather than the ThreadPoolExecutor.
* The only non-blocking operations in the code being called were:
  - time.sleep  (has an async variant)
  - asyncio.sleep (async)
  - self.socket.recv_multipart (async)
* Refactored the code so that the underlying async functions could be
  called via asyncio.
@oliver-sanders oliver-sanders modified the milestones: pending, 1.5.0 Apr 23, 2024
@oliver-sanders oliver-sanders modified the milestones: 1.5.0, 1.6.0 May 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dwsutherland dwsutherland

Labels
efficiency
Projects
None yet
Milestone
1.6.0
Development

Successfully merging a pull request may close this issue.

data store: remove threading oliver-sanders/cylc-uiserver
2 participants
@dwsutherland @oliver-sanders