Skip to content

wangdatan/yunikorn-core

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

YuniKorn - A Universal Scheduler

YuniKorn is a light-weighted, universal resource scheduler for container orchestrator systems. It was created to achieve fine-grained resource sharing for various workloads efficiently on a large scale, multi-tenant, and cloud-native environment. YuniKorn brings a unified, cross-platform scheduling experience for mixed workloads consists of stateless batch workloads and stateful services. Support for but not limited to, YARN and Kubernetes.

Architecture

Following chart illustrates the high-level architecture of YuniKorn.

Architecture

YuniKorn consists of the following components spread over multiple code repositories.

  • Scheduler core: the brain of the scheduler, which makes placement decisions (Allocate container X on node Y) according to pre configured policies. See more in current repo yunikorn-core.
  • Scheduler interface: the common scheduler interface used by shims and the core scheduler. Contains the API layer (with GRPC/programming language bindings) which is agnostic to container orchestrator systems like YARN/K8s. See more in yunikorn-scheduler-interface.
  • Resource Manager shims: allow container orchestrator systems talks to yunikorn-core through scheduler-interface. Which can be configured on existing clusters without code change. Currently, yunikorn-k8shim is available for Kubernetes integration.
  • Scheduler User Interface: the YuniKorn web interface for app/queue management. See more in yunikorn-web.

Key features

Here are some key features of YuniKorn.

  • Features to support both batch jobs and long-running/stateful services
  • Hierarchy queues with min/max resource quotas.
  • Resource fairness between queues, users and apps.
  • Cross-queue preemption based on fairness.
  • Customized resource types (like GPU) scheduling support.
  • Rich placement constraints support.
  • Automatically map incoming container requests to queues by policies.
  • Node partition: partition cluster to sub-clusters with dedicated quota/ACL management.

You can find more information about what are already supported and future plans in the Road Map.

How to use

The simplest way to run YuniKorn is to build a docker image and then deployed to Kubernetes with a yaml file, running as a customized scheduler. Then you can run workloads with this scheduler. See more instructions from the User Guide.

How can I get involved?

We welcome any form of contributions, code, documentation or even discussions. To get involved, please read following resources.

  • Before you contributing code or documentation to YuniKorn, please read our Developer Guide.
  • Please read How to Contribute to understand the procedure and guidelines of making contributions.

Other Resources

Demo videos

Communication channels

Blog posts

About

YuniKorn Scheduler core

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Go 99.6%
  • Makefile 0.4%