Skip to content

Releases: kestra-io/kestra

v0.12.0

28 Sep 15:31
@github-actions github-actions
v0.12.0
592ec52
Compare
Choose a tag to compare
v0.12.0

Features

Bug Fixes

Read more
Assets 3
Loading

v0.11.2

27 Sep 08:38
@github-actions github-actions
v0.11.2
9aef21c
Compare
Choose a tag to compare
v0.11.2

Bug Fixes

Chores

Assets 3
Loading

v0.11.1

11 Sep 14:57
@github-actions github-actions
v0.11.1
fef619b
Compare
Choose a tag to compare
v0.11.1

Bug Fixes

Chores

Assets 3
Loading

v0.9.9

07 Sep 09:20
@github-actions github-actions
v0.9.9
fb51fed
Compare
Choose a tag to compare
v0.9.9

Bug Fixes

Chores

Assets 3
Loading

v0.11.0

30 Aug 12:03
@github-actions github-actions
v0.11.0
d49f8be
Compare
Choose a tag to compare
v0.11.0

Features

Bug Fixes

  • core: flaky labels tests #1765 (Loïc Mathieu)
  • ui: make the dark theme's .text-muted visible (#1779) #1779 (yuri)
  • core: serialization inclusion is a single property not a list #1766 (Loïc Mathieu)
  • ui: Render chars instead of entities (#1781) #1781 (yuri)
  • core: add predestroy to prevent Scheduler from crashing the whole app when context stops #1795 (brian.mulier)
  • core: acl control added on 'Use' buttons in BlueprintsBrowser & BlueprintDetail #1819 (brian.mulier)
  • blueprints: now loading icons from Kestra's API as well #1820 (brian.mulier)
  • ui: auto-refresh button is now animated for better feedback #1803 (brian.mulier)
  • core: can now override the way to retrieve registered plugins in JsonSchemaGenerator #1813 (brian.mulier)
  • doc-gen: now also displaying properties with @PluginProperty inside bases in json schema (including conditions) #1836 (brian.mulier)
  • remove deprecated #1828 (Loïc Mathieu)
  • tests: longer timeout int MultipleConditionTriggerCaseTest (brian.mulier)
  • core: WorkingDirectory validation throws an exception when tasks is null or empty #1838 (Loïc Mathieu)
  • deps: force bouncycastle version for crypto plugin compatibility (#1844) [#1844](https://gi...
Read more
Assets 3
Loading

v0.10.1

18 Jul 07:41
@github-actions github-actions
v0.10.1
f673a19
Compare
Choose a tag to compare
v0.10.1

Bug Fixes

Chores

Commits

  • e7e200e: docs fix wdir and API (#1730) (Anna Geller) #1730
  • 007142c: fix(core,webserver): store labels as a list instead of a map (Loïc Mathieu) #1743
  • 17d728d: Improve readability of the OpenAPI spec (yuri1969) #1754
Assets 3
Loading

v0.9.8

11 Jul 08:32
@github-actions github-actions
v0.9.8
9260817
Compare
Choose a tag to compare
v0.9.8

Bug Fixes

  • CI: Set fixed plugin version to latest 0.9.X (YannC)

Chores

  • version: update to version 'v0.9.8'. (YannC)
Assets 3
Loading

v0.9.7

10 Jul 14:26
@github-actions github-actions
v0.9.7
32ce62c
Compare
Choose a tag to compare
v0.9.7

Bug Fixes

  • jdbc: DateTimeFormatter can be reused in the JdbcMapper (Loïc Mathieu)

Chores

  • version: update to version 'v0.9.7'. (YannC)

Commits

  • fix: Updated count variable (YannC)
Assets 3
Loading

v0.10.0

06 Jul 12:58
@github-actions github-actions
v0.10.0
bd93937
Compare
Choose a tag to compare
v0.10.0

Kestra Release Note - 0.10.0

This new release brings important new core features such as Blueprints, new Script plugins, integration of basic authentication, and a secret function making the Open Source Edition even more robust.

Features

Blueprints

Blueprints are a curated, organized, and searchable catalog of ready-to-use examples designed to help you kick-start your workflow.

It’s available as a new section on the left bar menu and directly within the editor view**. All blueprints are validated and documented so that they just work.

You can easily customize and integrate them into your new or existing flows with a single click on the “Use” button.

Check out the documentation.

Untitled

Untitled2

Enterprise Edition users could write their own organization blueprints so they can share and grow a library of internal blueprints within teams.
Untitled3

Improved Support for Scripting

In this release, we revamped scripting tasks to bring more flexibility and control.
Each script task is now, by default, running in its own dedicated container.You can attach any Docker image you want if you need a specific one.
New tasks are available for Shell, Python, R, and Node.js:

  • Script: to write and run ad-hoc scripts. It comes with a beforeCommands property to execute any instruction needed before running the main script (installing dependencies for example)
    id: script
    namespace: release
    
    tasks:
      - id: run_python
        type: io.kestra.plugin.scripts.python.Script
        beforeCommands:
          - pip install requests
        warningOnStdErr: false
        script: | 
          import requests
          import json
    
          response = requests.get("https://api.github.com")
          data = response.json()
          print(data)
  • Command: to run arbitrary commands in a single task configuration. This task can be very powerful and associated with the WorkingDirectory task. For example, to clone a Git repository and then execute the corresponding scripts.
    id: command
    namespace: release
    
    tasks:
      - id: working
        type: io.kestra.core.tasks.flows.WorkingDirectory
        tasks:
    
          - id: clone_repo
            type: io.kestra.plugin.git.Clone
            url: https://github.com/your_repository
    
          - id: run_python
            type: io.kestra.plugin.scripts.python.Command
            beforeCommands:
              - pip install -r requirements.txt
        dockerOptions:
        	    image: ghcr.io/kestra-io/pydata:latest
            commands:
              - python main.py

  • LocalFile: to create files in the local filesystem or to send files from the local filesystem to the internal storage. This task allows to uncouple inline scripts to their execution.

        id: "local-file"
    namespace: release
    tasks:
      - id: workingDir
        type: io.kestra.core.tasks.flows.WorkingDirectory
        tasks:
        - id: inputFiles
          type: io.kestra.core.tasks.storages.LocalFiles
          inputs:
            hello.txt: "Hello World"
            address.json: "{{ outputs.myTaskId.uri }}"
        - id: bash
          type: io.kestra.plugin.scripts.shell.Command
          commands:
            - cat hello.txt
    	- sed -n 's/.*"country":"\([^"]*\)".*/\1/p' address.json

It also allows to expose outputs to internal storage. In the following example, we create two files with Bash commands and expose those into Kestra internal storage with the outputs property LocalFiles

    id: "local-files"
    namespace: release
    
    tasks:
      - id: workingDir
        type: io.kestra.core.tasks.flows.WorkingDirectory
        tasks:
        - id: bash
          type: io.kestra.plugin.scripts.shell.Command
          commands:
            - mkdir -p sub/dir
            - echo "Hello from Bash" >> sub/dir/bash1.txt
            - echo "Hello from Bash" >> sub/dir/bash2.txt
        - id: outputFiles
          type: io.kestra.core.tasks.storages.LocalFiles
          outputs:
            - sub/**

Those tasks run by default on DOCKER but you can use the runner: PROCESS property to run it as a process on the Kestra host.

Note: the old scripting tasks w ill be deprecated, removed from the core and being retro compatible within the new plugins.

DAG task

Creating directed acyclic graphs, a common pattern in data orchestration, was already possible in Kestra by through Flow dependencies. With a brand new DAG task, it’s now even easier to do; directly between tasks at the Flow level.

id: magicDAG
namespace: dev
tasks:
  - id: dag
    type: io.kestra.core.tasks.flows.Dag
    tasks:
      - task:
          id: customers
          type: io.kestra.plugin.fs.http.Download
          uri: https://raw.githubusercontent.com/dbt-labs/jaffle_shop/main/seeds/raw_customers.csv
      - task:
          id: orders
          type: io.kestra.plugin.fs.http.Download
          uri: https://raw.githubusercontent.com/dbt-labs/jaffle_shop/main/seeds/raw_orders.csv
      - task:
          id: payments
          type: io.kestra.plugin.fs.http.Download
          uri: https://raw.githubusercontent.com/dbt-labs/jaffle_shop/main/seeds/raw_payments.csv
      - task:
          id: transform
          type: io.kestra.core.tasks.log.Log
          message: |
            Transforming data from: 
              {{outputs.customers.uri}}
              {{outputs.orders.uri}}
              {{outputs.payments.uri}}
        dependsOn:
          - customers
          - orders
          - payments

image

Label on executions

So far, it was only possible to add labels on a flow level by adjusting the workflow code. This release adds the ability to set custom labels for specific Executions.

Also, the labels added on a flow level will be automatically propagated to Execution labels.

For example you can add labels for “experiment” executions so you can retrieve them in the UI easily.
Untitled4

Screenshot 2023-07-05 at 15 43 45

Basic Authentication

The community was asking about authentication in the open source to secure their instance in production. We heard you ! You can now add basic authentication to your Kestra instance with username/password values in the Kestra configuration.

server:
  basic-auth:
    enabled: true
    username: admin
    password: *****

Secret Function

We introduced a secret() function allowing to read base64 secret value from environment variables.

id: secret
namespace: release
tasks:
  - id: get_secret
    type: io.kestra.core.tasks.debugs.Return
    format: '{{ secret("my_secret")}}'

Worker Group

This release introduce worker groups, a ****set of workers that can be targeted specifically for a task execution or a polling trigger evaluation. For this, the task or the polling trigger must define the workerGroup.key property with the key of the worker group to target. A default worker group can also be configured at the namespace level.

Here are common use cases in which Worker Groups can be beneficial:

  • Execute tasks and polling triggers on specific compute instances (e.g., a VM with a GPU and preconfigured CUDA drivers).
  • Execute tasks and polling triggers on a worker with a specific Operating System (e.g., a Windows server).
  • Restrict backend access to a set of workers (firewall rules, private networks, ...).
  • Execute tasks and polling triggers close to a remote backend (region selection).

Here is how you can ensure that a task is executed only by specific worker instances:

id: gpuTask
namespace: dev
tasks:
  - id: hello
    type: io.kestra.core.tasks.log.Log
    message: |
      This task will be executed on a specific remote worker that has access to a GPU
    workerGroup:
    key: gpuWorkerGroupKey

Polling Triggers

We have made significant improvements to the Polling trigger in the latest version of Kestra to enhance performance, strengthen security measures, simplify maintenance, and clarify the system architecture.
In previous versions, the trigger evaluation process was directly handled by the Scheduler. This meant that both the Worker and the Scheduler potentially accessed external systems, leading to manage the security and scalability of two services.

From now on, The Scheduler delegates the evaluation of polling triggers to the Worker. This architectural change has several advantages. Firstly, it simpli...

Read more
Assets 3
Loading

v0.9.5

20 Jun 13:12
@github-actions github-actions
v0.9.5
4a7dbbb
Compare
Choose a tag to compare
v0.9.5

Bug Fixes

  • ui: fix query issue in home component (YannC)

Chores

  • version: update snapshot version 'v0.9.5-SNAPSHOT'. (Ludovic DEHON)
  • version: update to version 'v0.9.5'. (YannC)
Assets 3
Loading