Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

security: Apply security best practices #318

Merged
merged 4 commits into from
May 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,43 @@ updates:
interval: "weekly"
commit-message:
prefix: "chore"

- package-ecosystem: docker
directory: /docker/kaito
schedule:
interval: daily

- package-ecosystem: docker
directory: /docker/presets/inference/llama-2
schedule:
interval: daily

- package-ecosystem: docker
directory: /docker/presets/inference/tfs-onnx
schedule:
interval: daily

- package-ecosystem: docker
directory: /docker/presets/inference/tfs
schedule:
interval: daily

- package-ecosystem: docker
directory: /docker/presets/tuning
schedule:
interval: daily

- package-ecosystem: gomod
directory: /
schedule:
interval: daily

- package-ecosystem: pip
directory: /presets/inference/text-generation
schedule:
interval: daily

- package-ecosystem: pip
directory: /presets/tuning/tfs
schedule:
interval: daily
9 changes: 8 additions & 1 deletion .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ on:
schedule:
- cron: "0 7 * * 1" # Mondays at 7:00 AM

permissions:
contents: read

jobs:
analyze:
name: Analyze
Expand All @@ -17,6 +20,11 @@ jobs:
actions: read

steps:
- name: Harden Runner
uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0
with:
egress-policy: audit

- name: Checkout repository
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
with:
Expand All @@ -33,4 +41,3 @@ jobs:

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@cdcdbb579706841c47f7063dda365e292e5cad7a

4 changes: 2 additions & 2 deletions .github/workflows/create-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ jobs:
egress-policy: audit

- name: Set up Go ${{ env.GO_VERSION }}
uses: actions/setup-go@v5
uses: actions/setup-go@0c52d547c9bc32b1aa3301fd7a9cb496313a4491 # v5.0.0
with:
go-version: ${{ env.GO_VERSION }}

Expand All @@ -33,7 +33,7 @@ jobs:
ref: ${{ github.event.client_payload.tag }}

- name: Goreleaser
uses: goreleaser/goreleaser-action@v5
uses: goreleaser/goreleaser-action@7ec5c2b0c6cdda6e8bbb49444bc797dd33d74dd8 # v5.0.0
with:
version: latest
args: release --rm-dist --timeout 60m --debug
Expand Down
27 changes: 27 additions & 0 deletions .github/workflows/dependency-review.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Dependency Review Action
#
# This Action will scan dependency manifest files that change as part of a Pull Request,
# surfacing known-vulnerable versions of the packages declared or updated in the PR.
# Once installed, if the workflow run is marked as required,
# PRs introducing known-vulnerable packages will be blocked from merging.
#
# Source repository: https://github.com/actions/dependency-review-action
name: 'Dependency Review'
on: [pull_request]

permissions:
contents: read

jobs:
dependency-review:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0
with:
egress-policy: audit

- name: 'Checkout Repository'
uses: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744 # v3.6.0
- name: 'Dependency Review'
uses: actions/dependency-review-action@0efb1d1d84fc9633afcdaad14c485cbbc90ef46c # v2.5.1
6 changes: 3 additions & 3 deletions .github/workflows/e2e-preset-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,10 @@ jobs:
full_matrix: ${{ steps.images.outputs.full_matrix }}
steps:
- name: Checkout
uses: actions/checkout@v4
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
with:
submodules: true
fetch-depth: 0
submodules: true
fetch-depth: 0

# This script should output a JSON array of model names
- name: Determine Affected Models
Expand Down
7 changes: 6 additions & 1 deletion .github/workflows/helm-chart.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,11 @@ jobs:
publish-helm:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0
with:
egress-policy: audit

- name: Checkout
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
with:
Expand All @@ -24,7 +29,7 @@ jobs:
ref: ${{ github.event.client_payload.tag }}

- name: Publish Workspace Helm chart
uses: stefanprodan/[email protected]
uses: stefanprodan/helm-gh-pages@0ad2bb377311d61ac04ad9eb6f252fb68e207260 # v1.7.0
with:
token: ${{ secrets.GITHUB_TOKEN }}
charts_dir: charts/kaito
Expand Down
5 changes: 4 additions & 1 deletion .github/workflows/lint-go.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@ on:
env:
GO_VERSION: '1.22'

permissions:
contents: read

jobs:
build:
runs-on: ubuntu-latest
Expand All @@ -32,7 +35,7 @@ jobs:
fetch-depth: 0

- name: Set up Go ${{ env.GO_VERSION }}
uses: actions/setup-go@v5
uses: actions/setup-go@0c52d547c9bc32b1aa3301fd7a9cb496313a4491 # v5.0.0
with:
go-version: ${{ env.GO_VERSION }}

Expand Down
20 changes: 14 additions & 6 deletions .github/workflows/markdown-link-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,26 @@ name: markdown link
on:
pull_request:
paths:
- '**.md'
- 'docs/**'
- "**.md"
- "docs/**"

permissions:
contents: read

jobs:
markdown-link-check:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0
with:
egress-policy: audit

- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
- uses: gaurav-nelson/github-action-markdown-link-check@v1
- uses: gaurav-nelson/github-action-markdown-link-check@5c5dfc0ac2e225883c0e5f03a85311ec2830d368 # v1
with:
# this will only show errors in the output
use-quiet-mode: 'yes'
use-quiet-mode: "yes"
# this will show detailed HTTP status for checked links
use-verbose-mode: 'yes'
config-file: '.github/markdown.links.config.json'
use-verbose-mode: "yes"
config-file: ".github/markdown.links.config.json"
7 changes: 6 additions & 1 deletion .github/workflows/pr-title-lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,12 @@ jobs:
check:
runs-on: ubuntu-latest
steps:
- uses: thehanimo/[email protected]
- name: Harden Runner
uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0
with:
egress-policy: audit

- uses: thehanimo/pr-title-checker@1d8cd483a2b73118406a187f54dca8a9415f1375 # v1.4.2
with:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
pass_on_octokit_error: true
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/preset-image-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,9 @@ jobs:
outputs:
matrix: ${{ steps.affected_models.outputs.matrix }}
is_matrix_empty: ${{ steps.check_matrix_empty.outputs.is_empty }}
steps:
steps:
- name: Checkout
uses: actions/checkout@v4
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
with:
submodules: true
fetch-depth: 0
Expand Down Expand Up @@ -82,7 +82,7 @@ jobs:
max-parallel: 3
steps:
- name: Checkout
uses: actions/checkout@v4
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
with:
submodules: true
fetch-depth: 0
Expand Down
7 changes: 6 additions & 1 deletion .github/workflows/publish-gh-image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,13 @@ jobs:
run: |
echo "tag=$(echo ${{ github.event.inputs.release_version }})" >> $GITHUB_OUTPUT

- name: Harden Runner
uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0
with:
egress-policy: audit

- name: Checkout
uses: actions/checkout@v4
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
with:
fetch-depth: 0

Expand Down
18 changes: 9 additions & 9 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -1,27 +1,27 @@
name: unit-tests

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true

on:
push:
branches: [main]
paths-ignore: ['docs/**', '**.md', '**.mdx', '**.png', '**.jpg']
paths-ignore: ["docs/**", "**.md", "**.mdx", "**.png", "**.jpg"]
pull_request:
branches: [main, release-**]
paths-ignore: ['docs/**', '**.md', '**.mdx', '**.png', '**.jpg']
paths-ignore: ["docs/**", "**.md", "**.mdx", "**.png", "**.jpg"]

permissions:
contents: read
packages: write

env:
GO_VERSION: '1.22'
GO_VERSION: "1.22"

jobs:
unit-tests:
runs-on: ubuntu-latest
runs-on: ubuntu-latest
environment: unit-tests
steps:
- name: Harden Runner
Expand All @@ -35,8 +35,8 @@ jobs:
submodules: true
fetch-depth: 0

- name: Set up Go ${{ env.GO_VERSION }}
uses: actions/setup-go@v5
- name: Set up Go ${{ env.GO_VERSION }}
uses: actions/setup-go@0c52d547c9bc32b1aa3301fd7a9cb496313a4491 # v5.0.0
with:
go-version: ${{ env.GO_VERSION }}

Expand All @@ -49,7 +49,7 @@ jobs:
make inference-api-e2e

- name: Upload Codecov report
uses: codecov/codecov-action@v4
uses: codecov/codecov-action@54bcd8715eee62d40e33596ef5e8f0f48dbbccab # v4.1.0
with:
## Comma-separated list of files to upload
files: ./coverage.txt
Expand Down
22 changes: 22 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
repos:
- repo: https://github.com/gitleaks/gitleaks
rev: v8.16.3
hooks:
- id: gitleaks
- repo: https://github.com/golangci/golangci-lint
rev: v1.52.2
hooks:
- id: golangci-lint
- repo: https://github.com/jumanjihouse/pre-commit-hooks
rev: 3.0.0
hooks:
- id: shellcheck
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
hooks:
- id: end-of-file-fixer
- id: trailing-whitespace
- repo: https://github.com/pylint-dev/pylint
rev: v2.17.2
hooks:
- id: pylint
2 changes: 1 addition & 1 deletion docker/kaito/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ RUN --mount=type=cache,target=${GOCACHE} \

# Use distroless as minimal base image to package the manager binary
# Refer to https://github.com/GoogleContainerTools/distroless for more details
FROM --platform=$BUILDPLATFORM gcr.io/distroless/static:nonroot
FROM --platform=$BUILDPLATFORM gcr.io/distroless/static:nonroot@sha256:55c636171053dbc8ae07a280023bd787d2921f10e569f3e319f1539076dbba11
WORKDIR /
COPY --from=builder /workspace/manager .
USER 65532:65532
Expand Down
2 changes: 1 addition & 1 deletion docker/presets/inference/llama-2/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# --build-arg VERSION={{VERSION}} \
# --build-arg MODEL_TYPE={{MODEL_TYPE}} \

FROM python:3.8-slim
FROM python:3.8-slim@sha256:95bfecec648356cdd0b28c8b00ce00009baff10c99d1126a82d1aca716453a1a
WORKDIR /workspace

# Install git
Expand Down
2 changes: 1 addition & 1 deletion docker/presets/inference/tfs/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM python:3.10-slim
FROM python:3.10-slim@sha256:684b1aaf96a7942b3c3af438d162e0baa3510aa7af25ad76d238e0c746bdec79

ARG WEIGHTS_PATH
ARG MODEL_TYPE
Expand Down
2 changes: 1 addition & 1 deletion docker/presets/tuning/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM python:3.10-slim
FROM python:3.10-slim@sha256:684b1aaf96a7942b3c3af438d162e0baa3510aa7af25ad76d238e0c746bdec79

ARG WEIGHTS_PATH
ARG MODEL_TYPE
Expand Down
Loading