Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: model bad name can crash kserve #3062

Open
1 task done
Loumy opened this issue Aug 5, 2024 · 0 comments
Open
1 task done

[Bug]: model bad name can crash kserve #3062

Loumy opened this issue Aug 5, 2024 · 0 comments
Labels
kind/bug Something isn't working priority/normal An issue with the product; fix when possible untriaged Indicates the newly create issue has not been triaged yet

Comments

@Loumy
Copy link

Loumy commented Aug 5, 2024

Is there an existing issue for this?

  • I have searched the existing issues

Deploy type

Downstream version (eg. OpenShift AI 2.4)

Version

2.10.0

Current Behavior

When you setup a bad name for a new model to server, in dashbord, it allow you but in background kserve crash and restart each 30s.

Expected Behavior

Check the name validity in the dashboard

Steps To Reproduce

Try to run a model with a name like ibm-granite/granite-3b-code-instruct-accelerator

Workaround (if any)

use a name shorter

What browsers are you seeing the problem on?

Firefox

Anything else

{"level":"error","ts":"2024-08-05T09:10:48Z","msg":"Reconciler error","controller":"inferenceservice","controllerGroup":"serving.kserve.io","controllerKind":"InferenceService","InferenceService":{"name":"ibm-granitegranite-3b-code-instruct-accelerator","namespace":"demo-cl"},"namespace":"demo-cl","name":"ibm-granitegranite-3b-code-instruct-accelerator","reconcileID":"48dc6125-2c7a-4e7b-a46a-bc0fbef0c67a","error":"fails to reconcile component: fails to create NewRawKubeReconciler for predictor: failed creating host name: invalid domain name \"ibm-granitegranite-3b-code-instruct-accelerator-predictor-demo-cl.example.com\": url: Invalid value: \"ibm-granitegranite-3b-code-instruct-accelerator-predictor-demo-cl\": must be no more than 63 characters","errorVerbose":"failed creating host name: invalid domain name \"ibm-granitegranite-3b-code-instruct-accelerator-predictor-demo-cl.example.com\": url: Invalid value: \"ibm-granitegranite-3b-code-instruct-accelerator-predictor-demo-cl\": must be no more than 63 characters\nfails to create NewRawKubeReconciler for predictor\ngithub.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice/components.(*Predictor).Reconcile\n\t/go/src/github.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice/components/predictor.go:313\ngithub.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice.(*InferenceServiceReconciler).Reconcile\n\t/go/src/github.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice/controller.go:208\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227\nruntime.goexit\n\t/usr/lib/golang/src/runtime/asm_amd64.s:1650\nfails to reconcile component\ngithub.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice.(*InferenceServiceReconciler).Reconcile\n\t/go/src/github.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice/controller.go:213\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227\nruntime.goexit\n\t/usr/lib/golang/src/runtime/asm_amd64.s:1650","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/opt/app-root/src/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
@Loumy Loumy added kind/bug Something isn't working priority/normal An issue with the product; fix when possible untriaged Indicates the newly create issue has not been triaged yet labels Aug 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Something isn't working priority/normal An issue with the product; fix when possible untriaged Indicates the newly create issue has not been triaged yet
Projects
None yet
Development

No branches or pull requests

1 participant