User:Accraze/MachineLearning/Kserve

From Wikitech
Jump to navigation Jump to search

Kserve

Serverless Inferencing on k8s

Requirements

Hardware

  • Minimum cluster is 4 cpus and 8Gi memory

Software

  • k8s 1.16 - 1.18
 * 1.19 is not suitable for kfserving (yet)
  • istio service mesh
 \`cluster-local-gateway\`\` is required to serve
 cluster-internal traffic for transformer and explainer use cases.
 https://knative.dev/docs/install/installing-istio/#updating-your-install-to-use-cluster-local-gateway
  • knative serving (and eventing if we want transformers/explainers)
  • cert manager / letsencrypt

Commands

  • Deploy / Update service:
 kubectl apply -f service.yaml -n kubeflow-user
  • Delete/Remove model:
 kubectl delete -f service.yaml -n kubeflow-user
  • Lookup info:
 kubectl describe inferenceservices.serving.kserve.io <model-name>

Links