MLOps Pathmlopspath.ru

07 · Serving-платформа7.1–7.7 · Runtime и оркестраторы7.5.1сложный

KServe как control plane

Зачем это нужно

KServe (formerly KFServing) — Kubernetes-native control plane для ML inference: CRD InferenceService, model storage initialization, integration with Istio/Knative. Runtime (MLServer, Triton, vLLM) — data plane внутри Pod'ов, которыми управляет KServe.

Для MDP capstone KServe — рекомендуемый способ deploy: один YAML вместо ручной сборки Deployment+Service+HPA.

Основные идеи

InferenceService — верхний объект.

apiVersion: serving.kserve.io/v1beta1 kind: InferenceService metadata: name: churn namespace: ml-prod spec: predictor: model: modelFormat: name: sklearn storageUri: s3://ml-models/churn/v44/ resources: requests: cpu: "500m" memory: "512Mi" limits: cpu: "2" memory: "1Gi"

KServe controller создаёт Deployment, Service, optionally VirtualService, storage-initializer initContainer для storageUri.

Режимы развёртывания.

Режим Инфраструктура Масштабирование и rollout
Standard Kubernetes Deployment/Service Обычный Kubernetes/HPA; без Knative revisions и serverless canary
Knative / serverless Knative Service + queue-proxy Scale-to-zero, revisions и canary traffic split

Выбирайте Standard для предсказуемого always-on serving. canaryTrafficPercent относится к serverless/Knative mode; в Standard mode используйте другой явный механизм rollout (например, Argo Rollouts или Istio).

Компоненты spec.

Section Purpose
predictor Main model serving (required)
transformer Pre/post processing container
explainer Optional interpretability
predictor.canaryTrafficPercent Traffic split

Model formats. sklearn, xgboost, lightgbm, onnx, pytorch, tensorflow, triton — map to cluster ServingRuntime or built-in defaults.

Storage URI. s3://, gs://, hdfs://, pvc://. Credentials via service account or secret annotation.

Revision model (Knative/serverless mode). Each spec change → new Revision; traffic split between revisions for canary.

Scale-to-zero (Knative/serverless mode). No traffic → scale pods to 0; first request cold start. Disable for latency-sensitive prod or set minReplicas annotation.

Ingress. KServe creates Istio VirtualService or Kourier route — host churn.ml.example.com. Integrates module 3.6.

ServingRuntime CRD. Platform team defines cluster-wide templates:

apiVersion: serving.kserve.io/v1alpha1 kind: ClusterServingRuntime metadata: name: mlserver-sklearn spec: supportedModelFormats: - name: sklearn containers: - name: kserve-container image: docker.io/seldonio/mlserver:1.6.1

Teams reference format name; platform controls image version.

KServe vs Seldon.

KServe Seldon
CNCF / Kube ecosystem Strong Vendor
LLM first-class (vLLM) Growing Less focus
Graph ensembles Basic Richer
Greenfield K8s ML Preferred Legacy

Observability. Request logs, Prometheus metrics from queue-proxy (if Knative) and runtime; integrate module 5.

Как это выглядит на практике

GitOps flow:

  1. Model v44 in MinIO after MMS approval.

  2. MR: storageUri: .../v44/ in inference/churn.yaml.

  3. Argo sync → KServe rolls revision.

  4. Smoke: curl https://churn.ml.example.com/v1/models/churn:predict (or V2 infer).

Canary (только Knative/serverless mode):

spec: predictor: model: storageUri: s3://ml-models/churn/v45/ canaryTrafficPercent: 15

Monitor 24h → set 100% or rollback URI to v44.

Transformer sidecar pattern:

spec: transformer: containers: - image: registry.mdp.ru/ml/churn-transformer:2 env: - name: FEAST_REDIS value: feast-redis.ml.svc predictor: model: modelFormat: name: sklearn storageUri: s3://ml-models/churn/v44/

Troubleshooting.

Status Meaning
Ready=False, ModelLoadFailed Bad URI, credentials, format
Revision not receiving traffic Istio/ingress misconfig
Storage initializer error MinIO path typo

kubectl describe inferenceservice churn -n ml-prod

Что сделать после занятия

  • Объясните разницу control plane (KServe) и data plane (MLServer).

  • Напишите минимальный InferenceService для capstone sklearn model.

  • Что делает storage-initializer initContainer?

Официальные материалы