7.5.1 · блок 7

KServe как control plane

Зачем это нужно

KServe (formerly KFServing) — Kubernetes-native control plane для ML inference: CRD InferenceService, model storage initialization, integration with Istio/Knative. Runtime (MLServer, Triton, vLLM) — data plane внутри Pod'ов, которыми управляет KServe.

Для MDP capstone KServe — рекомендуемый способ deploy: один YAML вместо ручной сборки Deployment+Service+HPA.

Основные идеи

InferenceService — верхний объект.


apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: churn
  namespace: ml-prod
spec:
  predictor:
    model:
      modelFormat:
        name: sklearn
      storageUri: s3://ml-models/churn/v44/
      resources:
        requests:
          cpu: "500m"
          memory: "512Mi"
        limits:
          cpu: "2"
          memory: "1Gi"

KServe controller создаёт Deployment, Service, optionally VirtualService, storage-initializer initContainer для storageUri.

Режимы развёртывания.

| Режим | Инфраструктура | Масштабирование и rollout |

|---|---|---|

| **Standard** | Kubernetes Deployment/Service | Обычный Kubernetes/HPA; без Knative revisions и serverless canary |

| **Knative / serverless** | Knative Service + queue-proxy | Scale-to-zero, revisions и canary traffic split |

Выбирайте Standard для предсказуемого always-on serving. canaryTrafficPercent относится к serverless/Knative mode; в Standard mode используйте другой явный механизм rollout (например, Argo Rollouts или Istio).

Компоненты spec.

| Section | Purpose |

|---------|---------|

| **predictor** | Main model serving (required) |

| **transformer** | Pre/post processing container |

| **explainer** | Optional interpretability |

| **predictor.canaryTrafficPercent** | Traffic split |

Model formats. sklearn, xgboost, lightgbm, onnx, pytorch, tensorflow, triton — map to cluster ServingRuntime or built-in defaults.

Storage URI. s3://, gs://, hdfs://, pvc://. Credentials via service account or secret annotation.

Revision model (Knative/serverless mode). Each spec change → new Revision; traffic split between revisions for canary.

Scale-to-zero (Knative/serverless mode). No traffic → scale pods to 0; first request cold start. Disable for latency-sensitive prod or set minReplicas annotation.

Ingress. KServe creates Istio VirtualService or Kourier route — host churn.ml.example.com. Integrates module 3.6.

ServingRuntime CRD. Platform team defines cluster-wide templates:


apiVersion: serving.kserve.io/v1alpha1
kind: ClusterServingRuntime
metadata:
  name: mlserver-sklearn
spec:
  supportedModelFormats:
    - name: sklearn
  containers:
    - name: kserve-container
      image: docker.io/seldonio/mlserver:1.6.1

Teams reference format name; platform controls image version.

KServe vs Seldon.

| | KServe | Seldon |

|---|--------|--------|

| CNCF / Kube ecosystem | Strong | Vendor |

| LLM first-class (vLLM) | Growing | Less focus |

| Graph ensembles | Basic | Richer |

| Greenfield K8s ML | Preferred | Legacy |

Observability. Request logs, Prometheus metrics from queue-proxy (if Knative) and runtime; integrate module 5.

Как это выглядит на практике

GitOps flow:

1. Model v44 in MinIO after MMS approval.

2. MR: storageUri: .../v44/ in inference/churn.yaml.

3. Argo sync → KServe rolls revision.

4. Smoke: curl https://churn.ml.example.com/v1/models/churn:predict (or V2 infer).

Canary (только Knative/serverless mode):


spec:
  predictor:
    model:
      storageUri: s3://ml-models/churn/v45/
    canaryTrafficPercent: 15

Monitor 24h → set 100% or rollback URI to v44.

Transformer sidecar pattern:


spec:
  transformer:
    containers:
      - image: registry.mdp.ru/ml/churn-transformer:2
        env:
          - name: FEAST_REDIS
            value: feast-redis.ml.svc
  predictor:
    model:
      modelFormat:
        name: sklearn
      storageUri: s3://ml-models/churn/v44/

Troubleshooting.

| Status | Meaning |

|--------|---------|

| Ready=False, ModelLoadFailed | Bad URI, credentials, format |

| Revision not receiving traffic | Istio/ingress misconfig |

| Storage initializer error | MinIO path typo |

kubectl describe inferenceservice churn -n ml-prod

Что сделать после занятия

[ ] Объясните разницу control plane (KServe) и data plane (MLServer).
[ ] Напишите минимальный InferenceService для capstone sklearn model.
[ ] Что делает storage-initializer initContainer?

Официальные материалы

Открыть интерактивную версию