Operator
TPR(Third Party Resource) 在k8s 1.7 被集成,并命名为CRD(Custom Resource Definition).
通过CRD,K8S可以动态的添加和管理资源,controller跟踪这些资源。
CRD+custom Controller = decalartive API(声明式API),一般分为通用性controller和operator.
通用型controller一般用于平台需求,operator一般用于部署特定应用.
用于开发operator的工具有kubebuilder和operator-sdk, 他们都是基于controller-runtime开发.
https://github.com/operator-framework/awesome-operators
https://access.redhat.com/documentation/zh-cn/openshift_container_platform/4.2/html/operators/index
开发示例:
https://github.com/kubernetes/sample-controller
Operator功能:
- 监控K8S资源的变化
- 响应K8S资源的变化(部署,升级,回滚等)
- 执行K8S资源的变化 (备份,扩容,配置等)
kubebuilder
sig维护的kubebuilder能方便的开发operator.
https://github.com/kubernetes-sigs/kubebuilder
Install:
curl -L -o kubebuilder "https://go.kubebuilder.io/dl/latest/$(go env GOOS)/$(go env GOARCH)"
chmod +x kubebuilder && sudo mv kubebuilder /usr/local/bin/
create project:
mkdir k8s-nim-operator
cd k8s-nim-operator
kubebuilder init --domain nvidia.com --repo github.com/NVIDIA/k8s-nim-operator --project-name k8s-nim-operator
Create API
kubebuilder create api --group apps --version v1alpha1 --kind NIMService --controller --resource --namespaced
kubebuilder create api --group apps --version v1alpha1 --kind NIMCache --controller --resource --namespaced
kubebuilder create api --group apps --version v1alpha1 --kind NIMBuild --controller --resource --namespaced
kubebuilder create api --group apps --version v1alpha1 --kind NIMPipeline --controller --resource --namespaced
create webhook:
kubebuilder create webhook --group apps --version v1alpha1 --kind NIMService --defaulting --programmatic-validation
kubebuilder create webhook --group apps --version v1alpha1 --kind NIMCache --defaulting --programmatic-validation
Test:
make install // deploy CRDs on k8s.
make uninstall
make run // run controller locally.
kubectl apply -k config/samples/ // install CR instances
Deploy:
//Build the operator:
make build
//Build and push container image:
make docker-build docker-push IMG=<your-registry>/k8s-nim-operator:tag
//Deploy controller to cluster:
make deploy IMG=<your-registry>/k8s-nim-operator:tag
make undeploy
可以创建helm chart发布operator。
operator-sdk
redhat的operator-sdk(基于kubebuilder)能快速的开发operator.
支持go, ansible和helm三种方式.
https://github.com/operator-framework/operator-sdk
https://sdk.operatorframework.io/docs/installation/
operator的run三种模式:
- 在集群外部本地运行(开发测试).
- 作为deployment在集群内部运行.
- 通过OLM部署.