Intermediate

If you are a cluster operator looking to expand your grasp of Kubernetes, this page and its linked topics extend the information provided on the foundational cluster operator page. From this page you can get information on key Kubernetes tasks needed to manage a complete production cluster.

Introductions to Kubernetes typically discuss simple stateless applications. As you move into more complex development, testing, and production environments, you need to consider more complex cases:

Communication: Ingress and Networking

• Ingress

Storage: Volumes and PersistentVolumes

• Volumes
• Persistent Volumes

Workloads

• DaemonSets
• Stateful Sets
• Jobs
• CronJobs

Pods

• Pod Lifecycle
  • Init Containers
  • Pod Presets
  • Container Lifecycle Hooks

And how Pods work with scheduling, priority, disruptions:

• Taints and Tolerations
• Pods and Priority
• Disruptions
• Assigning Pods to Nodes
• Managing Compute Resources for Containers
• Configuration Best Practices

Securing your cluster includes work beyond the scope of Kubernetes itself.

In Kubernetes, you configure access control:

• Controlling Access to the Kubernetes API
• Authenticating
• Using Admission Controllers

You also configure authorization. That is, you determine not just how users and services authenticate to the API server, or whether they have access, but also what resources they have access to. Role-based access control (RBAC) is the recommended mechanism for controlling authorization to Kubernetes resources. Other authorization modes are available for more specific use cases.

• Authorization Overview
• Using RBAC Authorization

You should create Secrets to hold sensitive data such as passwords, tokens, or keys. Be aware, however, that there are limitations to the protections that a Secret can provide. See the Risks section of the Secrets documentation.

Monitoring the health and state of your cluster is important. Collecting metrics, logging, and providing access to that information are common needs. Kubernetes provides some basic logging structure, and you may want to use additional tools to help aggregate and analyze log data.

Start with the basics on Kubernetes logging to understand how containers do logging and common patterns. Cluster operators often want to add something to gather and aggregate those logs. See the following topics:

• Logging Using Elasticsearch and Kibana
• Logging Using Stackdriver

Like log aggregation, many clusters utilize additional software to help capture metrics and display them. There is an overview of tools at Tools for Monitoring Compute, Storage, and Network Resources. Kubernetes also supports a resource metrics pipeline which can be used by Horizontal Pod Autoscaler with custom metrics.

Prometheus, another CNCFCloud Native Computing Foundation project, is a common choice to support capture and temporary collection of metrics. There are several options for installing Prometheus, including using the stable/prometheus helm chart, and CoreOS provides a prometheus operator and kube-prometheus, which adds on Grafana dashboards and common configurations.

A common configuration on Minikube and some Kubernetes clusters uses Heapster along with InfluxDB and Grafana. There is a walkthrough of how to install this configuration in your cluster. As of Kubernetes 1.11, Heapster is deprecated, as per sig-instrumentation. See Prometheus vs. Heapster vs. Kubernetes Metrics APIs for more information alternatives.

Hosted monitoring, APM, or data analytics services such as Datadog or Instana also offer Kubernetes integration.

Cluster Administration:

• Troubleshoot Clusters
• Debug Pods and Replication Controllers
• Debug Init Containers
• Debug Stateful Sets
• Debug Applications
• Using explorer to investigate your cluster