9 Essential Infrastructure Security Considerations for Kubernetes
September 15, 2022
Bridgecrew

Part of the reason why securing Kubernetes can be challenging is that Kubernetes isn't a single, simple framework. It's a complex, multi-layered beast. Each layer — from code and containers to clusters and external cloud services — poses its own set of security challenges and requires unique solutions. Any issue at one layer, such as a vulnerability in a container, gets amplified if another layer has a security issue, such as an unencrypted database.

The good news in the face of this complexity is that it's possible to use a single methodology — infrastructure as code (IaC) — to secure all layers of Kubernetes. Furthermore, when you define the various layers of your Kubernetes environment using an IaC approach and include automated IaC security in your workflow, you maximize your ability to prevent misconfigurations that could create Kubernetes security vulnerabilities in your clusters.

To explain how to leverage IaC as the foundation for Kubernetes infrastructure security, this blog walks through nine Kubernetes security best practices that apply to the various layers or components of Kubernetes. It also shows how IaC or related security automation techniques can help mitigate each challenge.


1. Kubernetes host infrastructure security

From a Kubernetes security standpoint, let's start our discussion at (what some might consider) the most basic layer of a Kubernetes environment: the host infrastructure. Here, we're talking about the bare metal and/or virtual servers that serve as Kubernetes nodes.

Securing this infrastructure starts with ensuring that each node (whether it's a worker or a master) is hardened against security risks. An easy way to do this is to provision each node using IaC templates that enforce security best practices at the configuration and operating system level.

When you write your IaC templates, ensure that the image template and/or the startup script for your nodes are configured to run only the software that is strictly necessary to serve as nodes. Extraneous libraries, packages, and services should be excluded. You may also want to provision nodes with a kernel-level security hardening framework like SELinux or AppArmor, as well as basic hygiene like encrypting any attached storage.

2. IAM security for Kubernetes clusters

Depending on where and how you run Kubernetes, you may use an external cloud service to manage access controls for your Kubernetes environment. For example, if you use AWS EKS, you'll use AWS IAM to grant varying levels of access to Kubernetes clusters based on individual users' needs. (Note that this is different from internal access controls within Kubernetes clusters, which are defined using Kubernetes RBAC.)

Using a least privilege approach to manage IAM roles and policies in IaC minimizes the risk of manual configuration errors that could grant overly permissive access to the wrong user or service. You can also scan your configurations with IaC scanning tools to automatically catch overly permissive or unused IAM roles and policies.

3. Container registry security

Container registries, which are platforms that store container images, aren't a native part of Kubernetes. But they're widely used as part of a Kubernetes-based application deployment pipeline to host the images that are deployed into a Kubernetes environment.

Access control frameworks vary between container registries. Some registries (like AWS ECR) can manage access via public cloud IAM frameworks. Either way, you can typically define access permissions using code, then apply and manage them using IaC.

In doing so, you'll want to follow the principle of least privilege: container images should only be accessible by registry users who need to access them. Just as important, unauthorized users should also be prevented from uploading images to a registry. Insecure registries are a great way for threat actors to push malicious images into your Kubernetes environment.

4. Avoiding the default namespace

This is a low-hanging yet easy-to-miss one. By default, every Kubernetes cluster contains a namespace named "default." As the name implies, the default namespace is where workloads will reside by default unless you create other namespaces.

Although using the default namespace isn't the worst mistake you can make, it presents two security concerns.

One is that everyone will know the name of your namespace, a very important configuration value, making it that much easier for attackers to exploit your environment. Again, that doesn't mean you'll instantly be hacked just because you use the default namespace, but the less information you make available to the bad guys, the better.

The other, more significant concern with default namespaces is that if everything runs there, then your workloads are not segmented. It's a better idea to create separate namespaces for separate workloads. Doing so makes it harder for a breach against one workload to escalate into a cluster-wide issue.

As a bonus, Google Cloud says that using the default namespace could be bad from an administrative perspective, too.

Alternatively, you can create new namespaces using kubectl or define them in a YAML file. You can also scan existing YAML files to detect instances where workloads are configured to run in the default namespace.

5. Avoid privileged containers

At the container level, a key security consideration is making sure that containers can't run in privileged mode.

This is easy to do using IaC. Simply write a security context that denies privilege escalation, and make sure to include the context when defining a pod:

securityContext:
allowPrivilegeEscalation: false

Then, make sure privilege isn't granted directly with the "privilege" flag or by granting CAP_SYS_ADMIN. Here again, you can use IaC scanning tools to check for the absence of this security context and to catch any other privilege escalation settings within pod settings.

6. Isolate pods at the network level

By default, any pod running in Kubernetes can talk to any other pod over the network. That's why, unless your pods actually need to talk to each other (which is usually only the case if they are part of a related workload), you should isolate them to achieve a higher level of segmentation between workloads.

As long as you have a Kubernetes networking layer that supports Network Policies (most do, but some Kubernetes distributions default to CNIs that lack this support), you can write a Network Policy to isolate pods at the network level or to place specific restrictions on network connections between pods. For example:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: test-network-policy
namespace: default
spec:
podSelector:
matchLabels:
role: db
policyTypes:
- Ingress
- Egress
ingress:
- from:
- ipBlock:
cidr: 172.17.0.0/16
except:
- 172.17.1.0/24
- namespaceSelector:
matchLabels:
project: myproject
- podSelector:
matchLabels:
role: frontend
ports:
- protocol: TCP
port: 6379
egress:
- to:
- ipBlock:
cidr: 10.0.0.0/24
ports:
- protocol: TCP
port: 5978

A policy like this specifies which other pods the selected pod (here, pods with the "role=db" label) can connect to for both ingress and egress.

Here again, the beauty of defining all of this in code is that we can scan the code to check for configuration mistakes or oversights that may grant more network access than we intended.

7. Avoid "latest" container images

When you tell Kubernetes to pull an image from a container registry, it will automatically pull the version of the specified image labeled with the "latest" tag in the registry.

While this may seem logical — after all, you typically want the latest version of an application — it can be risky from a security perspective because relying on the "latest" tag makes it more difficult to track the specific version of a container that you are using. In turn, you may not know whether your containers are subject to security vulnerabilities or image poisoning attacks that impact specific images on an application.

To avoid this mistake, specify image versions, or better yet, the image manifest when pulling images, and audit your Kubernetes configurations to detect instances that lack specific version selection for images. It's a little more work, but it's worth it from a Kubernetes security perspective.

8. Scan open source components

Regardless of which container image version you use, you should never assume it's secure. The same is true of Helm charts. Millions of public container images and Helm charts turn out to contain security misconfigurations and vulnerabilities, sometimes very severe ones.

Before using open-source Helm charts from Artifact Hub or elsewhere, you should scan them for misconfigurations. Similarly, before deploying container images, you should scan them using tools to identify vulnerable components within them. This isn't exactly IaC, but it's similar because it allows you to automate the identification of security risks within Kubernetes before putting them into production.

9. Enable audit logging

If a security incident (or, for that matter, a performance incident) does occur, Kubernetes audit logs tend to be very helpful in researching it. This is because audit logs record every request to the Kubernetes API server and its outcome.

Unfortunately, audit logs aren't enabled by default in most Kubernetes distributions. To turn this feature on, add lines like these to your kube-apiserver policy file:


--audit-log-path=/var/log/kubernetes/apiserver/audit.log
--audit-policy-file=/etc/kubernetes/audit-policies/policy.yaml

This tells Kubernetes where to store audit logs and where to find the policy file that configures what to audit.

Because audit logs offer critical security information, it's worth including a rule in your Kubernetes configuration scans to check whether audit logs are enabled.

As you can tell, Kubernetes is not very secure-by-default, and, unfortunately, there is no "one dumb trick" that can solve all your Kubernetes security woes. Instead, Kubernetes security requires a multi-pronged approach that addresses the security risks that exist across the various layers of Kubernetes.

Fortunately, by leveraging an IaC-based approach to defining security rules in Kubernetes, you can more easily configurations and minimize the risk of configuration issues that will lead to security breaches within any layer of your Kubernetes stack.

Share this

Industry News

September 29, 2022

CloudBees announced the acquisition of ReleaseIQ to expand the company’s DevSecOps capabilities, empowering customers with a low-code, end-to-end release orchestration and visibility solution.

September 29, 2022

SmartBear continues expanding its commitment to the Atlassian Marketplace, adding Bugsnag for Jira and SwaggerHub Integration for Confluence.

Bugsnag developers monitoring application stability and documenting in Jira no longer need to interrupt their workflow to access the app. Developers working in SwaggerHub can use the macro to push API definitions and changes directly to other teams and business stakeholders that work within Confluence. By increasing the presence of SmartBear tools on the Atlassian Marketplace, the company continues meeting developers where they are.

September 29, 2022

Ox Security exited stealth today with $34M in funding led by Evolution Equity Partners, Team8, and M12, Microsoft's venture fund, with participation from Rain Capital.

September 29, 2022

cnvrg.io announced that the new Intel Developer Cloud is now available via the cnvrg.io Metacloud platform, providing a fully integrated software and hardware solution.

September 28, 2022

Kong introduced a number of new performance, security and extensibility features across its entire product portfolio, including major new releases of Kong Gateway, Kong Konnect, Kong Mesh, Kong Insomnia and Kong Ingress Controller, as well as new projects from the Kong Incubator.

September 28, 2022

BroadPeak Partners announced the availability of the new K3 API Connector.

September 28, 2022

Aqua Security announced a new end-to-end software supply chain security solution.

September 27, 2022

DevOps Institute will host SKILup Festival in Singapore on November 15, 2022.

September 27, 2022

Delinea announced the latest release of DevOps Secrets Vault, its high-speed vault for DevOps and DevSecOps teams.

September 27, 2022

The Apptainer community announced version 1.1.0 of the popular container system for secure, high-performance computing (HPC). Improvements in the new version provide a smaller attack surface for production deployments while offering features that improve and simplify the user experience.

September 26, 2022

Secure Code Warrior unveiled Coding Labs, a new mechanism that allows developers to more easily move from learning to applying secure coding knowledge, leading to fewer vulnerabilities in code.

September 26, 2022

ActiveState announced the availability of the ActiveState Artifact Repository.

September 26, 2022

Split Software announced the availability of its Feature Data Platform in the Microsoft Azure Marketplace.

September 22, 2022

Katalon announced the launch of the Katalon Platform, a modern and comprehensive software quality management platform that enables teams of any size to easily and efficiently test, launch, and optimize apps, products, and software.

September 22, 2022

StackHawk announced its Deeper API Security Test Coverage release.