1. Overview

Without secure hosts, you cannot have secure containers. Host machines are a critical component in the container environment, and they must be secured with the same care as containers. Runtime defense employs machine learning to build models of known good activity so that anomalous runtime behavior can quickly be detected. Twistlock applies our model-based approach to runtime defense to protect the hosts in your environment.

2. Host Models

Whereas container runtime protection creates models for images, host runtime protection creates models for systemd services. When Defender is installed on a host, it automatically creates runtime models for every service and container that runs on the host.

When a new service is detected, Twistlock creates a new model for it, and puts the model into learning mode. Baseline models are provided for common Linux services, such as httpd (Apache) and sshd. All services, whether they have baseline models or not, are subject to a learning period before their models are fully activated.

There is one model per service, and models are global. A model for one host service is shared and reused amongst all hosts. For example, consider a service foo, which runs on host-A and host-B. If foo acquires the DOCKER capability on host-A and the USERS_ADMIN capability on host-B, then the model, shared across both hosts, would enumerate both capabilities.

You can view all runtime models for your host service in Console under Monitor > Runtime > Host Models. The table on this page has two pivots points:

  • Host — Lists all the hosts in your environment. Click on a host to see the services on the host and their associated models.

  • Service — Lists all the services in your environment. You can review the state of the models, and either relearn a model, or activate a manual learning mode for a model. Clicking on a service shows you the capabilities that have been whitelisted for it.

2.1. Capabilities

Host models map capabilities to services. Capabilities are Twistlock-curated units of process and file system actions that express the things that services routinely need to do. They can be independently enabled or disabled on a per-service basis, and provide fine-grained control over what a service can and cannot do.

Capabilities encapsulate the intersection of what services need to do and what attackers want to do. Capabilities were borne from the following observations:

  • Attacker objectives are well known. They need to gain a footprint on the host, establish persistence, elevate permissions, and so on.

  • The path to achieving these objectives is also well known. Attackers must use specific utilities and manipulate specific files in order to advance their position. In many cases, these are the same utilities and files that legitimate services need to use.

By selectively assigning capabilities to services so that thet can do their job, but nothing more, Twistlock can limit what an attacker can do when they hijack a service and try to exploit it to run in non-legitmate ways.

Capabilities have two key traits:

  • Processes they can run.

  • Files they can access.

For example, the USER_ADMIN capability permits user management. The following process and configuration files fall under its purview:

Processes:
  - adduser
  - useradd
file system:
  - /etc/group
  - /etc/passwd

2.2. List of capabilities

The following capabilities can be applied to a service model. The last column in the table provides representative examples of files and processes that are associated with each capability.

Capability Description Example

SYSTEM_LOGS

Reading and changing system logs

Example files:

  • /var/log/syslog (System log)

  • /var/log/audit/audit.log (Audit daemon log)

  • /var/log/auth.log (Authentication logs)

  • /var/log/kern.log (Kernel logs)

DOCKER_SOCKET

Access to Docker socket

Example files:

  • /var/run/docker.sock

DOCKER

Changing Docker configuration

Example files:

  • /etc/docker

Example processes:

  • docker-containerd

  • docker-containerd-current

  • docker-proxy

K8S

Changing Kubernetes configuration

Example files:

  • /etc/kubernetes

  • ~/.kube

Example processes:

  • kubelet

  • kubectl

OPENSHIFT

Changing OpenShift configuration

Example files:

  • /usr/share/openshift

Example processes:

  • oc

  • openshift

USERS

Reading users and groups

Example files:

  • /etc/passwd (User database)

  • /etc/group (User groups)

USERS_ADMIN

Changing users and groups

All files in the USERS capability, but these files are monitored for write activity.

SSH

Changing sshd configuration

Example files:

  • /etc/ssh

  • /etc/ssh/sshd_config

  • SSH keys under ~/.ssh

Example processes:

  • ssh

  • sshd

  • vpn

SHELL

Running a shell

Example files:

  • /etc/profile

Processes:

  • bash

  • ash

  • busybox

NET

Changing network configuration

Example processes:

  • iptables

  • socat

  • nc

HOSTS_CONFIG

Changing host configuration

Example files:

/etc/hosts /etc/resolv.conf

GCLOUD

Changing Google Cloud configuration

INSTALLER

Installing software

Example processes:

  • apt

  • rpm

  • Writing any binary to disk.

SERVICE_OPERATOR

Managing services

Example processes:

  • systemd

  • service

  • systemctl

PRIVILEGED_PROCESSES

Running privileged processes

2.3. Violations

When a service requests a capability that isn’t in its model, Twistlock generates an audit. Audits can be viewed under Monitor > Events > Host Audits.

Consider an HTTP server that executed the useradd command. There should be no reason for an HTTP server to execute the useradd command. The useradd command is part of the USERS_ADMIN capability, and the http-server service’s model won’t have this capability whitelisted. As a result, you get an audit:

Service http-server attempted to obtain capability USERS_ADMIN by executing /usr/sbin/useradd

You can create rules that prevent a service from running a process or accessing a file that is part of a capability (the default action is alert). Create a new host runtime rule, select the Capabilities tab, and set the action for the capability to Prevent. In the General tab, be sure to scope the rule to a specific service Otherwise, you will disrupt other services.

With this rule in place, further attempts to exploit http-server are blocked.

3. Enabling host runtime protection

Runtime protection for hosts is enabled by default. When Defender is installed on a host, it automatically starts building runtime models for all services. Twistlock ships with a default rule named Default - alert on suspicious runtime behavior, which alerts on all anomalous activity. To see the rule, open Console, then go to Defend > Runtime > Host Policy.

As part of the default rule, Twistlock Advanced Threat Protection (TATP) is enabled. TATP supplements runtime protection by alerting you when:

  • Malware is found anywhere on the host file system.

  • Connections are made to banned IP addresses.

Create new rules to enhance host protection.

Do not whitelist or blacklist capabilities globally by setting the host filters to a wildcard because it can interrupt the execution of legitmate services.

3.1. Anomalous app detection

Twistlock learns the normal set of apps running on your hosts (a so-called app baseline), and automatically identifies apps added abnormally. After all running apps complete the learning phase, any subsequently started apps generate a one time audit.

3.2. Log inspection

Twistlock lets you collect and analyze operating systems and application logs for security events. For each inspection rule, specify the log file to parse and any number of inspection expressions. Inspection expressions support the RE2 regular expression syntax.

A number of predefined rules are provided for apps such as sshd, mongod, and nginx.

3.3. Networking

Twistlock lets you secure host networking. You can filter DNS traffic and alert on inbound and outbound connections.

3.3.1. DNS

When DNS monitoring is enabled, Twistlock filters DNS lookups. By default, DNS monitoring is disabled. Dangerous domains are detected as follows:

  • Twistlock Intelligence Stream -- Twistlock’s threat feed contains a list of known bad domains.

  • Explicit whitelists and blacklists — Host runtime rules let you augment the Twistlock’s Intelligence Stream data with your own lists of known good and bad domains.

In your runtime rules, set Effect to configure how Defender handles DNS lookups from containers:

  • Disable — DNS monitoring is disabled.

  • Alert — DNS monitoring is enabled. Anomalous activity generates audits.

  • Prevent — DNS monitoring is enabled. Anomalous activity generates audits. Anomalous DNS lookups are dropped.

3.3.2. IP connectivity

You can raise alerts when inbound or outbound connections are established. Specify inbound ports, and outbound IPs and ports.

Outbound connections are event-driven, which means that as soon as a process attempts to establish a connection, you’ll be notified. Twistlock polls inbound connections, which means you’ll be notified periodically, and not necessarily the moment an inbound connection is established.

3.4. Access logging

Set up rules to audit host events.

3.5. File integrity management (FIM)

Changes to critical files can reduce your overall security posture, and they can be the first indicator of an attack in progress. Twistlock FIM continually watches the files and directories in your monitoring profile for changes. You can configure to FIM to detect:

  • Reads or writes to sensitive files, such as certificates, secrets, and configuration files.

  • Binaries written to the file system.

  • Abnormally installed software. For example, files written to a file system by programs other than apt-get.

A monitoring profile consists of rules, where each rule specifies the path to monitor, the file operation, and exceptions.

runtime defense hosts fim rule

The file operations supported are:

  • Writes to files or directories. When you specify a directory, recursive monitoring is supported.

  • Reads. When you specify a directory, recursive monitoring isn’t supported.

  • Attribute changes. The attributes watched are permissions, ownership, timestamps, and links. When you specify a directory, recursive monitoring isn’t supported.