This guide provides direction, advice, and recommendations for putting Twistlock into operation in your environment. The guide is based on the extensive experience of our Customer Success Team, who on-boards all customers, advises them how to best to secure their environment, and helps them to configure and deploy Twistlock to meet those objectives.
The guide is presented as a timeline-based journey that starts at day zero, when you’ve just downloaded the software. We offer the timeline to help you plan for the work ahead and measure your progress. We don’t expect you to follow the timeline exactly, but to pick and choose what works for you and your organization.
Our Customer Success Team is always available to assist you, and we encourage you to contact us. We are always available to advise you on the best practices for deploying and operating Twistlock in your environment.
The master timeline provides a birds-eye view of the steps for operationalizing Twistlock. Each segment in the master timeline is fully explored in a dedicated section. Dedicated sections discuss the considerations, tradeoffs, and strategies for working through each step. Links to detailed articles on our Documentation Portal are provided throughout.
Operationalizing Twistlock can be broken down into a number of discrete steps:
Learn: Learn about Twistlock concepts and how it all works.
Plan and Deploy: Map Twistlock onto your environment. Pick a deployment pattern and customize it for your needs. Factor in automation, high availability, and disaster recovery. Install Twistlock in your environment.
Secure your environment (Config, Observe, Gain Ops Experience): Configure Twistlock features and define policy. Gain operational experience. Most customers focus on putting one subsystem into operation at a time, and they typically do it in the following order:
Vulnerability management
Compliance
Runtime defense and firewalls
Maintenance and operations: Respond to incidents and alerts. Add and tweak rules and policy as new apps are brought online. Upgrade Twistlock as new releases are published. The timeline should be used as a framework for deploying Twistlock. Don’t pay too much attention to the number of weeks for each step. The steps are more important than the actual time spent. Timelines vary substantially from organization to organization.
Twistlock is a collaborative tool. You’re going to get the most value when representatives from Development, DevOps, and Security actively participate in planning and operations. When teams work in silos, the CI/CD pipeline will be disjointed and exasperating. When they are aligned, your CI/CD pipeline will be fluid and frictionless, with security controls transparently governing the flow.
The participants in this story are:
Security: Has the expertise and authority to sign off on policy decisions. Understands policy structure. Whitelists and blacklists specific CVEs. Analyzes and interprets audit data.
DevOps: Understands the infrastructure. Understands the environment’s topology. Installs Twistlock. Deploys Defenders. Handles upgrades. Configures integration with the registry, Jenkins, and so on.
Development: Understands the apps that run in your environment. Understands app composition, package manifests, and relationships between components. Fixes security issues that stall the CI/CD pipeline. Helps Security tune the models that protect running software. Helps remediate runtime security issues.
Typically, one group "owns" the Twistlock tool, and it tends to be Security. You’ll need a staff of one to two Security people and one to two DevOps people to man the Twistlock tool.
Developers are rarely granted elevated access to Twistlock Console (the central dashboard). Rather, they should be granted read-only access to vulnerability reports and tools. Twistlock provides a number of roles that provide different levels of access to Twistlock.
Since DevOps and Security start working with the tool from day zero, they’re the first to grasp concepts and workflows. Don’t leave Development in the dark. The deeper Development integrates Twistlock into their own workflow, the better your results will be, with faster time to mitigation and shorter service interruptions. Schedule regular meetings with Development to discuss expectations, demonstrate how Twistlock works, and how it fits into their workflow. We don’t know how many times developers actively seek us out an the tradeshow floor to get a demo because no has ever shown them the big picture.
Download the software. Come up to speed on Twistlock concepts, and study the install flow. If you like, set up Onebox. Onebox installs Console and Defender onto a single host, and provides a simple environment to play with the product.
Resources:
Contextual help. Every page in Console has a help button in the top right corner. If you’ve stood up a simple Twistlock environment, and you’re poking around through Console, use the contextual help to get more info about a given page.
Plan how Twistlock should be deployed to your environment.
The first consideration is Console. Console serves as both the management interface and the API. Use it to define policy and monitor your environment. When you deploy Twistlock, install Console first.
The next consideration is Defender. Defender enforces the policies you set in Console. It must run on every host you want to secure. Defender connects to Console over the network to retrieve policies and report data.
The big problems to solve in the planning phase are:
Where should Console run?
How many instances of Console should be deployed?
Which hosts must be protected by Defender?
How does Defender connect over the network to Console?
Twistlock can work within the constraints of almost any topology. The placement of Console drives the design. The discussion in following sections will help you assess your options and select the best deployment pattern. Use one of the following deployment patterns as a starting point:
1 Console per environment.
1 Console for the production environment and 1 Console for all other environments (dev, test, staging, pre-prod, etc).
1 Console for all environments (also known as a "single pane of glass").
Projects, which support multi-tenancy and/or very large scale environments (more than 5,000 hosts).
An environment is a logical grouping of hosts, such as a cluster, that supports the execution of your app workload. Here, environment is loosely defined because its scope can differ from organization to organization. For example, a product group might have its own environment, segmented into dev, test, and prod. A large organization might have separate such environments for each product group in a division. Or it might simply classify everything as prod or non-prod that’s in two large environments shared across the organization.
The simplest deployment pattern calls for installing one Console per environment. When you have many environments, however, this pattern is di cult to manage. Projects, with it’s support for multi-tenancy, can help by allowing you to manage a large environment from one central Console.
The relationship between your teams and your environments, along with who has authority to set policy, further constrains how you can deploy Twistlock.
How do you separate the management of your different environments? How much sharing between environments is there? For example, if dev and prod are completely isolated from each other, then you could deploy one Console to each environment. Alternatively, you could use Twistlock’s native support for multi-tenancy (known as Projects) to deploy a supervisor Console to each environment, and manage access to each environment from single central Console.
What about security rules and policies? Do the people managing each environment have the autonomy to configure their own rules? If so, then the same patterns are applicable (one Console per environment, or multi-tenancy with projects). If not, you could deploy a single Console to manage both environments.
Your network topology also constrains the available options. Defenders must be able to communicate with Console. If a Defender can’t reach a Console, you might be forced to install a Console in that environment. For example, an air-gapped environment would be forced to run a dedicated instance of Console.
Twistlock supports integration with enterprise directory services and identity providers so that you can reuse existing users and groups to manage access to Console. Which identity provider will you use? How will you assign roles? Twistlock offers a number of preconfigured roles for the various personas that can interact with the tool.
Best practice
Deploy Twistlock to at least one environment other than prod. For example, consider running Twistlock in both your prod and corresponding pre-prod (or staging) environment. If you deploy Twistlock to just your prod environment, then every change to Twistlock (upgrades, new rules) could incapacitate a mission critical prod environment. Instead, test and simulate changes in your pre-prod environment before rolling them out to your prod environment.
Twistlock’s license does not restrict the number of Consoles you can deploy. You’re free to run and operate as many Consoles as you like. Currently, more than 3/4 of our customers deploy multiple Consoles.
Resources:
Integrating with Active Directory, OpenLDAP, or SAML.
After completing the planning phase, deploying Twistlock is pretty easy. Refer to one of our dedicated install guides for Kubernetes, OpenShift, Pivotal Container Service, Docker Swarm, Amazon ECS, DC/OS, and Windows.
Best practice
Document your install. There is no difference between installing and upgrading the product. Console is stateful, but it stores its state in a volume outside the container. Defenders are stateless. So upgrades simply require replacing the running containers with newer ones.
Many customers cannot recall how they installed Twistlock when it comes time to upgrade months later. When working on your first install, keep good notes about the parameters of your install. You’ll use them again when you upgrade Twistlock.
After installing Twistlock, observe it for the first month. Get familiar with the data that’s being reported. Review the default rules. Study how Twistlock rules work. Individual rules make up your policy. Rule order has a big impact on how a policy is enforced. More specific rules must always precede more general rules.
Start looking at your top 10 CVEs. Resolve them if you want, but you’ll come back to this later when you start operationalizing vulnerability management.
There are four Twistlock systems to bring online and operationalize: Vulnerability management, compliance, runtime defense, and firewalls. Each system should be operationalized in stages:
Stage 1: Configure the feature. Turn on what you need. All systems ship with sensible default rules.
Stage 2: Observe the incoming data (audits, reports, etc), and get familiar with the feature and its capabilities.
Stage 3: Put your policy into place one step at a time. Tighten your rules in measured steps, giving all teams time to acclimatize at each step.
When Defender is installed, it automatically starts scanning images, containers, and hosts for vulnerabilities. Study this data first before configuring any additional vulnerability scanning. Next, configure Twistlock to scan your registries. Then install and configure the Twistlock Jenkins plugin. If you use CI tool other than Jenkins, then use twistcli, a stand-alone command line utility, to scan the images emitted from the build. Finally, configure Twistlock to scan your serverless functions.
Twistlock can gate both the CI and CD segments of your pipeline. Images are built in the CI segment of the pipeline. After an image is built, Twistlock scans it for vulnerabilities according to thresholds defined in your policy. If the image passes the scan, it is promoted to the registry. If the image doesn’t pass the scan then the build is failed.
In the CD segment, the orchestrator pulls updated images into the production to execute. Here again, Twistlock scans images for vulnerabilities according to thresholds you set in your policy before they run. If they comply with your policy, they’re allowed to execute. If not, an alert is raised, and the container is optionally blocked from running.
There are three points where Twistlock can block a container:
|
Why are there separate gates for the CI and CD segments? The CD gate protects against drift over time. An image deemed clean by the CI scanner today can be pushed to the registry. However, a few weeks later, new threat data might uncover a Critical severity vulnerability in an image previously deemed clean, and that should raise an alert.
The following matrix shows the starting point for your vulnerability management policy. In the beginning, you’ll want to minimize any disruption to your pipeline. Simply raise alerts in the CI and CD segments, and observe the affect. Set the severity threshold to Critical in your Twistlock vulnerability management policy to limit the scope to just the most important CVEs.
Then start clamping down the CI segment by blocking (failing) builds that emit non-compliant images. Ease into blocking by only failing builds when Critical vulnerabilities have vendor fixes. Set up grace periods so that devs have time to mitigate issues before builds are hard failed. Then lower the threshold from Critical to High to fail builds that emit images with both Critical and High vulnerabilities.
Next, configure blocking in the CD segment to prevent non-compliant images from executing in your prod environment. Although many customers strive to achieve blocking policies in both the CI and CD, everyone has a different comfort level. Your desired end-state might be to block in CI, but only to alert in CD.
For blocking in the CD segment, follow the same recipe as the CI segment. Start with easy achievable goals and slowly ratchet up. Blocking in the CD requires mature processes and trust in the Twistlock Intelligence Stream. You need to satisfy yourself that Twistlock’s threat feed minimizes false positives, such that blocking policies only trigger when there are, in fact, issues to be addressed.
Twistlock has hundreds of compliance checks Most of them are based on the CIS Benchmarks, including the Docker Engine benchmark, Kubernetes benchmark, and Distribution Independent Linux benchmark. Our security research team graded each check (Critical, High, Medium, and Low) so that you can focus on the issues most likely to expose your environment tp a direct attack from someone on the outside. The default rule sets all Critical and High checks to alert, and all Medium and Low checks to ignore.
When deploying Twistlock, you need to tune the default compliance policy to suit your environment. For example, many orchestrator components require more privileges than the average containerized app. Twistlock has a check, which we graded as High, for containers running as root. Some orchestrator containers, however, must run as root in order to function, so you’ll have to add rules that explicitly exempt these containers.
Review all your compliance issues in Compliance Explorer. For every 100 compliance issues, you should expect:
67% of issues are caused by your apps. Since you have full control over your apps, meet with your developers and remediate them.
33% of issues are caused by infrastructure issues.
Half of these issues will be due to the vendor configuration, which you can’t change. Create rules to ignore these issues.
The other half will be due to insecure defaults. Remediate them.
Runtime defense automatically models the intent of a container image so that it can be secured at runtime.
Models are composed whitelist rules that govern what the container can do, based on what Twistlock has learned about its baseline behavior. In most cases (90%), Twistlock builds models that fully capture the container’s intent. For the remaining cases (10%), the models do not fully account for everything the container does at runtime. For these cases, you must manually augment the model.
When you leave models untuned, you can get thousands of spurious audits. Your goal is to eradicate false positives and get your environment quiet. There should be no spurious audits. Spurious audits obscure alerts from truly anomalous, malicious activity, which you do want to monitor closely.
Most of the work to clean up spurious audits is front-loaded when an app is first deployed. When tuning models, focus on a single app at a time. Use the table filters in Console to show audits for just a single app. Sit down with the developer and review the audits to identify what behaviors the model hasn’t captured, then x the model. You have two choices for fixing a model: relearn or add a rule. In most cases, relearn the model to fix the issue. Relearning opens the model to new information about how the running container behaves. In a few cases, relearning simply cannot account for all the activity inside the container. In such a case, create a new runtime rule to augment the model.
You should create as few rules as possible. As a rule of thumb, for every 100 image models, you’ll need about 5 rules to augment the models.
Twistlock Cloud Native Firewalls learn the network topology of your applications and provide application-tailored micro segmentation for all your microservices. Twistlock offers two types of firewalls to protect your applications at the container level.
CNAF is a layer 7 web application firewall (WAF) If you have a container that handles web requests, configure CNAF to protect it. Initially, set CNAF to alert and monitor CNAF audits for rogue requests. Analyze rogue requests, then create rules to specifically block those types of requests.
CNNF is a layer 3 firewall that automatically models inter-container traffic. As part of the automatic behavioral learning at runtime, Twistlock builds out a topology of connections from one container to another. Initially, set CNNF to alert. After you’ve tuned your runtime models, set the action in your policy to "Prevent" to drop attempts to establish connections not whitelisted in the learned models.
After you’re up and running, your work consists of:
Responding to incidents, and other events. If rules are tuned, events should fire when there is high likelihood of an attack, when systems fall out of compliance, or when critical vulnerabilities are detected in your environment. Incident Explorer elevates raw audit data to actionable security intelligence by automatically correlating individual events generated by the firewall and runtime sensors to surface unfolding attacks.
Tuning rules and policies as new apps are released and existing ones are upgraded.
Upgrading Twistlock as new versions of the software are released.
After Twistlock is configured and tuned, it will collect lots of data. You need to determine how to extract data from Twistlock and send it to the right places in the right format so that it can be consumed and processed. Alerts, notifications, and other integration points help you get the right data to the right place.
Twistlock lets you configure multiple, separate, independent alert channels, so that each team can get just the data they want in the format that best suits their needs. For example, your vulnerability team could get data via email, the compliance team could get data in CSV files, and the security operations center (SOC) could get data from syslog.
If you run a SIEM, such as Splunk, then you can configure Twistlock to direct all audit messages to syslog, and then configure your SIEM to ingest audits from there. All Twistlock audits are well structured and fully documented to ease integration.
Twistlock supports numerous alert channels, including email, Slack, JIRA, and others. You can alert on any rule, as well as some other events, such as Defender health (when it gets disconnected) and admin activity (when changes are made to a rule or configuration in Console).
Alert labels close the loop between production and remediation. If you use Kubernetes or Docker labels to tag your resources, you can configure Twistlock to append any of the label key-value pairs to Twistlock audits. When an event fires, if the associated object has any of the specified labels, it is appended to the event. If the label contains email addresses, you can further configure Twistlock to send the audit to the recipients. These mechanisms route feedback directly to the owner or group responsible for the resource.
The API lets you build all sorts of integrations. For example, you might have a centralized tool where all vulnerability data is aggregated. You can use Twistlock’s API to extract the vulnerability data from your container ecosystem, and send it your central dashboard, with its own parsers and notifications system.
Best practice
Label your container resources, then leverage alert labels to automatically notify the right party when security issues arise. Besides segmenting and classifying resources, labels also let you track resources through their lifecycle, from the CI/CD pipeline to production.
Twistlock’s alert labels let you declare a list of labels. When something happens that violates your policy, the key-value pair of any declared label connected to the resource in question is automatically appended to generated audits. You can further automate alerting by declaring labels that contain email addresses. If a policy violation is triggered by a resource with a label that contains email addresses, an email alert is automatically sent to all targets in the list.