Containerizing your Security Operations Center

Containerizing your SOC

@jimmesta

OWASP Santa Barbara Founder

AppSec California Organizer

Works at Invoca

Was consulting, now “defensing”

I really like containers

Greetings from Sunny AppSec California!

Location: Santa Monica, Ca.

Date: January 23-25, 2017

Why: Because Winter

2017.appseccalifornia.org

Security Confessions:

A Time for Healing

What are employees saying about your security program?

Security as a [ ]

Security as a Service?

Security as a Magic Unicorn?

Security as a Bottleneck

Security as a Black Hole

Security as a “No” Machine

Security as a Hot Potato

Security as a PDF Generator

We are all under-staffed

We are all over budget

We are all too busy

Can we DevSecOps our way out of this?

Step 1: Install XCode Command Line Tools

xcode-select --install

Nice! It looks like I get to compile some stuff.

Step 2: Make sure Java is updated

Dang. My Java is out of whack. What did I do? I’ll just update…

java --version

20 minutes later...

Step 3: Install Homebrew

But I use Macports and ZSH.. where’s my .bash_profile?

source ~/.bash_profile

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/.../)"

brew tap homebrew/versions

Step 4: Update $PATH and Install Dependencies

Wait a minute. I need a local Postgres DB to run this thing?

echo PATH=/usr/local/bin:/usr/local/sbin:$PATH >> ~/.bash_profile

brew install nmap && brew install postgresql

Step 5: Initialize the DB

What?! Postgres didn’t initialize? Forget this. Hacking is hard.

cp /user/local/Cellar/postgresql/9.4.0/.../...

initdb /usr/local/var/postgres

launchctl load -w ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist

You just lost a golden opportunity to foster a co-worker's interest in security.

How can we make our security tooling more about using the tool and less about maintenance?

whitesourcesoftware.com

Docker is an Open Source engine to pack, ship, and run any application as a lightweight container.

Why U No Just Virtualbox!?

Traditional Virtual Machines

Source: https://www.docker.com/what-docker

Docker Infrastructure

Source: https://www.docker.com/what-docker

Docker provides a user friendly API to create containers.

Images use layers for efficiency and speed.

Build the Docker image once and use it all over the place.

Minimize concerns around compatibility and dependencies.

So what about our “Securious” Dev who just wanted to run Metasploit?

Step 1: Install Docker

Nice! That was super easy to point and click.

Step 2: Run Command

That just..worked?

docker run -t -i linuxkonsult/kali-metasploit

Step 3: Profit

FROM linuxkonsult/kali

MAINTAINER Tom Eklöf "tom@linux-konsult.com"

ENV DEBIAN_FRONTEND noninteractive

ADD ./init.sh /init.sh

RUN apt-get -y update ; apt-get -y --force-yes install ruby metasploit-framework

CMD /init.sh

Dockerfile

#!/bin/bash

/usr/share/metasploit-framework/msfupdate

/usr/share/metasploit-framework/msfconsole

init.sh

What could go wrong with blindly pulling Docker images?

Always inspect and vet the Dockerfile before blindly using it.

Tools like Clair, Quay, and Anchore can help identify vulnerabilities in images.

https://github.com/zaproxy/zaproxy/wiki/Docker

https://hub.docker.com/r/jmbmxer/threadfix/

https://github.com/blacktop/docker-cuckoo

https://github.com/wazuh/docker-ossec-elk

Security is Catching on

docker run is useful and all but how do we get these containers out to the world for others to use?

K8S - A Gentle Introduction

Kubernetes is an open-source platform built to automate deployment, scaling and orchestration of containers.

K8S is portable. Clusters can be deployed on a public/private cloud, on prem, and even on your laptop.

K8S is customizable. It is modular and extensible to fit nearly any use-case.

K8S is scalable. It provides self-healing, auto scaling, and replication.

There are others!

- Don’t orchestrate for the sake of orchestration (or because the cool kids are doing it)

- Containers first, then orchestration

- docker-compose does a fine job for many things

Core Concepts

https://flic.kr/p/bNpyRp

clustervirtual machines that Kubernetes

manages

clusternodemaster node node

clusternodemaster node node

node node node

node node node

node node node

node node

node node node

node node nodemaster

master

node

podgroup of

containers sharing storage

and network

podcontainer container container

volume A volume B

network interface

pod.yaml

apiVersion: v1kind: Podmetadata: name: owasp-appspec: containers: - name: owasp-app image: owasp-app - name: nginx-ssl image: nginx ports: - containerPort: 80 - containerPort: 443

pod.yamlnodemaster node node

pod.yamlnodemaster node node

pod.yamlnodemaster node node

deploymentensure N pods

are up and running

deploy.yaml

kind: DeploymentapiVersion: extensions/v1beta1metadata: name: frontendspec: replicas: 4 selector: role: web template: metadata: name: web labels: role: web spec: containers:

- name: owasp-app image: owasp-app - name: nginx-ssl image: nginx ports: - containerPort: 80 - containerPort: 443

deploy.yaml

kind: DeploymentapiVersion: extensions/v1beta1metadata: name: frontendspec: replicas: 4 selector: role: web template: metadata: name: web labels: role: web spec: containers:

- name: owasp-app image: owasp-app - name: nginx-ssl image: nginx ports: - containerPort: 80 - containerPort: 443

deploy.yaml

kind: DeploymentapiVersion: extensions/v1beta1metadata: name: frontendspec: replicas: 4 selector: role: web template: metadata: name: web labels: role: web spec: containers:

- name: owasp-app image: owasp-app - name: nginx-ssl image: nginx ports: - containerPort: 80 - containerPort: 443

deploy.yaml

kind: DeploymentapiVersion: extensions/v1beta1metadata: name: frontendspec: replicas: 4 selector: role: web template: metadata: name: web labels: role: web spec: containers:

- name: owasp-app image: owasp-app - name: nginx-ssl image: nginx ports: - containerPort: 80 - containerPort: 443

deploy.yamlnodemaster node node

deploy.yamlnodemaster node node

deploy.yamlnodemaster node node

10.0.0.1 10.0.0.210.0.0.3

10.0.0.4

serviceabstraction layer that enables pod communication

servicenodemaster node node

10.0.0.1 10.0.0.210.0.0.3

10.0.0.4

servicemaster

service

servicemaster

service

service

servicemaster

service

service

public load balancer

tools.beardsec.com

svc.yaml

kind: ServiceapiVersion: v1metadata: name: web-frontendspec: ports: - name: http port: 80 targetPort: 80 protocol: TCP selector: role: web type: LoadBalancer

svc.yaml

kind: ServiceapiVersion: v1metadata: name: web-frontendspec: ports: - name: http port: 80 targetPort: 80 protocol: TCP selector: role: web type: LoadBalancer

svc.yaml

kind: ServiceapiVersion: v1metadata: name: web-frontendspec: ports: - name: http port: 80 targetPort: 80 protocol: TCP selector: role: web type: LoadBalancer

svc.yaml

kind: ServiceapiVersion: v1metadata: name: web-frontendspec: ports: - name: http port: 80 targetPort: 80 protocol: TCP selector: role: web type: LoadBalancer

namespacemanage different environments in the same cluster

ns.yamlkind: NamespaceapiVersion: v1metadata: name: sec-tools

kubectl

master apiserver

HTTPS

schedulerReplication controller

node

node kubelet

pod pod pod

proxy External LB

Sounds great! What about security?

apiserver

Authentication(Who can access the

cluster?kubectl

Authorization(What can

they access?)

Admission Control

(Which policies are applied for

this user?

Access Granted

https://

K8S Security Model

- K8S API typically serves traffic over TLS- Self-Signed Cert provisioned on

operators laptop in $USER/.kube/config

Transport Security

apiserver

Authentication(Who can access the

cluster?kubectl

Authorization(What can

they access?)

Admission Control

(Which policies are applied for

this user?

Access Granted

https://

- Supports many authentication modules:HTTP Basic, OpenID, Tokens, Client Cert, Keystone

- Multiple modules can be specified

Authentication

apiserver

Authentication(Who can access the

cluster?kubectl

Authorization(What can

they access?)

Admission Control

(Which policies are applied for

this user?

Access Granted

https://

- Every HTTP request is authorized get, list, create, update, etc.

- Request attributes are checked against policy

Authorization

apiserver

Authentication(Who can access the

cluster?kubectl

Authorization(What can

they access?)

Admission Control

(Which policies are applied for

this user?

Access Granted

https://

Authorization

--authorization-mode=AlwaysAllow allows all requests;

use if you don’t need authorization.

--authorization-mode=ABAC allows for a simple

local-file-based user-configured authorization policy.

--authorization-mode=RBAC is an experimental

implementation which allows for authorization to be driven by the

Kubernetes API.

Role Resource TypesRole

ClusterRole

RoleBinding

ClusterRoleBinding

http://kubernetes.io/docs/admin/authorization/

rb.yaml

kind: RoleBindingapiVersion: rbac.authorization.k8s.io/v1alpha1metadata: name: read-pods namespace: sec-toolssubjects: - kind: User name: jimmyroleRef: kind: Role namespace: sec-tools name: pod-reader apiVersion: rbac.authorization.k8s.io/v1alpha1

- Intercept requests prior to object creation

- May mutate incoming request to apply system defaults

Admission Controllers

apiserver

Authentication(Who can access the

cluster?kubectl

Authorization(What can

they access?)

Admission Control

(Which policies are applied for

this user?

Access Granted

https://

Admission Controllers

AlwaysPullImages

DenyEscalatingExec

ResourceQuota

http://kubernetes.io/docs/admin/admission-controllers/

Secrets Everywhere!

K8S Secret Object

- Secrets can only be accessed by pods in the same namespace

- Secrets are only sent to nodes with pods that require it

- Not written to disk - stored on tmpfs- Deleted once dependent pod is removed

Buyer Beware

- Secrets are stored in plaintext on the apiserver (etcd)- Protect etcd with your life

- Don’t forget what OWASP taught you!- Secrets in logs, app security, etc.

- Anyone with root on any node can read secrets by impersonating kubelet

Vault

- It works! But no official K8S support (yet)

- API driven, do what you will- Customize your deployment

#!/bin/bash

PASSWORD="$(vault read -field=value secret/password | base64)"

# Create YAML object from stdin

cat <<EOF | kubectl create -f -

apiVersion: v1

kind: Secret

metadata:

name: mysecret

type: Opaque

data:

password: "${PASSWORD}"

EOF

```

secret.yaml

apiVersion: v1kind: Secretmetadata: name: owasp-secretstype: Opaquedata: username:d293IHlvdSBkZWNvZGVkIGl0 password: Z29vZCBmb3IgeW91 host:bm90aGluZyBqdWljeSB0aG91Z2g=

deploy.yaml

kind: DeploymentapiVersion: extensions/v1beta1metadata: name: frontendspec: replicas: 4 selector: role: web template: metadata: name: web labels: role: web spec: containers:

- name: owasp-app image: owasp-app

env: - name: OWASP_PASS valueFrom: secretKeyRef: name: owasp-secrets key: password

ports: - containerPort: 443

Security Hygiene

- Restrict SSH access to nodes- Only use trusted images- Regularly apply updates to your K8S

environment (including kubectl)- Log all of the things- Apply SecurityContext to deploymentsrunAsNonRoot, readOnlyRootFilesystem

Demo (sort of)

- 2 node cluster running on GCE- Kubernetes 1.4

- Maintain one K8S cluster- Deploy and scale security tooling - DevSecOps all the things- We are part of this container journey

together

Security can be an enabler

ResourcesKubernetes Bootcamp

CloudSOC

Minikube

Questions?