38
Breaking && Securing your ‘Docker’ized environments Manideep K

Breaking and fixing_your_dockerized_environments_owasp_appsec_usa2016

Embed Size (px)

Citation preview

Breaking && Securing your ‘Docker’ized environments

Manideep K

Shameless Bragging

• Masters Student + Security Researcher at “Carnegie Mellon”

• Authored a book on Info Sec & Ethical Hacking at the age of 20

• Featured in INDIA’s largest news papers and news channels

• Trained 15,000+ people in Infosec including corporates, students & cyber cops

• 10 certifications : ISO 27001:2013 ISMS LA, CCNA, CEH, JNCIP-SEC etc.

• Ex Team Lead – Core Security & Data Analytics at TCS

• Interest areas : Container Security, Application Security etc.

More details on “www.manideepk.com”

2

Experience with Docker containers

• Research at Carnegie Mellon University under renowned professors

• Cloud Security Research Intern @Adobe

• Co-Authored CIS Docker 1.12 Benchmark (You will see points from it today)

• Research Intern at Carnegie Mellon

3

Agenda Introduction

- One minute quick dive on containers- Container environment - Risk Areas

Breaking && Securing - Images - Container runtime- Daemon - Host - Communication and Registry security (if time permits)

Holistic view of entire container pipeline

Few tips for enterprises

Future

Conclusion

References

Q&A

4

Agenda Introduction

- One minute quick dive on containers- Container environment - Risk Areas

Breaking && Securing - Images - Container runtime- Daemon - Host

Holistic view of entire container pipeline

Few tips for enterprises

Future

Conclusion

References

Q&A

5

Containers?

Lightweight Application centric No more - “it works on my machine” Micro-services

Namespaces Isolates containers (PID, User, Network, IPC, Mount , UTS)CgroupsLimits, accounts and isolates resource usage (CPU, memory etc.)

BUZZ……….!Are containers brand new?

Img Ref: www.docker.com

6

Container environment (Docker)

Img Ref: Modified version of image on www.docker.com

7

8

Risk Areas

Images

Container Runtime

Daemon

Host System

Kernel/ Containers Isolation(Shared kernel)

Communication

Container pipeline (Docker)

Client <=> daemon communication

Communication with public/private registry

Registry’s security

Host securityDaemon security

Containers Images

9

Agenda Introduction

- One minute quick dive on containers- Container environment - Risk Areas

Breaking && Securing - Images - Container runtime- Daemon - Host

Holistic view of entire container pipeline

Few tips for enterprises

Future

Conclusion

References

Q&A

10

11

Containers do not contain

53% of decision makers are worried about security of containers*

Containers are not production-ready

Container Security (Docker) developed “a lot” in the past two years, is still developing and has lot of scope

Docker containers are now “production-ready**” . Google spins up more than 2 billion containers per week

Containers are the “FUTURE”

* Forrester/Red Hat Report , January 2015 ** You have to make them secure

Docker saysAt Docker we believe in “Secure by Default” *

NCC Group Report on containers saysDocker default security settings are quite good **

I say (and even you will, by end of this presentation)

“Docker is not secure by default, you have to make it secure…!”

* https://blog.docker.com/2016/04/docker-security/** https://www.nccgroup.trust/us/about-us/newsroom-and-events/blog/2016/april/understanding-and-hardening-linux-

containers/

12

My approach for explanation

“Point” (on later slides)

What breaks it & How to secure it

Why this approach?

13

Breaking and Securing

“Images”

14

Image Layers – Copy on Write (CoW) model

Manifest file of an image inside registry Image’s lifecycle

(Img Ref: www.docker.com)

Introduction to Images

15

• Where can I get Images?

- Docker hub , Docker Private Store

• Can I use them (directly)?

- No, Not Docker hub (at least general images) for enterprises (personal?)

- How about Docker Private store?

• Official images are scanned with Nautilus (general images are not)

• My analysis with images (sampling)

• Banyanops analysis* proved that Docker hub images are prone to severe vulnerabilities

such as Heartbleed, Shellshock etc. (2015 report)

Quick Facts from the report :

30% official images are vulnerable

70% general images are vulnerable

* : Below report was released in 2015. Though Docker initiated Nautilus after that and started mitigating the vulnerabilities in official images, it still has many unpatched. Also, unofficial Docker hub images are still untouched and are left with lot of vulnerabilities. So, though the percentage of vulnerabilities might not be exactly the same today, the essence of the report still remains the same**

** : You can open the official images on Docker hub and see the tags section of each image which shows the scan report of Nautilus for that particular image

Report Link: https://banyanops.com/blog/analyzing-docker-hub/

16

Lesson Learnt….!

Do Not directly use Docker hub images (not atleast general images and not at-least for enterprise use)

17

Manideep, Then what to do?

Prepare your own Docker Images (Oh Gosh, Lot of security controls to follow….!)

• How to write Dockerfiles (any other similar mechanism such as Packr) securely?

• How to maintain/consume images securely?

18

Writing Dockerfiles securely (our “Points” are here….)

1) Follow version pinning mechanisms for base images, packages etc. (Caching issue)

CIS Docker 1.12– 5.27

2) Download packages securely - Ex: GPG (MITM issue) : CIS Docker 1.12 – 4.11

3) Create a user for containers (Priv issue / root issue) : CIS Docker 1.12 – 4.1

4) Do not use any kind of update instructions alone in the Dockerfile (Caching issue) CIS Docker 1.12 – 4.7

5) Remove setuid, setgid permissions within the images (Priv Escalation) : CIS Docker 1.12 – 4.8

6) Do not install unnecessary packages within the images (Increased attack surface)

CIS Docker 1.12 – 4.3

7) Do not write secrets in Dockerfiles (Information Disclosure). Use Keywhiz, Vault etc.CIS Docker 1.12 – 4.10

8) Use COPY instead of ADD (Increased attack surface) : CIS Docker 1.12 – 4.9

9) Use HEALTHCHECK command (Best practice) : CIS Docker 1.12 – 4.6

10) Use gosu instead of sudo wherever possible ( TTY and signal forwarding issue)

11) Do not prefer to run multiple applications within one container (Increased attack surface)

19

How to maintain / consume images securely?

1) Docker Content Trust (Enable it!)

• Based on Thandy framework and is little complex to understand

• Ensures integrity, authenticity and freshness guarantees to images (Without DCT?)

• Extensive analysis done (Seems secure…)

• Create your own Notary setup

2) Images should be “vulnerability-free” and should be “frequently” scanned with tools like Twistlock, Nautilus etc. Tools should be capable of doing binary level analysis /hash based match and just not version string matching.

3) All images / packages within images must be up-to-date (Except compatibility issues)

4) Follow all CIS Docker benchmark standards

20

Breaking and Securing

“Container runtime”

21

Container runtime security

1) Ensure Docker commands get latest images (Caching issue) : CIS Docker 1.12 – 5.27

2) Enable PIDs cgroup per container (fork bomb) : CIS Docker 1.12 - 5.28

3) Do not use default docker0 bridge (ARP spoofing, MITM etc.) : CIS Docker 1.12 - 5.29

4) Set memory limits per container (DOS) : CIS Docker 1.12 - 5.10

5) Do not use privileged containers (Violation of least privilege) : CIS Docker 1.12 - 5.4

6) Bind incoming traffic to any specific interface (Info disclosure etc.) : CIS Docker 1.12 - 5.13

7) Do not share host’s process, user, UTS, IPC, network namespaces to containers (Info disclosure etc.)

8) Do not disable capabilities (significance of SYS_TIME, NET_RAW)

9) Prefer to set no new privileges flag (no runtime priv. gain atacks) : CIS Docker 1.12 – 5.25

10) Do not disable seccomp profiles (Container breakout) : CIS Docker 1.12 – 5.21

11) Set CPU shares appropriately (DOS) : CIS Docker 1.12 – 5.11

12) Use Apparmor / SELinux profiles (No access restriction) : CIS Docker 1.12 – 5.1, 5.2

13) Mounts (Violation of least privilege)

14) Beware of non namespaced components : kernel keyring etc. (Container breakout)

15) Adhere CIS Docker 1.12 benchmark

22

Breaking and Securing

“Daemon”

23

1) Enable user namespaces

- When user namespaces are enabled (which are disabled by default) root inside the container will be “no more” root on the host. (Without user namespaces?)

- Enable user namespaces when launching daemon with “- - userns-remap” flag

- Know about the restrictions (no read only filesystem for containers, no - -privileged flag etc.)

2) Disable communication between containers

- By default, all containers are on the same bridge and can communicate (intercept traffic)

- Add “- - icc = false” flag when launching daemon which would prevent container communication

- Use user-defined networks for allowing group of containers to communicate (DO NOT use legacy container linking method as it leaks secrets stored in environment variables between containers)

3) Enable TLS & authentication for communication between daemon and client

- By default, communication between docker daemon and client are not encrypted and there is no authentication between them.

- Enable TLS & CA authentication at daemon level which would prevent attacks such as MITM, server or client impersonation etc.

Daemon Security

24

Daemon Security

4) Enable ulimits for your docker daemon (You can do it at runtime per container)

- ulimits are great way to restrict lot of resources such as CPU time limit, max file descriptors that can be opened, max no of processes etc.

- By default, no ulimits are set . You can enable them by using flags.

Ex: “- -default-ulimit nproc=10”

5) Use authorization plugins

- By default, every user having access to daemon can run any Docker command. Auth plugins

provides granular control on “who runs what” at daemon level.

- You can enable auth plugins while launching daemon with the flag

“- -authorization-plugin=plugin1 –authorization-plugin=plugin2 …………….”

6) Ensure only trusted users will have access to docker daemon as all the memberswithin the docker group will have effective root privileges

25

Breaking and Securing

“The host where containers are hosted”

26

Security controls on the host

1) Maintain up-to-date Linux kernel- They have all vulnerabilities fixed and latest security controls included- Ex: PIDs cgroup (prevents fork bomb) is only available in kernels > 4.3

2) Apply kernel hardening policies such as GRSecurity, PaX etc.- These are kernel patch sets which help in preventing multiple kind of memory, access control

exploits etc. These would prevent exploitation even though attacker was able to break out of container. Also, ensure that these kernel patches do not interfere with your Docker setup (test with sysctl and then lock down)

3) Harden the host as per your enterprise / CIS / SANS guidelines

4) Keep Docker always up-to-date

27

Container pipeline (Holistic View) – Communication & Registry Security

Client <=> daemon communication

Communication with public/private registry

Registry’s security

Host securityDaemon security

ContainersImages

Img Ref: Modified version of image on www.docker.com

*

*

28

Agenda Introduction

- One minute quick dive on containers- Container environment - Risk Areas

Breaking && Securing - Images - Container runtime- Daemon - Host

Holistic view of entire container pipeline

Few tips for enterprises

Future

Conclusion

References

Q&A

29

Few tips for enterprises

• Setup your own in-house registry having “secure and ready-to-consume” images

• Prepare customized auth plugins, seccomp, apparmor profiles as per your requirements

• Do not share secrets in environment variables

• Use tools like Twistlock, Scalock etc. to secure your container environments. These tools provide capabilities like Image lockdown, RBAC, automatic profiling etc.

• Use TPM, TCB etc. in Docker environments (Add docker daemon to TCB)

• Use image optimization techniques (Do not compromise security!)

• Container specific rules to be created in SIEM

• Group containers on VM’s (basing on trust, operational categories)

• Employ least privilege, defense in depth, reduced attack surface principles

• Employ file integrity monitoring tools for critical files

• Prepare separate patch management, vulnerability management etc. procedures for container environments

• Adhere to latest CIS Docker benchmark or prepare your own internal benchmarks customizing standard CIS Docker benchmark as per your organizational requirements

30

Agenda Introduction

- One minute quick dive on containers- Container environment - Risk Areas

Breaking && Securing - Images - Container runtime- Daemon - Host

Holistic view of entire container pipeline

Few tips for enterprises

Future

Conclusion

References

Q&A

31

Future (My 2¢)

• Hardware level Isolation ( Container’s performance + VM security )

- Clear Containers (QEMU+KVM : Very light weight VM ‘s. Though not up to Docker

performance, they are optimizing more).

- Has lot of scope. My area of work for next few days.

• No more shared kernel as in clear containers

• Trusted place for getting Images. Ex: Docker store

• Advancement in container security tools : Twistlock, Scalock are only tools which cover

entire container ecosystems today. They are adding more features

• In-built “proper” hardening of host when Docker installed (GRSEC, PaX etc.)

• Docker Images encryption – similar to VM (Most security controls of VM will be soon shipped to container ecosystem)

• Portability between container technologies (lot of current work)

• Name spacing the missing components: Kernel keyring, SYS_TIME etc.

• Enhanced security controls on the daemon, images and container runtime

Ex: better default seccomp by in-built profiling etc.

32

My Next Plans

• Research on hardware level isolation to containers

• In-depth Research on other container technologies

• Whitepaper (or possibly a book? ) on Docker Security

• Continue contribution to CIS Docker Benchmark (Please join!)

• Will be presenting at RSA conference (poster under RSA Scholar Program), Grehack etc.

• Feel free to reach me if you have similar interests or want to collaborate

33

Conclusion

34

References

1. https://www.nccgroup.trust/globalassets/our-research/us/whitepapers/2016/april/ncc_group_understanding_hardening_linux_containers-1-1pdf

2. http://www.oreilly.com/webops-perf/free/files/docker-security.pdf

3. http://container-solutions.com/content/uploads/2015/06/15.06.15_DockerCheatSheet_A2.pdf

4. CIS Docker Benchmark 1.12

5. http://csrc.nist.gov/news_events/cif_2015/research/day1_research_430-530.pdf

6. http://www.slideshare.net/SreenivasMakam/docker-security-overview?qid=49963984-be1a-4322-a3a8-e5a2963127c0&v=&b=&from_search=1

7. http://www.slideshare.net/Docker/docker-security-workshop-slides

8. http://www.slideshare.net/Docker/securing-the-container-pipeline-at-salesforce-by-cem-gurkok-63493231

9. https://clearlinux.org/features/clear-containers

10. https://arxiv.org/pdf/1501.02967.pdf

11. http://www.slideshare.net/Docker/the-golden-ticket-docker-and-high-security-microservices-by-aaron-grattafiori?qid=dd633f11-1c78-42ce-86f0-88068690a6ee&v=&b=&from_search=2

12. https://docs.docker.com/engine/security/

13. http://www.slideshare.net/Docker/docker-security-deep-dive-by-ying-li-and-david-lawrence

35

Special Thanks to

Adobe

36

Q & A time

37

Thank you…!

Reach me on www.manideepk.com for any questions

38