I'm zany for zones! - sysunconfigsysunconfig.net/unixtips/zanyzones.pdf · I'm zany for...

I'm zany for zones!Linda KateleySolaris 10 Adoption Specialistlinda.kateley@sun.com

© 2006, Sun Microsystems, Inc. 2

Agenda

• Zone Basics• Zones/Containers Admin

> Filesystem> Patching> migration

• Next generation SCLA• Next generation Xen

Dynamic Tracing (DTrace)

Solaris Containers

Predictive Self-Healing

ZFS

Secure Execution

Integrated SAN Support

Compatibility Guarantee

Solaris 10

Traditional Resource Management

Ne tw o rk

Application

Server

Utilization Level

CustomerAA

Web Web ServerServer

CC

Web Web ServerServer

BB

Web Web ServerServer

DD

App App ServerServer

EE

DB DB ServerServer

● One application per server

● Size every server for the peak

● Avg. utilization rate is 20%–30%

Solaris ContainerResource Management

ProjectProjectPro je ctPro je ct ProjectProject

ProjectProject ProjectProject

Fair- Sh areCPU Sch e d u le r

Ne tw o rkIPQo S

Do m ain 1Do m ain 1

g ro u p .d b ag ro u p .d b a oracl2oracl2oracl1oracl1

iASiAS user .bobuser .bob

● Workload Metering

● Sub-CPU Partitioning

● Control CPU, Memory, and Network

Zones Block Diag ram

network device

(ce0)

storage complex

global zone (serviceprovider.com)blue zone (blueslugs.com)

web services

(Apache 1.3.22, J2SE)enterprise services

(Oracle 8i, IAS 6)

foo zone (foo.net)

network services

(BIND 8.3, sendmail)

login services

(OpenSSH sshd 3.4)

zoneadmd

beck zone (beck.org)

web services

(Apache 2.0)network services

(BIND 9.2, sendmail)

remote admin/monitoring

(SNMP, SunMC, WBEM)

platform administration

(syseventd, devfsadm, ...)

core services

(ypbind, automountd)

core services

(ypbind, inetd, rpcbind)

core services

(inetd, ldap_cachemgr)

core services

(inetd, rpcbind, ypbind,

automountd, snmpd, dtlogin,

sendmail, sshd, ...)

zone root: /aux0/blueslugs zone root: /aux0/foonet zone root: /aux0/beck

network device

(ge0)

zone management (zonecfg(1M), zoneadm(1M), zlogin(1), ...)

ce0:

2

ge0

:2

ce0

:1

zco

ns

zco

ns

zco

ns

zoneadmdzoneadmd

/usr

/usr

/usr

/op

t/yt

Ap

plic

ati

on

En

viro

nm

ent

Vir

tua

l

Pla

tfo

rm

ge0

:1

Creat ing a z one

g lob al# z onecfg - z z one1

z one1 : No such z one con figu red

Use 'crea te ' to b eg in configu ring a new z one .

z onecfg :z one1 > crea te

Sett ing 's fo r the z onezonecfg:zone1> set zonepath=/zoneroots/zone1

zonecfg:zone1> set autoboot=true

zonecfg:zone1> add net

zonecfg:zone1:net> set address=192.9.200.67

zonecfg:zone1:net> set physical=hme0

zonecfg:zone1:net> end

zonecfg:zone1> ^D

#zoneadm list -c

In s ta lling the z oneglobal# zoneadm -z zone1 installConstructing zone at /zoneroot/zone1/rootCreating dev directoriesCreating dev linksCopying packages and creating contents fileCopying files and directoriesSetting up /etc/motdSetting up /etc/inittabSetting up /etc/vfstabSetting up /var/yp/aliasesConfiguring files

boot the zone

global# zoneadm -z zone1 boot

– Took about .6 seconds on ferrari

● global# zlogin -C zone1

● [Connected to zone 'mydesktop' console]

● <Run through sysid tools as usual to do initial customization>

© 2006, Sun Microsystems, Inc. 11

Solaris 10 Containers

demo

© 2006, Sun Microsystems, Inc. 12

Solaris 10 Containers

Administrating zonesresource managementpoolspatchingfilesbackup

Zones and Resource Pools

cpu1

Resource Pool AResource Pool B

LocalZone1

LocalZone2

LocalZone3

Global Zone

cpu2 cpu3 cpu4 cpu5 cpu6 cpu7 cpu8

Default Resource Pool● Processor set (now)

● Scheduling Class (now)● Memory Set (S10U1)● Swap Set (TBD)

6

3

4

5

4

Two Level FSS

3

1

2

1

twilight

drop

fracture

global

Shares Allocatedto Zones Shares Allocated by

Zone Administrator

FSS-TS-IAControlling CPU Consumption

The Fair Share Scheduler can be used to control CPU consumption of the instances.

The Fair Share Scheduler is not the default scheduler and must be enabled using the dispadmin(1M)command:

# dispadmin -d FSS

ProjectsCommand Description

projadd(1M) adds a new project to the local project database

projmod(1M) modifies a project entry in the local project database

projdel(1M) deletes a project entry from the local project database

projects(1) displays project membership for a user

newtask(1) switches to a project

Projects/etc/project

projname:projid:comment:user-list:group-list:attributes

/etc/project contains five standard projects:

system, user.root, noproject, group.staff, default

The system project is used for all system processes and daemons.

All of roots processes run in the user.root project.

The noproject project is a special for IPQoS.

The group.staff project will be used for all users in the group staff

The default project serves as a catch-all and will be used for users not matching any of the other projects.

Projects/etc/project

projname:projid:comment:user-list:group-list:attributes

#projadd

-U user,user

-G group,group

-c comment or description

-K value=attributes

-p unique project number (if not given will give next available

name

#projects -l

ProjectsAdmin commands

#projects -l will show all defined projects

#id -p – will show users project

#newtask -p project exec – allows us to execute in a project

#prstat -J – show per project consumption

#prstat -T – show per task consumption

Projects cpu control- priv

#projmod -K “project.cpu-shares=(priv,value,action)” project

Privilege level determines who can modify

There are three privilege levels:

basic -the owner of the calling process

privileged -only privileged (superuser)users can change

system -the threshold is fixed for the lifetime of the operating system instance

Projects cpu control- value

#projmod -K “project.cpu-shares=(priv,value,action)” project

CPU Shares Configuration

Every project can be assigned a project.cpu-shares resource control.Projects that do not have this resource control are assigned 1 share by the system.

Shares are numeric values

Shares are not percent

projecta 50, projectb 50 is the same as projecta 200, projectb 200

Projects cpu control-action

#projmod -K “project.cpu-shares=(priv,value,action)” project

The action defines the action to be taken when the threshold is exceeded.

There are three possible actions:

deny -this denies resource requests for an amount that is greater than the threshold

signal -this sends the specified signal to the process exceeding the threshold value.

none -this causes no action when the threshold is exceeded

ProjectsAvailable Resource Controls

Resource Control Description

process.max-port-events maximum allowable number of events per event port process.crypto-buffer -limit maximum number of bytes allocated for copying process.max-crypto-sessions maximum number of entries in the session table process.add-crypto-sessions number of entries added when enlarging the session table process.min-crypto-sessions minimum number of entries in the session table process.max-msg-messages maximum number of messages on a message queue process.max-msg-qbytes maximum number of bytes of messages on a message queue process.max-sem-ops maximum number of semaphore operations per semop call process.max-sem-nsems maximum number of semaphores per semaphore set process.max-address-space maximum size of the address space in bytes process.max-file-descriptor maximum index in filedescriptor table process.max-core-size maximum core file size in bytes process.max-stack-size maximum size of the stack segment in bytes process.max-data-size maximum size of the data segment in bytes process.max-file-size maximum file size in bytes

ProjectsAvailable Resource Controls

Resource Control Description -contprocess.max-cpu-time maximum CPU time in seconds task.max-cpu-time maximum CPU time in secondstask.max-lwps maximum number of simultaneously available LWPs project.max-port-ids maximum allowable number of event ports project.max-shm-memory maximum size of System V shared memory in bytes project.max-shm-ids maximum number of System V shared memory

segments project.max-msg-ids maximum number of System V message queues project.max-sem-ids maximum number of System V semaphores project.cpu-shares the number of CPU shareszones.cpu-shares number of CPU shares per zone

Projects cpu control

You can also control cpu shares dynamically with

prctl(1M) get or set resource controls on a running process,task or project

rctladm(1M)display or modify global state of system resource controls

# prctl -n project.cpu-shares -r -v # -i project projname

-n name of value

-r replace

-v new value

-i project, task, process

Configuring per zone shares#dispadmin -d FSS#reboot

#zonecfg -z namezonecfg:zone1> add rctlzonecfg:zone1:rctl> set name=zone.cpu-shareszonecfg:zone1:rctl> add value

(priv=privileged,limit=10,action=none)zonecfg:zone1:rctl> endzonecfg:zone1> verifyzonecfg:zone1> commitzonecfg:zone1> ^D

#prctl -n zone.cpu-shares -r -v 25 -i zone zonename

© 2006, Sun Microsystems, Inc. 27

Solaris 10 Containers

Rm demo

PoolsSince solaris 2.6 we have had psrset.

The syntax looked like

#psrset -a name cpu0 cpu1

We could then bind a process to the set using

#pbind pid name

When the cpu was idle nothing else could use it

PoolsEnter pools

We can set a min and max number of cpu's in a pool which one or more processes, projects or task can be assigned to.

The controlling daemon is the poold which will start at boot with the existence of a

/etc/pooladm.conf file

Pools- configEnabling pools

#pooladm -e

Disabling pool

#pooladm -d

remember that pools will be enabled at boot with the existence of the file.

Pools- configCreating the file

#pooladm -s

This will create an xml /etc/pooladm.conf file which is best viewed with

#poolcfg -c info

Which says give me info about the currect config.

Pools- configModifing the config- first create the set

# poolcfg -c 'create pset linda (uint pset.min = 2; uint pset.max = 10)'

Then create a pool

# poolcfg -c 'create pool kateley'

Connect the set to the pool

# poolcfg -c 'associate pool kateley (pset linda)'

Zone Pools

Pools

Zones may be bound to pools

Automatically via zone configuration

#poolbind(1M) -p poolname -i zoneid zonename

All processes in zone bound to same pool

Zone File Sys tem sGlo b a l ro o t /

/ z o n e

1 2 3

/ u s r / d e v . . . . . . . .. . .

/ b in / u s r / d e v

Zo n e ro o t / Zo n e v ie w

Glo b a l v ie w

e tc . . .

... .... ....

Zone 1

© 2006, Sun Microsystems, Inc. 35

Solaris 10 Containers

• Sparse-root vs. whole-root• Read-write vs. read-only• File access vs. device access• Backups

File Systems

© 2006, Sun Microsystems, Inc. 36

Solaris 10 Containers

• RW or RO access in LZ and GZ• Easily accessible from GZ (by root)• Can be unmounted and remounted by GZ (if not used)• Simplest method• Method:

global# mount /dev/dsk/c1t0d0s6 /export/zones/zone1/opt/localglobal#mount -F lofs /dir /export/zones/zone1/dir

File System Creation – Direct Mount

© 2006, Sun Microsystems, Inc. 37

Solaris 10 Containers

• Can mount in multiple zones

• dir= is mount point in zone, special=name of dir to mount

• Method:

global# zonecfg -z zone1add fs

set dir=/opt/localset special=/export/opt/localset type=lofsend

exitglobal# zoneadm -z zone1 boot

File System Creation - lofs

© 2006, Sun Microsystems, Inc. 38

Solaris 10 Containers

• After LZ boots, GZ can unmount and re-mount• Method:

global# newfs /dev/dsk/c1t0d0s6global# zonecfg -z zone1

add fsset dir=/opt/localset special=/dev/dsk/c1t0d0s6set raw=/dev/rdsk/c1t0d0s6set type=ufsadd options [ro,nodevices]end

exitglobal# zoneadm -z zone1 boot

File System Creation – UFS Mount

© 2006, Sun Microsystems, Inc. 39

Solaris 10 Containers

• Method:

global# zonecfg -z zone1add device

set match=/dev/dsk/c1t0d0sexit

global# zoneadm -z zone1 boot

File System Creation – device in zone

© 2006, Sun Microsystems, Inc. 40

Solaris 10 Containers

• Can only be done before zone install• Method:

global# zonecfg -z zone1remove inherit-pkg-dir dir=/usrremove inherit-pkg-dir dir=/libremove inherit-pkg-dir dir=/platform

remove inherit-pkg-dir dir=/sbinexit

global# zoneadm -z zone1 boot

Whole root zone

© 2006, Sun Microsystems, Inc. 41

Solaris 10 Containers

• Can be created using a script or template• Method:

global# zonecfg -z zone1create -t zone

global#zonecfg -z zone1export -f filename

global#zonecfg -z zone2create -f filename

Creating

© 2006, Sun Microsystems, Inc. 42

Solaris 10 Containers

• http://www.opensolaris.org/os/community/zones/faq/

Info

© 2006, Sun Microsystems, Inc. 43

Solutions from Sun• It's all about Customer Choice

Multiple OS's Single OS

Hard Partitions Virtual Machines OS Virtualization Resource Management

Dynamic SystemDomains

Solaris Containers(Zones + SRM)

Solaris Containersfor Linux Application

Solaris ResourceManager (SRM)

Logical DomainsXen

VMware

Trend to flexibility Trend to isolation

© 2006, Sun Microsystems, Inc. 44

IndependentIndependentUsersUsers

SeparateSeparateNetworksNetworks

IndependentIndependentStorageStorage

IsolatedIsolatedContainersContainers

Solaris 10Solaris 10 Solaris 10Solaris 10 Solaris 10Solaris 10

Solaris 10Solaris 10 Solaris 10Solaris 10

129.30.9.1129.30.9.1 10.6.40.510.6.40.5 10.6.40.610.6.40.6

192.9.9.4192.9.9.4 76.32.129.15576.32.129.155

Extending Solaris ContainersToday ...

Single Kernel

Single Operating System

Solaris Solaris

© 2006, Sun Microsystems, Inc. 45

IndependentIndependentUsersUsers

SeparateSeparateNetworksNetworks

IndependentIndependentStorageStorage

IsolatedIsolatedContainersContainers

Solaris 10Solaris 10 Red Hat 3Red Hat 3 Red Hat 4Red Hat 4

CentOSCentOS Solaris 10Solaris 10

129.30.9.1129.30.9.1 10.6.40.510.6.40.5 10.6.40.610.6.40.6

192.9.9.4192.9.9.4 76.32.129.15576.32.129.155

Extending Solaris Containers... Tomorrow ...

Single Kernel

Multiple Operating Environments

Solaris Solaris

Xen

• Open source hypervisor technology developed at the University of Cambridge

http://www.cl.cam.ac.uk/Research/SRG/netos/xen/http://www.opensolaris.org/os/community/xen

• 2006: Hardware Virtualization Everywhere• x64 cpu capabilities (VT-x, Pacifica)• Workload consolidation• Community software wanted!

“Every grad student will have their own hypervisor”

Xen 3.x Architecture

Event Channel Virtual MMUVirtual CPU Control IF

Hardware (SMP, MMU, physical memory, Ethernet, SCSI/IDE)

NativeDeviceDriver

GuestOS(Solaris)

Device Manager & Control s/w

VM0

NativeDeviceDriver

GuestOS(XenLinux)

UnmodifiedUser

Software

VM1

Front-EndDevice Drivers

GuestOS(Solaris)

UnmodifiedUser

Software

VM2

Front-EndDevice Drivers

UnmodifiedGuestOS(WinXP))

UnmodifiedUser

Software

VM3

Safe HW IF

Xen Virtual Machine Monitor

Back-End Back-End

VT-x

32/64bit

AGPACPIPCI

SMP

dom0 domU1 domU2 domU3

Key Capabilities

• Checkpoint/Restart and Live Migration• N1 provisioning• Grid operations: virtual platform

• Multiple OSes running simultaneously• Linux, Solaris, Windows XP• No longer a boot-time decision

• Special purpose kernels• Drivers, filesystems

SPECweb99 Migration Experiment

From LinuxWorld 2005 Virtualization BoF

Solaris Con ta iner Conso le

● Browser b ased GUI to m anage Con ta iners

● Contro ls resource m anagem en t on So laris 8 OS and So laris 9 OS

● Contro ls Zones on So laris 1 0● Uses the Sun MC 3 .5 Up d ate 1

in fras tructu re

Conta iner Managem entContainerContainerManagement Management AgentAgent

Do m ain 1Do m a in 1 Do m ain 3Do m ain 3

Su n Se rv e r 1Su n Se rv e r 1 Su n Se rv e r 2Su n Se rv e r 2

Do m ain 2Do m a in 2

● Container Management– Create/Delete/Modify Containers

● Centralized Management of Multiple Systems– Manage all the Containers across the network

● Container Replication– Recreation a Container on a separate system

● Container and Process Monitoring– Zoom into a Container to verify its contents

Solaris Container ConsoleFea tures & Benefits

© 2006, Sun Microsystems, Inc. 53

Webmin

© 2006, Sun Microsystems, Inc. 54

About Webmin

• A web-based interface for UNIX system administration

• It comes with Solaris 10, or get it at http://www.webmin.com

•

linda.kateley@sun.com

February 2006