Running Asterisk on virtualized environments

http://www.irontec.com

Running Asterisk in virtualized environments

saghul

Saúl Ibarra Corretgé <[email protected]>

http://www.saghul.net

mailto:[email protected]

AMOOCON 2009 - Running Asterisk in virtualized environments

2

About me

● saghul– VoIP enthusiast, playing around with Asterisk since 2k5– Started testing virtualization solutions 3 years ago– GNU/Linux lover :)

● Irontec– Small company located at the north of Spain– Web design, VoIP systems, ... everything under GNU/Linux– Always trying to be at the bleeding edge of technology


3

About this presentation

● Presentation (ODP + PDF)● Configuration examples● Test results

http://www.irontec.com/~saul/amoocon2k9.tar.gz

http://www.irontec.com/~saul/amoocon2k9.tar.gz


4

Index

1. Introduction to virtualization2. Virtualization types3. Test-bench setup4. Testing

1. Xen2.KVM3.No virtualization

5. Results analysis6. Conclusions

Introduction to virtualization


6

Introduction to virtualization

● Virtualization– The ability to use a single physical computer to run several

“simulated” computers

● Why would we use it?– Testing / production environments– Cost reduction– Different testing scenarios

Virtualization types


8

Virtualization types

Performance is virtualization's Achilles Heel and depends mainly in the type of virtualization used.

Isolation between host and guests its also very important, usually: +isolation -performance→

● Emulation● Full virtualization● Paravirtualization● OS level virtualization


9

Virtualization types (I)

Emulation

● Entire hardware is simulated (no access to real hardware)● Guests are unmodified● Different architectures can be virtualized● Poor performance


10

Virtualization types (II)

Full virtualization

● Similar to emulation● Some instructions are run on the raw hardware (Intel VT or

AMD V is needed)● Host and guests have to be the same architecture● Guest OS runs unmodified● Often used instead of paravirtualization, when guest OS is

distributed in binary form (e.g. Windows)● Better performance than emulation


11

Virtualization types (III)

Paravirtualization

● Guest OS is modified so that it communicates whith the real hardware

● Applications run unmodified● Host and guests have to be the same architecture● Better performance than Full virtualization


12

Virtualization types (IV)

OS level virtualization

● Guests share resources with the host system● Weak isolation● Host performance can be compromised by guests● Guests are not really full-featured systems

Test-bench setup


14

Test-bench setup

Testing Equipment

● HP ProLiant ML310● Dual Intel(R) Xeon(R) CPU 3075 @ 2.66GHz● RAM: 2GiB DDR2 800MHz● 2 x 80GB HDD – RAID0● OS: Debian 5.0 (Lenny) 64-bit● Cheap 3Com Gigabit switch


15

Test-bench setup (I)


16

Test-bench setup (II)

● Software used:– Asterisk ;)– SIPp (stress-testing)– Tcpdump (sniffing RTP for use with SIPp)– sar (CPU usage, HD, IO stats)– sadf (extract sar's data)

● Limits:– ulimit -s unlimited # stack size– ulimit -n 65536 # open files– ulimit -u unlimited # user process num.

Testing


18

Testing

Send 2.000 calls with SIPp (100s RTP bidirectional media). 400 simultaneous calls maximum.

./sipp -sf uac_pcap.xml 10.10.0.241 -m 2000 -s 1234 -l 400 -mi 10.10.0.207 -mp 11000 -i 10.10.0.207 -trace_stat

./sipp -sf saghul_uas.xml -mi 10.10.0.205 -rtp_echo -mp 11000 -p 5070


19

Testing (II)

Analyze sar data and SIPp generated stats.

sar -c -n DEV -n SOCK -p -r -u -o stress.sar 10 0

sadf -d stress.sar -- -p -r -u -c -n DEV -n SOCK > stress.csv


20

Testing: Xen

● Total calls generated: 2000● Successlful calls: ~950● Max. Concurrent calls: 400


21

Testing: KVM



22

Testing: No virtualization



23

Call stats (Xen)

2009-04-2722:34:30:367 1240864470.3671862009-04-2722:37:30:372 1240864650.372034

2009-04-2722:40:30:377 1240864830.3770342009-04-2722:43:30:383 1240865010.383449

0

500

1000

1500

2000

2500

Call stats

TotalCallCreatedSuccessfulCall(C)FailedCall(C)


24

Call stats (KVM)

2009-04-2819:45:13:293 1240940713.2939462009-04-2819:47:13:296 1240940833.296806

2009-04-2819:49:13:299 1240940953.2998052009-04-2819:51:13:302 1240941073.302805

2009-04-2819:53:13:308 1240941193.308350

0

500

1000

1500

2000

2500

Call stats



25

Call stats (No virtualization)

2009-04-2722:56:26:043 1240865786.0432682009-04-2722:59:26:051 1240865966.051527

2009-04-2723:02:26:055 1240866146.0551272009-04-2723:05:26:506 1240866326.506768

0

500

1000

1500

2000

2500

Call stats



26

Failed calls (UDP retrans.)

1 2 3 4 5 6 7 8 9 10

0

200

400

600

800

1000

1200

Failed calls (UDP retrans.)

KVMXenNo-virt


27

CPU usage (Xen)

2009-04-27 20:34:37 UTC2009-04-27 20:35:57 UTC

2009-04-27 20:37:17 UTC2009-04-27 20:38:37 UTC

2009-04-27 20:39:57 UTC2009-04-27 20:41:17 UTC

2009-04-27 20:42:37 UTC2009-04-27 20:43:57 UTC

2009-04-27 20:45:17 UTC2009-04-27 20:46:37 UTC

2009-04-27 20:47:57 UTC2009-04-27 20:49:17 UTC

0

10

20

30

40

50

60

70

80

CPU usage

%user%system%iowait


28

CPU usage (KVM)

2009-04-28 17:45:29 UTC2009-04-28 17:46:29 UTC

2009-04-28 17:47:29 UTC2009-04-28 17:48:29 UTC

2009-04-28 17:49:29 UTC2009-04-28 17:50:29 UTC

2009-04-28 17:51:29 UTC2009-04-28 17:52:29 UTC

2009-04-28 17:53:29 UTC

0

10

20

30

40

50

60

70

80

90

CPU usage

%user%system%iowait


29

CPU usage (No virtualization)

2009-04-27 20:56:45 UTC2009-04-27 20:57:45 UTC

2009-04-27 20:58:45 UTC2009-04-27 20:59:45 UTC

2009-04-27 21:00:45 UTC2009-04-27 21:01:45 UTC

2009-04-27 21:02:45 UTC2009-04-27 21:03:45 UTC

2009-04-27 21:04:45 UTC2009-04-27 21:05:45 UTC

2009-04-27 21:06:45 UTC2009-04-27 21:07:45 UTC

0

5

10

15

20

25

30

35

40

45

50

CPU usage

%user%system%iowait


30

Memory usage (Xen)

2009-04-27 20:34:37 UTC2009-04-27 20:35:57 UTC

2009-04-27 20:37:17 UTC2009-04-27 20:38:37 UTC

2009-04-27 20:39:57 UTC2009-04-27 20:41:17 UTC

2009-04-27 20:42:37 UTC2009-04-27 20:43:57 UTC

2009-04-27 20:45:17 UTC2009-04-27 20:46:37 UTC

2009-04-27 20:47:57 UTC2009-04-27 20:49:17 UTC

0

2

4

6

8

10

12

14

16

18

20

Memory usage

%memused%swpused


31

Memory usage (KVM)

2009-04-28 17:45:29 UTC2009-04-28 17:46:19 UTC

2009-04-28 17:47:09 UTC2009-04-28 17:47:59 UTC

2009-04-28 17:48:49 UTC2009-04-28 17:49:39 UTC

2009-04-28 17:50:29 UTC2009-04-28 17:51:19 UTC

2009-04-28 17:52:09 UTC2009-04-28 17:52:59 UTC

2009-04-28 17:53:49 UTC

0

10

20

30

40

50

60

70

80

90

Memory usage

%memused%swpused


32

Memory usage (No virtualization)

2009-04-27 20:56:45 UTC2009-04-27 20:57:55 UTC

2009-04-27 20:59:05 UTC2009-04-27 21:00:15 UTC

2009-04-27 21:01:25 UTC2009-04-27 21:02:35 UTC

2009-04-27 21:03:45 UTC2009-04-27 21:04:55 UTC

2009-04-27 21:06:05 UTC2009-04-27 21:07:15 UTC

2009-04-27 21:08:25 UTC

0

1

2

3

4

5

6

7

8

Memory usage

%memused%swpused


33

Socket activity (Xen)

2009-04-27 20:34:37 UTC2009-04-27 20:35:47 UTC

2009-04-27 20:36:57 UTC2009-04-27 20:38:07 UTC

2009-04-27 20:39:17 UTC2009-04-27 20:40:27 UTC

2009-04-27 20:41:37 UTC2009-04-27 20:42:47 UTC

2009-04-27 20:43:57 UTC2009-04-27 20:45:07 UTC

2009-04-27 20:46:17 UTC2009-04-27 20:47:27 UTC

2009-04-27 20:48:37 UTC

0

200

400

600

800

1000

1200

1400

1600

1800

Socket activity

totsck


34

Socket activity (KVM)

2009-04-28 17:45:29 UTC2009-04-28 17:46:19 UTC

2009-04-28 17:47:09 UTC2009-04-28 17:47:59 UTC

2009-04-28 17:48:49 UTC2009-04-28 17:49:39 UTC

2009-04-28 17:50:29 UTC2009-04-28 17:51:19 UTC

2009-04-28 17:52:09 UTC2009-04-28 17:52:59 UTC

2009-04-28 17:53:49 UTC

0

500

1000

1500

2000

2500

Socket activity

totsck


35

Socket activity (No virtualization)

2009-04-27 20:56:45 UTC2009-04-27 20:57:45 UTC

2009-04-27 20:58:45 UTC2009-04-27 20:59:45 UTC

2009-04-27 21:00:45 UTC2009-04-27 21:01:45 UTC

2009-04-27 21:02:45 UTC2009-04-27 21:03:45 UTC

2009-04-27 21:04:45 UTC2009-04-27 21:05:45 UTC

2009-04-27 21:06:45 UTC2009-04-27 21:07:45 UTC

0

200

400

600

800

1000

1200

1400

1600

1800

Socket activity

totsck


36

Tweaking (Xen)

● Xen tweaking:– DomU in an LVM volume– 1GB of RAM for the DomU– Ethernet checksum disabled for the domU adapter:

ethtool -K eth0 tx off– System limits 'extended'– Had to 'tweak' dahdi_dummy.c so that it corretly handles

timing source.


37

Tweaking (KVM)

● KVM tweaking:– Guest in an LVM volume– 1GB of RAM for the guest– Paratirtualization drivers

● Virtio block device● Virtio network device

– System limits 'extended'

Results analysis


39

Results analysis

● Only a few calls are dropped because of UDP retransmissions

● 90% dropped calls exceeded response timeouts, etc of the SIPp scenario

– It's a very strict scenario!● Didn't see any “too many open files” log message ;)● CPU usage is higher in virtualized environments

– We didn't push Asterisk too hard– 100% CPU usage wasn't reached

● Networking tends to be slower when running virtual


40

Issues during testing period

● The server generating the traffic has to be really powerful

– It can algo get 'stressed'!

● SIPp crashed many times– Snapshot from 2009-01-21 seems to be most stable

● Network has to be 'isolated'– We don't want results to be distorted

Conclusions


42

Conclusions

● Yes, we CAN use Asterisk in a virtualized environment● Performance factors

– HDD– Network– CPU

● For 400-600 calls we can use virtualization– Remember we are SHARING resources!– For heavier loads a standalone server would be better– We can even use cards (pci passthrough)

● Specially for testing purpouses virtualization is a really good idea


43

Thanks!

BYE sip:[email protected] SIP/2.0

Via: SIP/2.0/UDP client.amoocon.de:5060;branch=z9hG4bKnashds7

Max-Forwards: 70

From: saghul <sip:[email protected]>;tag=8321234356

To: Amoocon <sip:[email protected]>;tag=9fxced76sl

Call-ID: [email protected]

CSeq: 1 BYE

Content-Length: 0

Thanks for watching!Thank you Olga and Manwe, for your continuous help

and support over long VoIP-hacking nights ;)

Any questions?


45

License

http://creativecommons.org/licenses/by-sa/3.0/

http://creativecommons.org/licenses/by-sa/3.0/

Business

Running Asterisk on virtualized environments