6
9/22/2015 Random vs. Sequential explained. | Open-E Blog http://blog.open-e.com/random-vs-sequential-explained/ 1/6 Test Pattern Transfer Request Size Percent Sequential Percent Random Percent Write Percent Read MB / second IO / second 1 2KB 100 0 100 0 24,75 12674 2 2KB 100 0 0 100 52,17 26708 3 2KB 0 100 33 67 0,75 383 4 64KB 100 0 0 100 103,82 1661 5 64KB 100 0 100 0 94,06 1504 6 256KB 100 0 0 100 141,68 566 7 256KB 100 0 100 0 100,65 402 Random vs. Sequential explained. (http://blog.open-e.com/random-vs- sequential-explained/) Very often, storage product users do not know what kind of performance numbers to expect in real operations. Why do storage vendors claim such great performance when measured by an industry accepted reference tool like IOmeter, but in production database applications are often very slow? The biggest difference comes from how the data is being read from the disk. Is it in sequential blocks or is it scattered in random blocks all over the drive? How big of a difference does a sequential data pattern vs. a random data pattern make? This may be obvious for experienced users but we receive a lot of questions about it; so a short and easy to follow example is in order. To show examples of best and worst performance results with IOmeter, we can use a local RAID5 controller with 3 hard disks. When running RAID5 with only 3 hard disks we do not expect to see great performance numbers, but it will be enough to show the difference elicited by using different test patterns. When you the read the marketing material about this particular server hardware you will find the claim that it provides over 25,000 I/O per second and over 140MB/second. These are very nice figures. (http://www.facebook.com/sharer/sharer.php? u=http%3A%2F%2Fblog.open- e.com%2Frandom- vs- sequential- explained%2F&display=popup) (https://twitter.com/intent/tweet? text=Random+vs.+Sequential+explained.&url=http%3A%2F%2Fblog.open- e.com%2Frandom- vs- sequential- explained%2F) (https://plus.google.com/share? url=http%3A%2F%2Fblog.open- e.com%2Frandom- vs- sequential- explained%2F&t=Random+vs.+Sequential+explained.) (http://www.linkedin.com/shareArticle? mini=true&url=http%3A%2F%2Fblog.open- e.com%2Frandom- vs- sequential- explained%2F&title=Random+vs.+Sequential+explained.&ro=false&summary=Very+ (http://blog.open-e.com)

Random vs. Sequential Explained

Embed Size (px)

DESCRIPTION

algorithms

Citation preview

Page 1: Random vs. Sequential Explained

9/22/2015 Random vs. Sequential explained. | Open-E Blog

http://blog.open-e.com/random-vs-sequential-explained/ 1/6

Test

Pattern

Transfer

Request Size

Percent

Sequential

Percent

Random

Percent

Write

Percent

Read

MB /

second

IO /

second

1 2KB 100 0 100 0 24,75 12674

2 2KB 100 0 0 100 52,17 26708

3 2KB 0 100 33 67 0,75 383

4 64KB 100 0 0 100 103,82 1661

5 64KB 100 0 100 0 94,06 1504

6 256KB 100 0 0 100 141,68 566

7 256KB 100 0 100 0 100,65 402

Random vs. Sequential explained.(http://blog.open-e.com/random-vs-sequential-explained/)Very often, storage product users do not know what kind of performance numbers to expect in real

operations. Why do storage vendors claim such great performance when measured by an industry

accepted reference tool like IOmeter, but in production database applications are often very slow?

The biggest difference comes from how the data is being read from the disk. Is it in sequential blocks

or is it scattered in random blocks all over the drive? How big of a difference does a sequential data

pattern vs. a random data pattern make? This may be obvious for experienced users but we receive a

lot of questions about it; so a short and easy to follow example is in order.

To show examples of best and worst performance results with IOmeter, we can use a local RAID5

controller with 3 hard disks. When running RAID5 with only 3 hard disks we do not expect to see great

performance numbers, but it will be enough to show the difference elicited by using different test

patterns.

 

 

When you the read the marketing material about this particular server hardware you will find the

claim that it provides over 25,000 I/O per second and over 140MB/second. These are very nice figures.

(http://www.facebook.com/sharer/sharer.php?

u=http%3A%2F%2Fblog.open-

e.com%2Frandom-

vs-

sequential-

explained%2F&display=popup)

(https://twitter.com/intent/tweet?

text=Random+vs.+Sequential+explained.&url=http%3A%2F%2Fblog.open-

e.com%2Frandom-

vs-

sequential-

explained%2F)

(https://plus.google.com/share?

url=http%3A%2F%2Fblog.open-

e.com%2Frandom-

vs-

sequential-

explained%2F&t=Random+vs.+Sequential+explained.)

(http://www.linkedin.com/shareArticle?

mini=true&url=http%3A%2F%2Fblog.open-

e.com%2Frandom-

vs-

sequential-

explained%2F&title=Random+vs.+Sequential+explained.&ro=false&summary=Very+often%2C+storage+product+users+do+not+know+what+kind+of+performance+numbers+to+expect+in+real+operations.+Why+do+storage+vendors+claim+such+great+performance+when+measured+by+an%26nbsp%3B...&source=)

(http://blog.open-e.com)

Page 2: Random vs. Sequential Explained

9/22/2015 Random vs. Sequential explained. | Open-E Blog

http://blog.open-e.com/random-vs-sequential-explained/ 2/6

IO / second:best / worst26708/ 383=70

MB / second: best / worst141,68 / 0,75 =189

IO / sec MB / sec

SSD powered iSCSI 10392 20.30

Local RAID5 383 0.75

claim that it provides over 25,000 I/O per second and over 140MB/second. These are very nice figures.

But imagine you are going to index a database (similar to test case #3). Your server will actually

perform at about 400 I/O per second and below 1 MB/second. Test case #3 uses the default IOmeter

test pattern: 100% Random, 2kB Block Size, 33% writes and 67% reads. This test pattern shows very

poor results.

Believe me, I have seen plenty of very disappointed users running the default test pattern on IOmeter.

I recall an incident at the CEBIT show few years ago when a RAID controller vendor engineer was

preparing their demo and noticed that one IOmeter ‘worker’ was running the default pattern; instead

of the desired 400MB/sec., it was showing only about 4MB/sec. This is 100 times slower!

Let’s check the ratios between the best and worst results in our test case above.

So in the worst case we see a result 189 times slower than the marketing material claims.

What is the problem? Why does the default IOmeter test pattern show such poor results?

The main reason comes from the behavior of hard disk drives. The time to retrieve data with a

random seek is much longer than with a sector to sector seek. Also the block size has huge influence

on the performance results. In the worst-case combination of big/small block size and

sequential/random seek results vary by over a factor of 100.

How can we improve the performance of an application?

First the application needs to be designed to move bigger blocks of data a time. Second you need

disks with better average access time. SATA disks mostly spin with 7200rpm. So 10,000 rpm or 15,000

rpm disks will help significantly with a random pattern.

But recently more and more users are deploying SSD’s to improve performance with a random access

pattern. We have tested 200GB Anobit SSD. (www.anobit.com (http://www.anobit.com/)) The SSD was

installed in a DSS V6 server and configured as an iSCSI target. The storage client was Windows 2008

with iSCSI Software Initiator.

Running the default test pattern on 2 workers in IOmeter we obtained the following results:

So it looks like the single SSD powered iSCSI target was 27 times faster than the local Windows 2008

RAID5 while running the IOmeter default test pattern. The IOmeter default pattern is a good test if we

are running an application such as database indexing. The SSD powered iSCSI target was 27 times

better because the SSD drive has extremely good IOPS compared to SATA hard disks. But if you run

any sequential pattern the results are comparable. Here are IOmeter screenshots from the default

test pattern on SSD powered iSCSI Target vs. local hard disks powered with RAID5.

(http://blog.open-e.com/wp-

content/uploads/2011/04/IOmeter-default_test_pattern_local_RAID.png?871a4d)

(http://blog.open-e.com/wp-

content/uploads/2011/04/IOmeter-default_test_pattern_iSCSI_SSD.png?871a4d)

Page 3: Random vs. Sequential Explained

9/22/2015 Random vs. Sequential explained. | Open-E Blog

http://blog.open-e.com/random-vs-sequential-explained/ 3/6

Rating: 4.8/5 (4 votes cast)

DSS V6 (http://blog.open-e.com/tag/dss-v6/) IOmeter (http://blog.open-e.com/tag/iometer/)

random (http://blog.open-e.com/tag/random/) sequential (http://blog.open-e.com/tag/sequential/)

SSD (http://blog.open-e.com/tag/ssd/)

Related Project

content/uploads/2011/04/IOmeter-default_test_pattern_iSCSI_SSD.png?871a4d)

These are some very interesting results. Not what you would necessarily expect.

So as you can see many factors influence overall storage performance. And understanding how

performance numbers are produced can give big insight into their validity. Careful use of

performance measurement tools can give you useful results. It is always a good idea to understand

how your storage clients will access their data on your servers.

Leave a Reply

Security code (please type it as shown above)*

Why a HOT-SPARE Hard Diskis a bad idea?(http://blog.open-e.com/why-a-hot-spare-hard-disk-is-a-bad-idea/)

Case Study: Virtualizationand High Availability indrinking water managementwith Open-E DSS V7(http://blog.open-e.com/case-study-virtualization-and-high-availability-in-drinking-water-management-with-open-e-dss-v7/)

Alternative possibilities – thesmall list of big ideas(http://blog.open-e.com/alternative-possibilities-the-small-list-of-big-ideas/)

NAME

EMAIL

URL

MESSAGE

Page 4: Random vs. Sequential Explained

9/22/2015 Random vs. Sequential explained. | Open-E Blog

http://blog.open-e.com/random-vs-sequential-explained/ 4/6

Post Comment

Search

Software-definedstorage: reliability,performance andscalability.

Check it out!

(http://www.open-

e.com/products/jovian-

data-storage-

software/)

FROM BLOG

POST CATEGORIES

Popular Comments

Case Study: POLY-TOOLS implements iSCSI cluster with Open-E DS…

17/Sep/2015

The main goal of POLY-TOOLS bennewart GmbH, a leading...

New update: Open-E JovianDSS High Availability cluster for NFS an…

10/Sep/2015

Today, after a couple of very busy months, we...

How to configure a switchless setup with Open-E DSS V7 Active-Act…

05/Aug/2015

Our QA team recently prepared a new step-by-step guide...

Events (http://blog.open-e.com/category/events/) (14)

Open-E Software (http://blog.open-e.com/category/open-e-software/) (59)

Storage Technology (http://blog.open-e.com/category/storage-technology/) (77)

Virtualization (http://blog.open-e.com/category/virtualization/) (18)

Webinars (http://blog.open-e.com/category/webinars/) (12)

Page 5: Random vs. Sequential Explained

9/22/2015 Random vs. Sequential explained. | Open-E Blog

http://blog.open-e.com/random-vs-sequential-explained/ 5/6

Open-E is a pioneering leader and developer of IP-based data storage software. Our software turnsany server into a SAN or NAS appliance. The products have the ability to support a variety oftechnologies - like cloud storage - and protocols, such as iSCSI, Fibre Channel or Infiniband and offersautomatic failover for high availability and business continuity. With over 24,000 installations, in over100 countries, Open-E Data Storage solutions are used by Fortune 500 customers world-wide.

About Open-E

TAGS

ACTIVE-ACTIVE FAILOVER FOR ISCSI (HTTP://BLOG.OPEN-E.COM/TAG/ACTIVE-ACTIVE-FAILOVER-FOR-ISCSI/) BACKUP(HTTP://BLOG.OPEN-E.COM/TAG/BACKUP/) BONDING (HTTP://BLOG.OPEN-

E.COM/TAG/BONDING/) CASE STUDY (HTTP://BLOG.OPEN-E.COM/TAG/CASE-STUDY/) CEBIT

(HTTP://BLOG.OPEN-E.COM/TAG/CEBIT/) CLOUD (HTTP://BLOG.OPEN-E.COM/TAG/CLOUD/)

CONSISTENT (HTTP://BLOG.OPEN-E.COM/TAG/CONSISTENT/) DATA REPLICATION (HTTP://BLOG.OPEN-

E.COM/TAG/DATA-REPLICATION/) DATA SECURITY (HTTP://BLOG.OPEN-

E.COM/TAG/DATA-SECURITY/) DATA STORAGE SOFTWARE (HTTP://BLOG.OPEN-E.COM/TAG/DATA-

STORAGE-SOFTWARE/) DSS V6 (HTTP://BLOG.OPEN-E.COM/TAG/DSS-V6/)FAILOVER (HTTP://BLOG.OPEN-E.COM/TAG/FAILOVER/) FC (HTTP://BLOG.OPEN-E.COM/TAG/FC/)

HARD DISK (HTTP://BLOG.OPEN-E.COM/TAG/HARD-DISK/) HARDWARE(HTTP://BLOG.OPEN-E.COM/TAG/HARDWARE-2/) HBA (HTTP://BLOG.OPEN-

E.COM/TAG/HBA/) HDD (HTTP://BLOG.OPEN-E.COM/TAG/HDD/) HIGH AVAILABILITY CLUSTER

(HTTP://BLOG.OPEN-E.COM/TAG/HIGH-AVAILABILITY-CLUSTER/) IOMETER

(HTTP://BLOG.OPEN-E.COM/TAG/IOMETER/) ISCSI (HTTP://BLOG.OPEN-

E.COM/TAG/ISCSI/) ISCSI FAILOVER (HTTP://BLOG.OPEN-E.COM/TAG/ISCSI-FAILOVER/)

LINUX (HTTP://BLOG.OPEN-E.COM/TAG/LINUX/) MULTIPATH (HTTP://BLOG.OPEN-

E.COM/TAG/MULTIPATH/) MULTIPATH IO (HTTP://BLOG.OPEN-E.COM/TAG/MPIO/) NAS

(HTTP://BLOG.OPEN-E.COM/TAG/NAS/) OPEN-E (HTTP://BLOG.OPEN-E.COM/TAG/OPEN-E/) OPEN-E DSS V6 (HTTP://BLOG.OPEN-E.COM/TAG/OPEN-E-

DSS-V6/) OPEN-E DSS V7 (HTTP://BLOG.OPEN-E.COM/TAG/OPEN-E-DSS-V7/) OPEN-E JOVIANDSS

(HTTP://BLOG.OPEN-E.COM/TAG/OPEN-E-JOVIANDSS/) OPEN-E PRODUCTS

(HTTP://BLOG.OPEN-E.COM/TAG/OPEN-E-PRODUCTS/) PING NODE (HTTP://BLOG.OPEN-E.COM/TAG/PING-NODE/)

RAID (HTTP://BLOG.OPEN-E.COM/TAG/RAID/) RAID5

(HTTP://BLOG.OPEN-E.COM/TAG/RAID5/) REBUILD (HTTP://BLOG.OPEN-E.COM/TAG/REBUILD/) SNAPSHOT

(HTTP://BLOG.OPEN-E.COM/TAG/SNAPSHOT/) SSD (HTTP://BLOG.OPEN-

E.COM/TAG/SSD/) SSD CACHE (HTTP://BLOG.OPEN-E.COM/TAG/SSD-CACHE/) STORAGE(HTTP://BLOG.OPEN-E.COM/TAG/STORAGE/) TAPE DRIVE (HTTP://BLOG.OPEN-

E.COM/TAG/TAPE-DRIVE/) VIRTUAL MACHINE (HTTP://BLOG.OPEN-E.COM/TAG/VIRTUAL-MACHINE/)

VMWARE (HTTP://BLOG.OPEN-E.COM/TAG/VMWARE/) VMWARE ESX

(HTTP://BLOG.OPEN-E.COM/TAG/VMWARE-ESX/) VMWARE READY (HTTP://BLOG.OPEN-

E.COM/TAG/VMWARE-READY/) WRITE BACK (HTTP://BLOG.OPEN-E.COM/TAG/WRITE-BACK/) WRITE THROUGH

(HTTP://BLOG.OPEN-E.COM/TAG/WRITE-THROUGH/)

Page 6: Random vs. Sequential Explained

9/22/2015 Random vs. Sequential explained. | Open-E Blog

http://blog.open-e.com/random-vs-sequential-explained/ 6/6

Copyright 2015 Open-E. All rights reserved.

September 17, 2015

The main goal of POLY-TOOLS bennewart GmbH, a leading...

September 10, 2015

Today, after a couple of very busy months, we...

Recent Articles

Case Study: POLY-TOOLS implements iSCSI cluster with Open-E DSS V7 (http://blog.o…

New update: Open-E JovianDSS High Availability cluster for NFS and iSCSI (http://blo…

Janusz Bąk

Chief Technical Officer

Kasia Kołodziej

Technical Content Specialist

Vera Neumeyer

PR & Marketing Manager

Author Profiles