1

In today’s digital oilfield, big data is gath-ered from many points throughout explo-ration, discovery, and processing. This data

often is described by its volume, velocity, andvariety; all three attributes may be foundwithin the oil and gas industry. The industryincreasingly relies upon it for data analyticsused to drive strategic decisions. Such analy-sis demands massive parallel and acceleratedprocessing capabilities enabled by advancedhardware and software solutions.

IT organizations must be fully preparedto deal with big data in order to competesuccessfully. This report explores how Dellsolutions help meet evolving big data chal-lenges throughout the energy industry nowand into the future. Technological advances,including density-optimized servers withleading edge co-processors, accelerators,and open source software, can be used tosolve global challenges in E&P all the waythrough to distribution in a highly competi-tive international market.

The big data challenge for oiland gas The capture and analysis of big data neededto drive decisions can be challenging. Someof the challenges include:

Data utilizationWhen the research firm IDC Energy Insightsresearched the topic of big data in the oil andgas industry, 69.9 %[1] were unaware of theterm. A better understanding of big data’s usesand applications will help oil and gas industryleaders build a successful competitive strategy.

Intelligent modelingResource discovery and processing relies onbig data for many valuable functions. For ex-ample, it may be used to intelligently modeland image the Earth’s structure and layers1,524 m to 10,668 m (5,000 ft to 35,000 ft)below the earth's surface for exploration pur-poses, by computing seismic wave data. Bigdata also helps characterize activity aroundexisting wells, such as machinery perform-ance, oil flow rates, and well pressures. Withapproximately one million wells currently pro-ducing oil and gas in the US, this dataset isextensive, rapidly growing, and valuable formaximizing the returns from these reserves.

Precise computationInformed decisions are best made with care-fully selected data. Accurate data capture andpresentation of information related to energydiscovery and E&P accelerates sound deci-sions in the industry. Specifically, IT and theright High Performance Computing (HPC) ar-

chitecture enable big data processing, speedup the time-to-oil, and ultimately provide aclear competitive advantage[2].

Exploration speed and focusFaster and more accurate analysis of big datapromotes the speed of exploration and dis-covery of new energy resources without com-promising safety. This provides a morefocused path to exploration, discovery, andproduction with lowest environmental impact. Through the utility of remote-sensor-ing devices and monitoring technology, infor-mation may be captured by personnel andequipment to drive better business decisions.

While the processing of big data hasmade great strides in the past few years, advances in the specific tools and architec-ture needed to process big data are expectedin the years ahead. IDC Energy Insights ex-pects the big data technology and services

September 2013

Big Data and the Digital Oilfield: Using Advanced Technologyto Gain a Competitive Edge The age of big data is upon us.

Contributed by Dell

1

IT organizations must be fully prepared to deal

with big data in order to compete successfully. IDC Energy Insights

expects the big data technology and servicesmarket to grow to US $16.9 billion by 2015.

market to grow to US $16.9 billion by 2015,which would represent a compound annualgrowth rate of 39.4% or about seven timesthat of the overall information and commu-nication technology market. [3]Keeping upwith technology advancements, includingthe storing and processing of big data, willcontinue to be a necessary ingredient in stay-ing competitive[4].

Enabling big data means enablingtime-to-oil By building a network infrastructure withthe right servers and processors, the bene-fit of big data can be realized and will expedite energy resource discovery and E&Pin many ways, including:

• Throughput: Maintain high datathroughput to get results faster;

• Scalability: Accommodate the com-plexities of seismic imaging and allowthe use of rapidly growing data sets indata acquisition, processing, or inter-pretation phases;

• Consolidation: Enable a common set of

management tools to consolidatepetabytes of data onto a single platform;

• Supportability: Provide standard proto-cols to accommodate applications run-ning on heterogeneous systems whenaccessing the same data set; and

• Speed: Shorten and reduce the discov-ery process by days and even weeksthrough rapid analysis of data by thepetabytes.

HPC and the big data environmentMany oil and gas workloads take advantageof massive parallel-processing technology tospeed up the time required to completecomplicated analyses. Leading-edge compo-nent technology such as Intel’s Xeon Phi co-processor and NVIDIA’s GPU acceleratorsprovide faster performance and operate eco-nomically and efficiently with resources suchas open source software.

Co-processors supplement the primaryprocessors by offloading processor-inten-sive tasks from the main processor to ac-celerate system performance. Operations

performed by the co-processor may includefloating point arithmetic, graphics, signalprocessing, string processing, encryption, orI/O interfacing with peripheral devices. Thisenables faster processing of data character-ized by high velocity and volume, such asreal-time streaming data from drill headsand equipment sensors.

Graphics processing units have highlyparallel architectures that make them moreeffective than general-purpose CPUs for algorithms where processing of largeblocks of data is done in parallel. They areefficient at rendering the data required byoil geophysical surveying, including seismicgeological modeling, improving resolution,and processing seismic data in complexsurfaces or low signal/noise ratios. High-density optimized infrastructure providesthe required performance and scalabilityneeded to keep up with these demandingworkloads. Servers, networking, and stor-age are purpose-built and tuned for maxi-mum performance and efficiency in thesedata-intensive environments. �

Case study one: Enabling big data volume[5]

Background: Network infrastructure is a criti-cal capability for accommodating big datavolume.

The data volumes for seismic image pro-cessing are very large. Single files can rangefrom 500 gigabytes to several terabytes (TB)in size. Projects may range from 4 TB to 24TB. The movement and storage of such datacreates congestion or gridlock that can causegeoscientists and seismic engineers to waitweeks or even months to access the results.

Problem: A company that processesgeophysical data for companies engaged inoil and gas exploration faced a challenge intheir network infrastructure to make a highvolume of big data available to their clients.The company routinely processes massiveamounts of seismic data so their clients candetermine where to drill the next well.

Infrastructure bandwidth is core to thiscompany’s offering. Network switching ca-pability is key to moving data quicklyenough for end-users to rapidly processseismic data for decision-making purposes.

However, the firm’s legacy switches did notprovide sufficient bandwidth to move datafast enough.

Solution: With two different data centersin disparate geographic locations, the com-pany used one Dell Force10 C300 resilientswitch at each location. With up to 384 line-rate 10/100/1000Base-T ports, coupledwith five microsecond switching latencyunder full load for 64-byte frames, the chas-sis-based switches provided the port den-sity and throughput needed to ensure thedata processing performance demanded bytheir client's end-users. With nearly 1,000nodes at one data center, and more than250 nodes at the other, the company wasable to leverage the switch’s backplane capacity of more than 1.5 terabits and morethan 950 megabits per second of L2/L3packet forwarding capacity.

Result: The company became more agileand was able to offer its current and poten-tial clients the capability to handle explo-ration data. They were able to reduce theircost-per-port and achieve a robust end-to-end line speed with an attractive total cost of

ownership. In addition, the company hadplanned for a 10GbE backbone in the future.The new Dell Force 10 enabled such a mi-gration seamlessly and averted any addi-tional challenges to support it.

Case study two: A three-fold increase in time-to-oil[6]

Background: Seismic data demands the righttools and the right capability to handle thehuge volume and velocity of data generatedby research.

Problem: A company providing seismicimaging and analysis service had a key business opportunity to partner with an off-shore vessel operator and collect seismicoffshore data. The company’s client soughtto improve its time-to-oil.

With this growing demand, the companyneeded to shift its analysis of seismic datato an offshore research vessel. This allowedthe company to process data quickly whileonboard without waiting until the vessel returned to port. Its clients could then havethe information sooner, providing them asignificant competitive advantage.

2

Advancing HPC in the digital oilfield: four case studies

3

The company’s proprietary software required a high performance hardware plat-form to operate, which was beyond the capabilities of the ship’s existing infrastruc-ture. The server capacity needed to conductanalyses and operations was limited by thephysical space, power, and available cool-ing aboard the vessel.

Solution: To maximize the onboard sys-tem’s density and power efficiency, thecompany chose Dell M1000 blade serverswith Intel Xeon Processor 5400 series. Thisprovided eight cores per blade and enabledthem to run their software while minimiz-ing space requirements. The Intel Xeonprocessors are designed to be energy effi-cient and enabled the vessel’s engineers tosupport the system without disrupting otheroperations.

Result: The combined Intel Xeon quad-core processors were more than twice asfast as the legacy onboard dual-coreprocessors, allowing them to handle theCPU-intensive demand easily. In addition,the design of the Intel Xeon processor 5400series incorporates a low-voltage powersupply but provides an increased process-ing capability for facility with a reducedpower capacity.

A significant outcome of the platformwas an accelerated project timeline. Thenew hardware configuration reduced thetotal project time from nine months downto six. This not only reduced cost for theirclients, but also provided these clients a substantial head start on developing en-ergy sources.

Case study three: Double theprocess power while reducing cost[7]

Background: In geophysical surveying usedfor oil and gas exploration, complex, com-puter-generated subsurface maps can depictgeological surfaces in 3-D with detailed con-tours and cross-sections. These visualizationsrequire increasingly complex and sophisti-cated algorithms to generate accurate resultsthat can be used to assess the developmentpotential of claims.

Problem: China’s leading engineeringand technical services company BGP spe-cializes in geophysical exploration and istop-ranked globally for land-based seismicexploration in its quest to identify energysources. This includes discovery of 260 oiland gas fields of different sizes, more than

80 billion tons of geological oil reserves,and 2 Tcm of gas reserves.

The company employs an advanced seis-mic data analysis platform with many appli-cations including seismic geologicalmodeling, pre-stack migration, 3-D visuali-zation, and advanced resolution for pro-cessing complex seismic data. Theirarchitecture and software enable sophisti-cated model processing and data interpre-tation. This platform provides more than300 batch and interactive processing seis-mic application modules in 20 categories.In addition, it has the capability to supportonshore and offshore 2-D and 3-D seismicdata processing and advanced 3-D seismicvisualization.

The company’s previous UNIX-basedprocessing system struggled to provide theneeded performance, cost, and hardwareexpandability and software compatibilitywith new and advanced applications. Itneeded the very best configuration to meetthe domestic demand for oil and gas explo-ration in China and to compete in the inter-national marketplace.

Solution: To meet its next-generation pro-cessing requirements, the company movedto an x86-based platform that implementedthe Intel Xeon E7 processor series. TheXeon-based servers delivered strong pro-cessing performance, supporting 80 coresand 160 threads, and memory capacity ashigh as 4 TB, a top choice for geologicaldata analysis systems.

Result: The new architecture cost 46%less than the existing Unix system and in-creased process time by as much as 49%.The company was able to double its pro-cessing capability, reduce its cost to achievethis, and create a technical capability to bemuch more competitive in an increasinglydemanding and complex exploration envi-ronment. (Intel Inc., 2012)

Case study four: A hundred-fold increase in HPC performance[8]

Background: Complex seismic algorithmsand acquisition techniques have acceler-ated the demand for HPC capacity and per-formance that can meet the demands of bigdata. The challenge is to strategically maxi-mize the value of HPC to deliver the mostbusiness value.

Problem: A multinational oil company relies heavily on HPC for the success of its

operations and technical functions. Thecompany set a long-term goal to achieve ahundred-fold improvement in technologyperformance, meaning if the technology de-livers a hundred-fold gain in performance,the cost must also be less than 100 timesthe expense.

Solution: The company formed and de-veloped an HPC Center of Excellence fo-cused on collaboration with internal IT andoperations teams and the suppliers and ven-dors who help to serve the company’s HPCneeds. This was a strategy to find and buildsolutions to their needs of applications, data,storage, network capability, and visualizationtechnology to deliver the business value andmeet market demands into the future. Moreaccurate seismic imaging, data center con-solidation, customized engineering models,software optimization, and the deploymentof advanced computing hardware are allpart of the collaborative efforts put in placeto reach this goal. All of these delivered sub-stantial performance improvements thathave put the company well on its way toachieving its hundred-fold goal.

Result: The company’s custom-devel-oped reservoir engineering model tripled itsHPC computing performance, with work-flows achieving significantly higher increases.Its software optimization improved by asmuch as twelve-fold and its HPC capacitydoubled. In addition, internal teams have de-veloped and optimized higher performingversions of critical software applications, pro-ducing sophisticated and cutting-edge algo-rithms. In the company’s Gulf of Mexico(GoM) exploration, its optimized processesfor data storage and processing enabled thecompany to make its projects larger anddenser. This reduced the cost by a multiple oftwo-and-a-half times what it was 10 yearsearlier. (Intel, Inc., 2012)

Open-source software provides flexible options to manage big dataThe growing availability of high-quality opensource software enables developers to usecode that can be tailored to fit specific com-puting needs without incurring the expenseof proprietary licensing costs. Open-sourcesoftware is flexible in its use and can be mod-ified to work with homegrown applicationsand commercial applications. The oil and gasindustry is using open source applications,particularly those used for data analysis.

4

Hadoop is one example of an open-source application that is gaining popularityas a cost-effective tool to process and storelarge data sets across commodity server clus-ters. According to analysts at TechNavio research, the global Hadoop market is set togrow 55.63% from 2012 to 2016. Accordingto the research, the main driver of this marketis the growing demand for big data analytics.[9]Many oil and gas companies have usedHadoop as part of their HPC operations.

Case study five: Chevron uses open-source software for energy exploration Problem: Chevron’s oil exploration effortsare comprehensive and include the move-ment of drillships into areas such as the Gulfof Mexico to survey targeted areas where oilor gas deposits may be located. A researchship can cost the company nearly $1 millionper day to be on task. The technology chal-lenge is to capture and process the highestquality of data possible while controlling thecost of exploration.

The drillship crew sends seismic wavesto the ocean floor and measures the re-flected waves as they bounce back to theship’s array of receivers. The array measuresthe amplitude and arrival times of the re-flected waves to build a time series that canthen be analyzed to create a simulatedgraphic model of the ocean floor that indi-cates potential locations of energy re-sources. The amount of data generated viathe seismic waves is overwhelming.Chevron collects and analyzes seismic in-formation that contains five dimensions.Processing this level of complex data andproducing an accurate visual simulation re-quires a supercomputer with extreme com-puting horsepower.

Solution: Chevron takes 25 steps in pro-cessing the seismic data to create a visualmodel for engineers to locate potential oilreservoirs. Hadoop is used to sort the dataand create models and simulations of theunderground environment.

The company claims that Hadoop costsone-tenth of the commercial software solu-

tion previously used. The use of Hadoop reduced the total cost of exploration, andthe company was able to achieve its objec-tives seamlessly with its use. Open-sourcesoftware was a key advantage for Chevronand is an option for other scientists and developers in the industry.

The oil and gas industry works within anenvironment increasingly populated by bigdata. The digital oilfield will increasinglyrely on efficient processing of such data, allowing firms to stay competitive by improving time-to-oil. The challenge ofsuccessfully processing big data for discov-ery, exploration, and development pur-poses is formidable and demandsprecise-fit, high-performance computingsolutions. The careful combination of theseadvanced technology solutions, which includes hardware infrastructure based ona new generation of density-optimizedservers, accelerators, and co-processors, aswell as open-source software, will allow oiland gas firms to best compete and stream-line time-to-oil.[10] �

For more than 26 years, Dell has empowered countries, com-munities, customers, and people everywhere to use tech-nology to realize their dreams. Customers trust Dell to deliver

technology solutions that help them do and achieve more,whether they're at home, work, school, or anywhere in the world.

Seismic processing, simulations, and other data-intensivecomputing require exceptional server performance, bandwidth,and efficiency. To support the massive volume of computationsrequired, Dell PowerEdge servers incorporate the latest IntelXeon series processors. With expanded memory bandwidth andchannels and integrated I/O to help reduce latency by up to30%, these processors deliver more than 80% more perform-ance than previous-generation Intel series processors.

The PowerEdge C-Series servers offer a streamlined approachfor targeted hyperscale environments. For these servers, Dell hasremoved the redundant hardware, broad OS support, and same-day parts replacement that these organizations do not need,

helping provide the requisite performance levels in dense, energy-efficient configurations. These servers also allow organi-zations to gain fast access to emerging technology instead ofwaiting for customized solutions or traditional general-purposeservers (with their additional features and extensive OS support).

IT managers can gain further cost efficiencies by moving tocomputing platforms based on open source applications, sharedinfrastructures, and today’s most advanced processors, networkconnectors, and serviceability.

Dell’s HPC and big data solutions are supported by Dell’sbroad portfolio of planning, implementation, and maintenanceservices. Those professional services can help oil and gas firmsaccelerate IT initiatives, manage the complexities of an HPC environment, and accelerate time-to-oil in their vital explorationactivities. Dell services can be tailored to meet specific require-ments and can include data center consulting, custom rack integration, online solution support, and other solutions.

About Dell’s advanced technology solutions for HPC environments

To learn more, contact Dell’s Data Center Solutions and/or High Performance Computing specialists at Dell. www.dellhpcsolutions.com • www.dell.com/poweredgec

References:[1] Rick Nicholson, Big Data in the Oil and Gas Industry, IDC Energy Insights, (https://www-950.ibm.com/events/wwe/grp/grp037.nsf/vLookupPDFs/RICK%20-%20IDC_Calgary_Big_Data_Oil_and-

Gas/$file/RICK%20-%20IDC_Calgary_Big_Data_Oil_and-Gas.pdf).[2] Adam Farris, “How Big Data is Changing the Oil & Gas Industry,” Analytics Magazine (http://www.analytics-magazine.org/november-december-2011/695-how-big-data-is-changing-the-oil-a-gas-industry, (Nov-

Dec 2011). [3] Jill Feblowitz, The Big Deal About Big Data in Upstream Oil and Gas, (http://www.hds.com/assets/pdf/the-big-deal-about-big-data-in-upstream-oil-and-gas.pdf).[4] Steven Croce, Brandon Draeger, and Buck Avey, “Designing for Hyperscale Computing,” http://www.dell.com/downloads/global/power/ps2q10-20100360-cloud.pdf[5] Dell Customer Stories: Tricon Geophysics, Inc., http://www.dell.com/Learn/us/en/uscorp1/corporate~case-studies~en/Documents~2011-tricon-10010209.pdf[6] Oceangoing Multicore, http://www.geotrace.com/news/pdf/Intel-ESS-GeotraceCaseStudyLR-Web.pdf[7] Cutting Edge Tool For Exploring the Earth, http://www.geotrace.com/news/pdf/Intel-ESS-GeotraceCaseStudyLR-Web.pdf[8] Shell Drills Down In Improvements in High Performance Computing, http://www.intel.com/content/www/us/en/high-performance-computing/high-performance-xeon-shell-paper.html?wapkw=oil+and+gas[9] Big Data Goes Hadoop, March 21, 2013, (http://www.giiresearch.com/press/7737.shtml).[10] Rachel King, Chevron Explores Open Source Using Hadoop, CIO Journal, June 5, 2012 (http://mobile.blogs.wsj.com/cio/2012/06/05/chevron-explores-open-source-using-hadoop/).