Biryaltsev E.V., Galimov M.R., Demidov D.E., Elizarov A.M.

HPC CLUSTER DEVELOPMENT AND OPERATION EXPERIENCE FOR SOLVING THE

INVERSE PROBLEMS OF SEISMIC PROSPECTING

HPC - INDUSTRIAL DEPLOYMENT2

MASS MARKET READY

OIL & GAS INDUSTRY AND TECHNOLOGY REPLACEMENT3

NEW SOLUTIONS REQUIRED

PRICE DROP

PRODUCTION & EXPLORATION COMPLEXITY

NUMERICAL SIMULATION4

GRID SIMULATION METHODS

FULL WAVE INVERSION5

LOW-FREQUENCY SEISMIC SOUNDING (LFS) 6

APPLICATIONS 7

1. Oil & gas prospecting

2. Fracturing control

3. Borehole environment monitoring

REQUIREMENTS8

ACCESSIBILITY

BIG DATA ARRAYS

PERFORMANCE

VISUALIZATION

30 000 models per cycle

109 – 1012 cells

~1 Tb per model

RESEARCH & DEVELOPMENT9

1. The best performance for the massive parallelism tasks.2. High computation density per equipment units.3. Perspective technology.

GPU — EFFICIENT CHOICE

GPU CLUSTER10

UPS Eaton 9155 Power Rating 8000 / 7200 (VA/Watts)0.7 Power factor

Server>80 Tb Data storage space + SSD cache

Virtualization serverCitirix XenServer platform2 x NVIDIA GK104GL with NVIDIA GRID K2 technology

Mellanox InfiniBand network switch 56 Gbit/s

6 GPU cluster nodes

4 x AMD FirePro W9100 (16Gb GDDR5, 5.24 Tflops peak)

2 x Intel Xeon E5-2643 v2 (25M cache, 3.50 GHz, 12 threads)

Performance35,6/64,9 Tflops

38th position in

Russian top 50 supercomputers rating

RAM –

320 GbOverall disk space

volumes

86 Тb

HARDWARE & SOFTWARE ENVIRONMENT11

VDI

GPU

3D

1. GPU clusters ОC Gentoo & Open Grid Engine

2. VexCL (vector expression template library) OpenCL/CUDA

3. Simulation & analyze software systemС++, MPI, Java

4. VDI serverCitix XenDesktop & NVidia GRID

5. Data warehousePostgreSQL & NFS (RAID)

WORKFLOW12

Sensors data

Simulation data

Processing

3D full wave simulation

GPU cluster

Interpretation

SUCCESSFUL SOLUTIONS13

Cloud solutionAmazon

SOFTWARE – HPC ENVIRONMENTOpenCL, CUDA, OpenMP, MPI

HARDWARE – GPU CLUSTER

Local solutionT-platforms, Meijin, ARBYTE SC, STSS

PROBLEMS14

DISTRIBUTED RENDERING

REMOTE & SHARED ACCESS

BIG DATA MANAGMENT

INTEGRATION

HPC 2D&3D

RD Store

INTEGRATED FRAMEWORKS 15

SCHLUMBERGER OCEAN

HUE SPACE

NVIDIA INDEX

COMMERCIAL AND

PROPRIETARY

NVIDIA TECHNOLOGY

C++, .NET, Java

SPECIALIZED FRAMEWORKS 16

Open Inventor, Nvidia Engines (SceniX, OptiX…), Equalizer, Chromium ...

HPC

Hadoop, HPCC, Array-based DBMS (Rasdaman, SciDB) ...

NVidia CUDA Toolkit, AMD OpenCL SDK ...

Big Data

Визуализация

CONCLUSION17

HPC HARWARE & SOFTWARE EXISTS

INDUSTRIAL SPECIALIZATION

COMPLETE DATA & PROCESSES INTEGRATION METHODOLOGY

COMMON HPC SOFTWARE ECOSYSTEM

SOFTWARE FRAMEWORKS HARD CHOICE

18

THANK YOU FOR YOUR ATTENTION!