HENP Networks and Grids for Global Science

  • Published on
    03-Feb-2016

  • View
    38

  • Download
    0

DESCRIPTION

HENP Networks and Grids for Global Science. Harvey B. Newman California Institute of Technology 3 rd International Data Grid Workshop Daegu, Korea August 26, 2004. Challenges for Global HENP Experiments. BaBar/D0 Example - 2004 500+ Physicists 100+ Institutes 35+ Countries. - PowerPoint PPT Presentation

Transcript

  • HENP Networks and Grids for Global Science Harvey B. Newman California Institute of Technology 3rd International Data Grid Workshop Daegu, Korea August 26, 2004

    Computing Model Progress CMS Internal Review of Software and Computing

  • Challenges for Global HENP Experiments Major Challenges (Shared with Other Fields) Worldwide Communication and Collaboration Managing Globally Distributed Computing & Data Resources Cooperative Software Development and Data AnalysisLHC Example- 20075000+ Physicists 250+ Institutes 60+ CountriesBaBar/D0 Example - 2004500+ Physicists 100+ Institutes 35+ Countries

    Computing Model Progress CMS Internal Review of Software and Computing

  • First Beams: Summer 2007Physics Runs: from Fall 2007TOTEMpp, general purpose; HILHCb: B-physicsALICE : HI pp s =14 TeV L=1034 cm-2 s-1 27 km Tunnel in Switzerland & FranceLarge Hadron Collider (LHC) CERN, Geneva: 2007 Start CMSAtlasHiggs, SUSY, QG Plasma, CP Violation, the Unexpected

    Computing Model Progress CMS Internal Review of Software and Computing

  • Challenges of Next Generation Science in the Information AgeFlagship Applications High Energy & Nuclear Physics, AstroPhysics Sky Surveys: TByte to PByte block transfers at 1-10+ GbpsFusion Energy: Time Critical Burst-Data Distribution; Distributed Plasma Simulations, Visualization and Analysis; Preparations for Fusion Energy Experiment eVLBI: Many real time data streams at 1-10 Gbps BioInformatics, Clinical Imaging: GByte images on demandProvide results with rapid turnaround, coordinating large but limited computing and data handling resources, over networks of varying capability in different world regionsAdvanced integrated applications, such as Data Grids, rely on seamless operation of our LANs and WANsWith reliable, quantifiable high performancePetabytes of complex data explored and analyzed by 1000s of globally dispersed scientists, in hundreds of teams

    Computing Model Progress CMS Internal Review of Software and Computing

  • LHC Data Grid Hierarchy:Developed at CaltechTier 1Online System CERN Center PBs of Disk; Tape RobotFNAL CenterIN2P3 Center INFN Center RAL Center InstituteInstituteInstituteInstitute Workstations~100-1500 MBytes/sec~10 Gbps1 to 10 GbpsTens of Petabytes by 2007-8. An Exabyte ~5-7 Years later.Physics data cache~PByte/sec10 - 40 Gbps~10 GbpsTier 0 +1Tier 3Tier 4Tier 2ExperimentCERN/Outside Resource Ratio ~1:2 Tier0/( Tier1)/( Tier2) ~1:1:1Emerging Vision: A Richly Structured, Global Dynamic System

    Computing Model Progress CMS Internal Review of Software and Computing

  • ICFA and Global Networks for Collaborative ScienceNational and International Networks, with sufficient (rapidly increasing) capacity and seamless end-to-end capability, are essential forThe daily conduct of collaborative work in both experiment and theoryExperiment development & construction on a global scaleGrid systems supporting analysis involving physicists in all world regions The conception, design and implementation of next generation facilities as global networksCollaborations on this scale would never have been attempted, if they could not rely on excellent networks

    Computing Model Progress CMS Internal Review of Software and Computing

  • History of Bandwidth Usage One Large Network; One Large Research SiteSLAC Traffic ~300 Mbps; ESnet Limit Growth in Steps: ~ 10X/4 Years Projected: ~2 Terabits/s by ~2014ESnet Accepted Traffic 1/90 1/04 Exponential Growth Since 92; Annual Rate Increased from 1.7 to 2.0X Per Year In the Last 5 Years

    Computing Model Progress CMS Internal Review of Software and Computing

    Chart1

    0.019

    0.0285

    0.0478

    0.0499

    0.0557

    0.0787

    0.089

    0.109

    0.12

    0.133

    0.106

    0.113

    0.143

    0.136

    0.162

    0.169

    0.173

    0.187

    0.214

    0.228

    0.245

    0.326

    0.295

    0.319

    0.314

    0.326

    0.309

    0.307

    0.361

    0.437

    0.414

    0.436

    0.462

    0.517

    0.534

    0.535

    0.579

    0.553

    0.673

    0.64

    0.642

    0.628

    0.792

    0.863

    0.8

    1.19

    1.17

    1.09

    1.16

    1.19

    1.43

    1.44

    1.64

    1.72

    1.84

    1.81

    1.76

    2.09

    2.28

    2.45

    2.4

    2.45

    2.83

    2.57

    2.79

    2.82

    3.17

    3.14

    3.32

    3.55

    3.44

    3.6

    3.62

    3.12

    3.41

    3.63

    3.54

    2.81

    3.26

    3.76

    3.31

    3.57

    3.85

    3.45

    3.75

    3.62

    4.53

    4.92

    4.358

    4.608

    5.215

    5.23

    5.92

    8.255

    6.98

    8.23

    8.8

    8.27

    8.65

    9.15

    9.03

    8.83

    10.08

    10.2

    10.9

    12.1

    12.9

    14.6

    14.4

    15.2

    17.1

    17.1

    18.7

    18.8

    18.9

    20.4

    20.1

    26.6

    28.6

    24.4

    25.3

    29.7

    28.1

    29.6

    31

    35.7

    39

    38.5

    34.9

    40.4

    39.7

    34.1

    41.8

    41.4

    56.6

    49.9

    49.11

    43.01

    49.81

    63.8

    69.7

    86

    101

    100

    117

    86.1

    114

    98

    103

    103

    124

    131

    140

    158

    208

    119

    121

    165

    159

    146

    186

    166

    180

    190

    205

    255

    260

    248

    Jan, 04

    TByte/Month

    ESnet Monthly Accepted Traffic 1/90-1/04

    Traffic

    MonthBytes AcptdTbytes AcptdPkts AcptdBytes/Pktx2 Growth50%75%1.02BudgetBYTES/MONTH

    Jan, 901.90E+100.021.28E+081.49E+022.20E+10

    Feb, 902.85E+100.031.75E+081.63E+022.33E+102.66E+103.33E+101.501.40

    Mar, 904.78E+100.052.49E+081.92E+022.47E+103.99E+104.99E+101.68

    Apr, 904.99E+100.052.68E+081.86E+022.62E+106.69E+108.37E+101.04

    May, 905.57E+100.063.27E+081.70E+022.77E+106.99E+108.73E+101.12

    Jun, 907.87E+100.083.81E+082.07E+022.94E+107.80E+109.75E+101.41

    Jul, 908.90E+100.094.63E+081.92E+023.11E+101.10E+111.38E+111.13

    Aug, 901.09E+110.115.28E+082.06E+023.30E+101.25E+111.56E+111.22

    Sep, 901.20E+110.125.48E+082.19E+023.49E+101.53E+111.91E+111.10

    Oct, 901.33E+110.136.84E+081.94E+023.70E+101.68E+112.10E+111.11

    Nov, 901.06E+110.115.63E+081.88E+023.92E+101.86E+112.33E+110.80

    Dec, 901.13E+110.115.53E+082.04E+024.15E+101.48E+111.86E+111.07

    Jan, 911.43E+110.146.99E+082.05E+024.40E+101.58E+111.98E+111.27

    Feb, 911.36E+110.146.89E+081.97E+024.66E+102.00E+112.50E+110.95

    Mar, 911.62E+110.168.17E+081.98E+024.94E+101.90E+112.38E+111.19

    Apr, 911.69E+110.178.32E+082.03E+025.23E+102.27E+112.84E+111.04

    May,911.73E+110.178.36E+082.07E+025.54E+102.37E+112.96E+111.02

    Jun, 911.87E+110.199.61E+081.95E+025.87E+102.42E+113.03E+111.08

    Jul,912.14E+110.211.13E+091.90E+026.22E+102.62E+113.27E+111.14

    Aug, 912.28E+110.231.19E+091.92E+026.59E+103.00E+113.75E+111.07

    Sep, 912.45E+110.251.11E+092.21E+026.98E+103.19E+113.99E+111.07

    Oct, 913.26E+110.331.57E+092.08E+027.40E+103.43E+114.29E+111.33

    Nov, 912.95E+110.301.46E+092.02E+027.84E+104.56E+115.71E+110.90

    Dec, 913.19E+110.321.38E+092.32E+028.31E+104.13E+115.16E+111.08

    Jan, 923.14E+110.311.62E+091.94E+028.80E+104.47E+115.58E+110.98Bytes/Month - Semilog

    Feb, 923.26E+110.331.50E+092.17E+029.32E+104.40E+115.50E+111.04

    Mar, 923.09E+110.311.54E+092.00E+029.88E+104.56E+115.71E+110.95

    Apr, 923.07E+110.311.51E+092.03E+021.05E+114.33E+115.41E+110.99

    May, 923.61E+110.361.70E+092.12E+021.11E+114.30E+115.37E+111.18

    Jun, 924.37E+110.442.00E+092.19E+021.17E+115.05E+116.32E+111.21

    Jul, 924.14E+110.411.86E+092.23E+021.24E+116.12E+117.65E+110.95

    Aug, 924.36E+110.441.86E+092.34E+021.32E+115.80E+117.25E+111.05

    Sep, 924.62E+110.462.02E+092.29E+021.40E+116.10E+117.63E+111.06

    Oct, 925.17E+110.522.25E+092.29E+021.48E+116.47E+118.09E+111.12

    Nov, 925.34E+110.532.31E+092.31E+021.57E+117.24E+119.05E+111.03

    Dec, 925.35E+110.542.37E+092.26E+021.66E+117.48E+119.35E+111.004.95E+128.89E+12

    Jan, 935.79E+110.582.59E+092.24E+021.76E+117.49E+119.36E+111.08

    Feb, 935.53E+110.552.48E+092.23E+021.86E+118.11E+111.01E+120.96

    Mar, 936.73E+110.673.08E+092.18E+021.98E+117.74E+119.68E+111.22

    Apr, 936.40E+110.642.87E+092.23E+022.09E+119.42E+111.18E+120.95

    May,936.42E+110.642.96E+092.17E+022.22E+118.96E+111.12E+121.00

    Jun, 936.28E+110.633.30E+091.90E+022.35E+118.99E+111.12E+120.98

    Jul, 937.92E+110.794.47E+091.77E+022.49E+118.79E+111.10E+121.26

    Aug, 938.63E+110.864.32E+092.00E+022.64E+111.11E+121.39E+121.09

    Sep, 938.00E+110.804.21E+091.90E+022.79E+111.21E+121.51E+120.93

    Oct,931.19E+121.194.60E+092.59E+022.96E+111.12E+121.40E+121.49

    Nov, 931.17E+121.174.61E+092.54E+023.14E+111.67E+122.08E+120.98

    Dec, 931.09E+121.094.30E+092.54E+023.32E+111.64E+122.05E+120.939.62E+121.75E+13

    Jan, 941.16E+121.164.59E+092.53E+023.52E+111.53E+121.91E+121.06

    Feb, 941.19E+121.194.99E+092.39E+023.73E+111.62E+122.03E+121.03

    Mar, 941.43E+121.435.77E+092.48E+023.95E+111.67E+122.08E+121.20Pkts/Month

    Apr, 941.44E+121.445.77E+092.50E+024.19E+112.00E+122.50E+121.01

    May, 941.64E+121.646.59E+092.49E+024.43E+112.02E+122.52E+121.14

    Jun, 941.72E+121.727.00E+092.46E+024.70E+112.30E+122.87E+121.05

    Jul, 941.84E+121.847.05E+092.61E+024.98E+112.41E+123.01E+121.07

    Aug, 941.81E+121.817.09E+092.55E+025.27E+112.58E+123.22E+120.98

    Sept, 941.76E+121.766.84E+092.57E+025.59E+112.53E+123.17E+120.97

    Oct, 942.09E+122.098.01E+092.61E+025.92E+112.46E+123.08E+121.19

    Nov, 942.28E+122.288.42E+092.71E+026.27E+112.93E+123.66E+121.09

    Dec, 942.45E+122.458.62E+092.84E+026.64E+113.19E+123.99E+121.072.08E+133.84E+13

    Jan, 952.40E+122.409.34E+092.57E+027.04E+113.43E+124.29E+120.98$388,19754.14

    Feb, 952.45E+122.459.73E+092.52E+027.46E+113.36E+124.20E+121.02$380,27553.04

    Mar, 952.83E+122.831.12E+102.54E+027.90E+113.43E+124.29E+121.16$329,21345.92

    Apr, 952.57E+122.579.96E+092.58E+028.37E+113.96E+124.95E+120.91$362,51950.56

    May, 952.79E+122.791.06E+102.63E+028.87E+113.60E+124.50E+121.09$333,93346.57

    Jun, 952.82E+122.821.15E+102.46E+029.40E+113.91E+124.88E+121.01$330,38146.08

    Jul, 953.17E+123.171.23E+102.58E+029.95E+113.95E+124.94E+121.12$293,90340.99

    Aug, 953.14E+123.141.24E+102.54E+021.05E+124.44E+125.55E+120.99$296,71141.38

    Sep, 953.32E+123.321.25E+102.65E+021.12E+124.40E+125.50E+121.06$280,62539.14

    Oct, 953.55E+123.551.29E+102.76E+021.18E+124.65E+125.81E+121.07$262,44336.60

    Nov, 953.44E+123.441.26E+102.73E+021.25E+124.97E+126.21E+120.97$270,83537.77

    Dec, 953.60E+123.601.19E+103.02E+021.33E+124.82E+126.02E+121.053.61E+136.74E+13$258,79836.09

    Jan, 963.62E+123.621.18E+103.07E+021.41E+125.04E+126.30E+121.01$268,92337.51

    Feb, 963.12E+123.121.13E+102.77E+021.49E+125.07E+126.34E+120.86$312,01943.52

    Mar, 963.41E+123.411.15E+102.97E+021.58E+124.37E+125.46E+121.09$285,48439.82

    Apr, 963.63E+123.631.17E+103.10E+021.67E+124.77E+125.97E+121.06$268,18237.40

    May, 963.54E+123.541.25E+102.84E+021.77E+125.08E+126.35E+120.98$275,00038.35

    Jun, 962.81E+122.811.00E+102.81E+021.88E+124.96E+126.20E+120.79$346,44148.32Bytes/pkt

    Jul, 963.26E+123.261.07E+103.04E+021.99E+123.93E+124.92E+121.16$298,62041.65

    Aug, 963.76E+123.761.24E+103.03E+022.11E+124.56E+125.71E+121.15$258,91036.11

    Sep, 963.31E+123.311.23E+102.70E+022.23E+125.26E+126.58E+120.88$294,10941.02

    Oct, 963.57E+123.571.17E+103.05E+022.37E+124.63E+125.79E+121.08$272,68938.03

    Nov, 963.85E+123.851.15E+103.36E+022.51E+125.00E+126.25E+121.08$252,85735.27

    Dec, 963.45E+123.451.10E+103.15E+022.66E+125.39E+126.74E+120.904.13E+137.74E+13$282,17439.35

    Jan, 973.75E+123.751.17E+103.22E+022.82E+124.83E+126.04E+121.09$267,77837.35

    Feb, 973.62E+123.621.20E+103.01E+022.98E+125.25E+126.56E+120.97$277,39438.69

    Mar, 974.53E+124.531.29E+103.52E+023.16E+125.07E+126.34E+121.25$221,67030.92

    Apr, 974.92E+124.921.34E+103.68E+023.35E+126.34E+127.93E+121.09$204,09928.47

    May, 974.36E+124.361.40E+103.11E+023.55E+126.89E+128.61E+120.89$230,41932.14

    Jun, 974.61E+124.611.40E+103.29E+023.76E+126.10E+127.63E+121.06$217,91830.39

    Jul, 975.22E+125.221.88E+102.78E+023.98E+126.45E+128.06E+121.13$192,55426.86

    Aug, 975.23E+125.231.38E+103.79E+024.22E+127.30E+129.13E+121.00$192,00126.78

    Sep, 975.92E+125.921.43E+104.14E+024.47E+127.32E+129.15E+121.13$169,62323.66

    Oct, 978.26E+128.261.63E+105.06E+024.73E+128.29E+121.04E+131.39$121,64316.97

    Nov, 976.98E+126.981.56E+104.47E+025.02E+121.16E+131.44E+130.85$143,86320.06

    Dec, 978.23E+128.231.75E+104.70E+025.31E+129.77E+121.22E+131.186.56E+131.24E+14$122,01317.02

    Jan, 988.80E+128.801.91E+104.61E+025.63E+121.15E+131.44E+131.07$120,36016.79

    Feb, 988.27E+128.271.78E+104.65E+025.97E+121.23E+131.54E+130.94$128,07317.86

    Mar, 988.65E+128.651.96E+104.41E+026.32E+121.16E+131.45E+131.05$122,44717.08

    Apr, 989.15E+129.152.07E+104.42E+026.70E+121.21E+131.51E+131.06$115,75616.14

    May, 989.03E+129.031.94E+104.65E+027.09E+121.28E+131.60E+130.99$117,29416.36

    Jun, 988.83E+128.832.17E+104.07E+027.52E+121.26E+131.58E+130.98$119,95116.73

    Jul, 981.01E+1310.082.94E+103.43E+027.96E+121.24E+131.55E+131.14$105,07614.65

    Aug, 981.02E+1310.202.33E+104.38E+028.44E+121.41E+131.76E+131.01$103,84014.48

    Sep, 981.09E+1310.902.84E+103.84E+028.94E+121.43E+131.79E+131.07$97,17113.55

    Oct, 981.21E+1312.102.46E+104.92E+029.47E+121.53E+131.91E+131.11$87,53412.21

    Nov, 981.29E+1312.903.24E+103.98E+021.00E+131.69E+132.12E+131.07$82,10611.45

    Dec, 981.46E+1314.602.59E+105.64E+021.06E+131.81E+132.26E+131.131.24E+142.36E+14$72,54610.12

    Jan, 991.44E+1314.403.02E+104.77E+021.13E+132.04E+132.56E+130.99$86,37212.05

    Feb, 991.52E+1315.202.80E+105.43E+021.19E+132.02E+132.52E+131.06$81,82611.41

    Mar, 991.71E+1317.103.58E+104.78E+021.26E+132.13E+132.66E+131.13$72,73410.14

    Apr, 991.71E+1317.103.21E+105.33E+021.34E+132.39E+132.99E+131.00$72,73410.14

    May, 991.87E+1318.703.53E+105.30E+021.42E+132.39E+132.99E+131.09$66,5119.28

    Jun, 991.88E+1318.803.72E+105.05E+021.50E+132.62E+133.27E+131.01$66,1579.23

    Jul, 991.89E+1318.903.75E+105.04E+021.59E+132.63E+133.29E+131.01$65,8079.18

    Aug, 992.04E+1320.404.20E+104.86E+021.69E+132.65E+133.31E+131.08$60,9688.50

    Sep, 992.01E+1320.103.64E+105.52E+021.79E+132.86E+133.57E+130.99$61,8788.63

    Oct, 992.66E+1326.604.48E+105.94E+021.89E+132.81E+133.52E+131.32$46,7586.52

    Nov, 992.86E+1328.604.98E+105.74E+022.01E+133.72E+134.66E+131.08$43,4886.07

    Dec, 992.44E+1324.403.55E+106.87E+022.13E+134.00E+135.01E+130.852.40E+144.66E+14$50,9737.11

    Jan, 002.53E+1325.304.57E+105.54E+022.25E+133.42E+134.27E+131.04$49,7366.94

    Feb, 002.97E+1329.704.28E+106.94E+022.39E+133.54E+134.43E+131.17$42,3685.91

    Mar, 002.81E+1328.104.41E+106.37E+022.53E+134.16E+135.20E+130.95$44,7816.25

    Apr, 002.96E+1329.604.78E+106.19E+022.68E+133.93E+134.92E+131.05$42,5115.93

    May, 003.10E+1331.004.38E+107.08E+022.84E+134.14E+135.18E+131.05$40,5915.66

    Jun, 003.57E+1335.705.36E+106.66E+023.01E+134.34E+135.43E+131.15$35,2474.92

    Jul, 003.90E+1339.005.62E+106.94E+023.18E+135.00E+136.25E+131.09$32,2654.50

    Aug, 003.85E+1338.505.96E+106.46E+023.37E+135.46E+136.83E+130.99$32,6844.56

    Sep, 003.49E+1334.905.37E+106.50E+023.57E+135.39E+136.74E+130.91$36,0555.03

    Oct, 004.04E+1340.405.99E+106.74E+023.79E+134.89E+136.11E+131.16$31,1474.34

    Nov, 003.97E+1339.705.44E+107.30E+024.01E+135.66E+137.07E+130.98$31,6964.42

    Dec, 003.41E+1334.104.14E+108.24E+024.25E+135.56E+136.95E+130.864.06E+141.697.96E+14$36,9015.15

    Jan, 014.18E+1341.806.14E+106.81E+024.50E+134.77E+135.97E+131.23$32,7794.57

    Feb, 014.14E+1341.404.43E+109.35E+024.77E+135.85E+137.32E+130.99$33,0964.62

    Mar, 015.66E+1356.606.73E+108.41E+025.06E+135.80E+137.25E+131.37$24,2083.38

    Apr, 014.99E+1349.905.59E+108.93E+025.36E+137.92E+139.91E+130.88$27,4583.83

    May, 014.91E+1349.116.43E+107.64E+025.67E+136.99E+138.73E+130.98$27,9003.89

    Jun, 014.30E+1343.016.09E+107.06E+026.01E+136.88E+138.59E+130.88$31,8574.44

    Jul, 014.98E+1349....

Recommended

View more >