HENP Networks and Grids for Global VOs: from Vision to Reality HENP Networks and Grids for Global VOs: from Vision to Reality Harvey B. Newman Harvey B.

  • Published on
    20-Jan-2016

  • View
    213

  • Download
    0

Transcript

<ul><li><p> HENP Networks and Grids for Global VOs: from Vision to Reality Harvey B. Newman California Institute of Technology GNEW2004 March 15, 2004</p><p>Computing Model Progress CMS Internal Review of Software and Computing</p></li><li><p>ICFA and Global Networks for HENPNational and International Networks, with sufficient (rapidly increasing) capacity and seamless end-to-end capability, are essential forThe daily conduct of collaborative work in both experiment and theoryDetector development &amp; construction on a global scaleGrid systems supporting analysis involving physicists in all world regions The conception, design and implementation of next generation facilities as global networksCollaborations on this scale would never have been attempted, if they could not rely on excellent networks</p><p>Computing Model Progress CMS Internal Review of Software and Computing</p></li><li><p>The Challenges of Next Generation Science in the Information AgeFlagship Applications High Energy &amp; Nuclear Physics, AstroPhysics Sky Surveys: TByte to PByte block transfers at 1-10+ Gbps eVLBI: Many real time data streams at 1-10 Gbps BioInformatics, Clinical Imaging: GByte images on demandHEP Data Example: From Petabytes in 2004, ~100 Petabytes by 2008, to ~1 Exabyte by ~2013-5.Provide results with rapid turnaround, coordinating large but limited computing and data handling resources, over networks of varying capability in different world regionsAdvanced integrated applications, such as Data Grids, rely on seamless operation of our LANs and WANsWith reliable, quantifiable high performancePetabytes of complex data explored and analyzed by 1000s of globally dispersed scientists, in hundreds of teams</p><p>Computing Model Progress CMS Internal Review of Software and Computing</p></li><li><p>Four LHC Experiments: The Petabyte to Exabyte ChallengeATLAS, CMS, ALICE, LHCB Higgs + New particles; Quark-Gluon Plasma; CP Violation Tens of PB 2008; To 1 EB by ~2015 Hundreds of TFlops To PetaFlops 6000+ Physicists &amp; Engineers; 60+ Countries; 250 Institutions</p><p>Computing Model Progress CMS Internal Review of Software and Computing</p></li><li><p>First Beams: April 2007Physics Runs: from Summer 2007TOTEMpp, general purpose; HILHCb: B-physicsALICE : HI pp s =14 TeV L=1034 cm-2 s-1 27 km Tunnel in Switzerland &amp; FranceLarge Hadron Collider (LHC) CERN, Geneva: 2007 Start CMSATLASAtlas</p><p>Computing Model Progress CMS Internal Review of Software and Computing</p></li><li><p>LHC: Higgs Decay into 4 muons (Tracker only); 1000X LEP Data Rate109 events/sec, selectivity: 1 in 1013 (1 person in a thousand world populations)</p><p>Computing Model Progress CMS Internal Review of Software and Computing</p></li><li><p>LHC Data Grid HierarchyTier 1Online System CERN Center PBs of Disk; Tape RobotFNAL CenterIN2P3 Center INFN Center RAL Center InstituteInstituteInstituteInstitute Workstations~100-1500 MBytes/sec2.5-10 Gbps0.1 to 10 GbpsTens of Petabytes by 2007-8. An Exabyte ~5-7 Years later.Physics data cache~PByte/sec~10 Gbps~2.5-10 GbpsTier 0 +1Tier 3Tier 4Tier 2ExperimentCERN/Outside Resource Ratio ~1:2 Tier0/( Tier1)/( Tier2) ~1:1:1Emerging Vision: A Richly Structured, Global Dynamic System</p><p>Computing Model Progress CMS Internal Review of Software and Computing</p></li><li><p>ICFA Standing Committee on Interregional Connectivity (SCIC)Created by ICFA in July 1998 in VancouverCHARGE: Make recommendations to ICFA concerning the connectivity between the Americas, Asia and EuropeAs part of the process of developing these recommendations, the committee should Monitor traffic Keep track of technology developmentsPeriodically review forecasts of future bandwidth needs, and Provide early warning of potential problemsRepresentatives: Major labs, ECFA, ACFA, North and South American Physics Community</p><p>Computing Model Progress CMS Internal Review of Software and Computing</p></li><li><p> SCIC in 2003-2004http://cern.ch/icfa-scicWGs formed in March 2002: Monitoring, Digital Divide, Advanced Technologies, RequirementsStrong Focus on the Digital Divide Continues Progress in MonitoringA World Survey of Natl and Intl Networks &amp; Optical Net Initiatives Presentations &amp; Demos at &gt; 40 Meetings and Workshops: E.g., Internet2, TERENA, AMPATH, APAN, CHEP2003, SC2003, Trieste, Telecom World 2003, WSIS/RSIS, GLORIAD Launch; Digital Divide and HEPGrid Workshop: February 16-20 in RioHENP increasingly visible to governmentsThrough Network advances (records), Grid developments, Work on the Digital Divide and issues of Global Collaboration (Rio Workshop; the WSIS Process)A Striking Picture is Emerging: of Remarkable Progress, and a Deepening Digital Divide Among Nations</p><p>Computing Model Progress CMS Internal Review of Software and Computing</p></li><li><p> SCIC in 2003-2004 http://cern.ch/icfa-scicStrong Focus on the Digital Divide Since 2002 Three 2004 Reports; Presented to ICFA Feb. 13 Main Report: Networking for HENP [H. Newman et al.]Includes Brief Updates on Monitoring, the Digital Divide and Advanced Technologies [*]A World Network Overview (with 27 Appendices): Status and Plans for the Next Few Years of National and Regional Networks, and Optical Network Initiatives Monitoring Working Group Report [L. Cottrell] Digital Divide in Russia [V. Ilyin] [*] Also See the 2003 SCIC Reports of the Advanced Technologies and Digital Divide Working Groups</p><p>Computing Model Progress CMS Internal Review of Software and Computing</p></li><li><p>ICFA Report: Networks for HENPGeneral Conclusions (1)Bandwidth Usage Continues to Grow by 80-100% Per YearCurrent generation of 2.5-10 Gbps backbones and major Intl links used by HENP arrived in the last 2 Years [US+Europe+Japan+Korea] Capability Increased from ~4 to several hundred times, i.e. much faster than Moores Law This is a direct result of the continued precipitous fall of network prices for 2.5 or 10 Gbps links in these regions Technological progress may drive BW higher, unit price lower More wavelengths on a fiber; Cheap, widespread Gbit EthernetGrids may accelerate this growth, and the demand for seamless high performanceSome regions are moving to owned or leased dark fiber The rapid rate of progress is confined mostly to the US, Europe, Japan , Korea, and the major TransAtlantic and Pacific routes This may worsen the problem of the Digital Divide</p><p>Computing Model Progress CMS Internal Review of Software and Computing</p></li><li><p>History of Bandwidth Usage One Large Network; One Large Research SiteSLAC Traffic ~300 Mbps; ESnet Limit Growth in Steps: ~ 10X/4 Years Projected: ~2 Terabits/s by ~2014ESnet Accepted Traffic 1/90 1/04 Exponential Growth Since 92; Annual Rate Increased from 1.7 to 2.0X Per Year In the Last 5 Years</p><p>Computing Model Progress CMS Internal Review of Software and Computing</p><p>Chart1</p><p>0.019</p><p>0.0285</p><p>0.0478</p><p>0.0499</p><p>0.0557</p><p>0.0787</p><p>0.089</p><p>0.109</p><p>0.12</p><p>0.133</p><p>0.106</p><p>0.113</p><p>0.143</p><p>0.136</p><p>0.162</p><p>0.169</p><p>0.173</p><p>0.187</p><p>0.214</p><p>0.228</p><p>0.245</p><p>0.326</p><p>0.295</p><p>0.319</p><p>0.314</p><p>0.326</p><p>0.309</p><p>0.307</p><p>0.361</p><p>0.437</p><p>0.414</p><p>0.436</p><p>0.462</p><p>0.517</p><p>0.534</p><p>0.535</p><p>0.579</p><p>0.553</p><p>0.673</p><p>0.64</p><p>0.642</p><p>0.628</p><p>0.792</p><p>0.863</p><p>0.8</p><p>1.19</p><p>1.17</p><p>1.09</p><p>1.16</p><p>1.19</p><p>1.43</p><p>1.44</p><p>1.64</p><p>1.72</p><p>1.84</p><p>1.81</p><p>1.76</p><p>2.09</p><p>2.28</p><p>2.45</p><p>2.4</p><p>2.45</p><p>2.83</p><p>2.57</p><p>2.79</p><p>2.82</p><p>3.17</p><p>3.14</p><p>3.32</p><p>3.55</p><p>3.44</p><p>3.6</p><p>3.62</p><p>3.12</p><p>3.41</p><p>3.63</p><p>3.54</p><p>2.81</p><p>3.26</p><p>3.76</p><p>3.31</p><p>3.57</p><p>3.85</p><p>3.45</p><p>3.75</p><p>3.62</p><p>4.53</p><p>4.92</p><p>4.358</p><p>4.608</p><p>5.215</p><p>5.23</p><p>5.92</p><p>8.255</p><p>6.98</p><p>8.23</p><p>8.8</p><p>8.27</p><p>8.65</p><p>9.15</p><p>9.03</p><p>8.83</p><p>10.08</p><p>10.2</p><p>10.9</p><p>12.1</p><p>12.9</p><p>14.6</p><p>14.4</p><p>15.2</p><p>17.1</p><p>17.1</p><p>18.7</p><p>18.8</p><p>18.9</p><p>20.4</p><p>20.1</p><p>26.6</p><p>28.6</p><p>24.4</p><p>25.3</p><p>29.7</p><p>28.1</p><p>29.6</p><p>31</p><p>35.7</p><p>39</p><p>38.5</p><p>34.9</p><p>40.4</p><p>39.7</p><p>34.1</p><p>41.8</p><p>41.4</p><p>56.6</p><p>49.9</p><p>49.11</p><p>43.01</p><p>49.81</p><p>63.8</p><p>69.7</p><p>86</p><p>101</p><p>100</p><p>117</p><p>86.1</p><p>114</p><p>98</p><p>103</p><p>103</p><p>124</p><p>131</p><p>140</p><p>158</p><p>208</p><p>119</p><p>121</p><p>165</p><p>159</p><p>146</p><p>186</p><p>166</p><p>180</p><p>190</p><p>205</p><p>255</p><p>260</p><p>248</p><p>Jan, 04</p><p>TByte/Month</p><p>ESnet Monthly Accepted Traffic 1/90-1/04</p><p>Traffic</p><p>MonthBytes AcptdTbytes AcptdPkts AcptdBytes/Pktx2 Growth50%75%1.02BudgetBYTES/MONTH</p><p>Jan, 901.90E+100.021.28E+081.49E+022.20E+10</p><p>Feb, 902.85E+100.031.75E+081.63E+022.33E+102.66E+103.33E+101.501.40</p><p>Mar, 904.78E+100.052.49E+081.92E+022.47E+103.99E+104.99E+101.68</p><p>Apr, 904.99E+100.052.68E+081.86E+022.62E+106.69E+108.37E+101.04</p><p>May, 905.57E+100.063.27E+081.70E+022.77E+106.99E+108.73E+101.12</p><p>Jun, 907.87E+100.083.81E+082.07E+022.94E+107.80E+109.75E+101.41</p><p>Jul, 908.90E+100.094.63E+081.92E+023.11E+101.10E+111.38E+111.13</p><p>Aug, 901.09E+110.115.28E+082.06E+023.30E+101.25E+111.56E+111.22</p><p>Sep, 901.20E+110.125.48E+082.19E+023.49E+101.53E+111.91E+111.10</p><p>Oct, 901.33E+110.136.84E+081.94E+023.70E+101.68E+112.10E+111.11</p><p>Nov, 901.06E+110.115.63E+081.88E+023.92E+101.86E+112.33E+110.80</p><p>Dec, 901.13E+110.115.53E+082.04E+024.15E+101.48E+111.86E+111.07</p><p>Jan, 911.43E+110.146.99E+082.05E+024.40E+101.58E+111.98E+111.27</p><p>Feb, 911.36E+110.146.89E+081.97E+024.66E+102.00E+112.50E+110.95</p><p>Mar, 911.62E+110.168.17E+081.98E+024.94E+101.90E+112.38E+111.19</p><p>Apr, 911.69E+110.178.32E+082.03E+025.23E+102.27E+112.84E+111.04</p><p>May,911.73E+110.178.36E+082.07E+025.54E+102.37E+112.96E+111.02</p><p>Jun, 911.87E+110.199.61E+081.95E+025.87E+102.42E+113.03E+111.08</p><p>Jul,912.14E+110.211.13E+091.90E+026.22E+102.62E+113.27E+111.14</p><p>Aug, 912.28E+110.231.19E+091.92E+026.59E+103.00E+113.75E+111.07</p><p>Sep, 912.45E+110.251.11E+092.21E+026.98E+103.19E+113.99E+111.07</p><p>Oct, 913.26E+110.331.57E+092.08E+027.40E+103.43E+114.29E+111.33</p><p>Nov, 912.95E+110.301.46E+092.02E+027.84E+104.56E+115.71E+110.90</p><p>Dec, 913.19E+110.321.38E+092.32E+028.31E+104.13E+115.16E+111.08</p><p>Jan, 923.14E+110.311.62E+091.94E+028.80E+104.47E+115.58E+110.98Bytes/Month - Semilog</p><p>Feb, 923.26E+110.331.50E+092.17E+029.32E+104.40E+115.50E+111.04</p><p>Mar, 923.09E+110.311.54E+092.00E+029.88E+104.56E+115.71E+110.95</p><p>Apr, 923.07E+110.311.51E+092.03E+021.05E+114.33E+115.41E+110.99</p><p>May, 923.61E+110.361.70E+092.12E+021.11E+114.30E+115.37E+111.18</p><p>Jun, 924.37E+110.442.00E+092.19E+021.17E+115.05E+116.32E+111.21</p><p>Jul, 924.14E+110.411.86E+092.23E+021.24E+116.12E+117.65E+110.95</p><p>Aug, 924.36E+110.441.86E+092.34E+021.32E+115.80E+117.25E+111.05</p><p>Sep, 924.62E+110.462.02E+092.29E+021.40E+116.10E+117.63E+111.06</p><p>Oct, 925.17E+110.522.25E+092.29E+021.48E+116.47E+118.09E+111.12</p><p>Nov, 925.34E+110.532.31E+092.31E+021.57E+117.24E+119.05E+111.03</p><p>Dec, 925.35E+110.542.37E+092.26E+021.66E+117.48E+119.35E+111.004.95E+128.89E+12</p><p>Jan, 935.79E+110.582.59E+092.24E+021.76E+117.49E+119.36E+111.08</p><p>Feb, 935.53E+110.552.48E+092.23E+021.86E+118.11E+111.01E+120.96</p><p>Mar, 936.73E+110.673.08E+092.18E+021.98E+117.74E+119.68E+111.22</p><p>Apr, 936.40E+110.642.87E+092.23E+022.09E+119.42E+111.18E+120.95</p><p>May,936.42E+110.642.96E+092.17E+022.22E+118.96E+111.12E+121.00</p><p>Jun, 936.28E+110.633.30E+091.90E+022.35E+118.99E+111.12E+120.98</p><p>Jul, 937.92E+110.794.47E+091.77E+022.49E+118.79E+111.10E+121.26</p><p>Aug, 938.63E+110.864.32E+092.00E+022.64E+111.11E+121.39E+121.09</p><p>Sep, 938.00E+110.804.21E+091.90E+022.79E+111.21E+121.51E+120.93</p><p>Oct,931.19E+121.194.60E+092.59E+022.96E+111.12E+121.40E+121.49</p><p>Nov, 931.17E+121.174.61E+092.54E+023.14E+111.67E+122.08E+120.98</p><p>Dec, 931.09E+121.094.30E+092.54E+023.32E+111.64E+122.05E+120.939.62E+121.75E+13</p><p>Jan, 941.16E+121.164.59E+092.53E+023.52E+111.53E+121.91E+121.06</p><p>Feb, 941.19E+121.194.99E+092.39E+023.73E+111.62E+122.03E+121.03</p><p>Mar, 941.43E+121.435.77E+092.48E+023.95E+111.67E+122.08E+121.20Pkts/Month</p><p>Apr, 941.44E+121.445.77E+092.50E+024.19E+112.00E+122.50E+121.01</p><p>May, 941.64E+121.646.59E+092.49E+024.43E+112.02E+122.52E+121.14</p><p>Jun, 941.72E+121.727.00E+092.46E+024.70E+112.30E+122.87E+121.05</p><p>Jul, 941.84E+121.847.05E+092.61E+024.98E+112.41E+123.01E+121.07</p><p>Aug, 941.81E+121.817.09E+092.55E+025.27E+112.58E+123.22E+120.98</p><p>Sept, 941.76E+121.766.84E+092.57E+025.59E+112.53E+123.17E+120.97</p><p>Oct, 942.09E+122.098.01E+092.61E+025.92E+112.46E+123.08E+121.19</p><p>Nov, 942.28E+122.288.42E+092.71E+026.27E+112.93E+123.66E+121.09</p><p>Dec, 942.45E+122.458.62E+092.84E+026.64E+113.19E+123.99E+121.072.08E+133.84E+13</p><p>Jan, 952.40E+122.409.34E+092.57E+027.04E+113.43E+124.29E+120.98$388,19754.14</p><p>Feb, 952.45E+122.459.73E+092.52E+027.46E+113.36E+124.20E+121.02$380,27553.04</p><p>Mar, 952.83E+122.831.12E+102.54E+027.90E+113.43E+124.29E+121.16$329,21345.92</p><p>Apr, 952.57E+122.579.96E+092.58E+028.37E+113.96E+124.95E+120.91$362,51950.56</p><p>May, 952.79E+122.791.06E+102.63E+028.87E+113.60E+124.50E+121.09$333,93346.57</p><p>Jun, 952.82E+122.821.15E+102.46E+029.40E+113.91E+124.88E+121.01$330,38146.08</p><p>Jul, 953.17E+123.171.23E+102.58E+029.95E+113.95E+124.94E+121.12$293,90340.99</p><p>Aug, 953.14E+123.141.24E+102.54E+021.05E+124.44E+125.55E+120.99$296,71141.38</p><p>Sep, 953.32E+123.321.25E+102.65E+021.12E+124.40E+125.50E+121.06$280,62539.14</p><p>Oct, 953.55E+123.551.29E+102.76E+021.18E+124.65E+125.81E+121.07$262,44336.60</p><p>Nov, 953.44E+123.441.26E+102.73E+021.25E+124.97E+126.21E+120.97$270,83537.77</p><p>Dec, 953.60E+123.601.19E+103.02E+021.33E+124.82E+126.02E+121.053.61E+136.74E+13$258,79836.09</p><p>Jan, 963.62E+123.621.18E+103.07E+021.41E+125.04E+126.30E+121.01$268,92337.51</p><p>Feb, 963.12E+123.121.13E+102.77E+021.49E+125.07E+126.34E+120.86$312,01943.52</p><p>Mar, 963.41E+123.411.15E+102.97E+021.58E+124.37E+125.46E+121.09$285,48439.82</p><p>Apr, 963.63E+123.631.17E+103.10E+021.67E+124.77E+125.97E+121.06$268,18237.40</p><p>May, 963.54E+123.541.25E+102.84E+021.77E+125.08E+126.35E+120.98$275,00038.35</p><p>Jun, 962.81E+122.811.00E+102.81E+021.88E+124.96E+126.20E+120.79$346,44148.32Bytes/pkt</p><p>Jul, 963.26E+123.261.07E+103.04E+021.99E+123.93E+124.92E+121.16$298,62041.65</p><p>Aug, 963.76E+123.761.24E+103.03E+022.11E+124.56E+125.71E+121.15$258,91036.11</p><p>Sep, 963.31E+123.311.23E+102.70E+022.23E+125.26E+126.58E+120.88$294,10941.02</p><p>Oct, 963.57E+123.571.17E+103.05E+022.37E+124.63E+125.79E+121.08$272,68938.03</p><p>Nov, 963.85E+123.851.15E+103.36E+022.51E+125.00E+126.25E+121.08$252,85735.27</p><p>Dec, 963.45E+123.451.10E+103.15E+022.66E+125.39E+126.74E+120.904.13E+137.74E+13$282,17439.35</p><p>Jan, 973.75E+123.751.17E+103.22E+022.82E+124.83E+126.04E+121.09$267,77837.35</p><p>Feb, 973.62E+123.621.20E+103.01E+022.98E+125.25E+126.56E+120.97$277,39438.69</p><p>Mar, 974.53E+124.531.29E+103.52E+023.16E+125.07E+126.34E+121.25$221,67030.92</p><p>Apr, 974.92E+124.921.34E+103.68E+023.35E+126.34E+127.93E+121.09$204,09928.47</p><p>May, 974.36E+124.361.40E+103.11E+023.55E+126.89E+128.61E+120.89$230,41932.14</p><p>Jun, 974.61E+124.611.40E+103.29E+023.76E+126.10E+127.63E+121.06$217,91830.39</p><p>Jul, 975.22E+125.221.88E+102.78E+023.98E+126.45E+128.06E+121.13$192,55426.86</p><p>Aug, 975.23E+125.231.38E+103.79E+024.22E+127.30E+129.13E+121.00$192,00126.78</p><p>Sep, 975.92E+125.921.43E+104.14E+024.47E+127.32E+129.15E+121.13$169,62323.66</p><p>Oct, 978.26E+128.261.63E+105.06E+024.73E+128.29E+121.04E+131.39$121,64316.97</p><p>Nov, 976.98E+126.981.56E+104.47E+025.02E+121.16E+131.44E+130.85$143,86320.0...</p></li></ul>

Recommended

View more >