CloudCast: Cloud Computing for Short-term Mobile Weather

Forecasts

Dilip Kumar Krishnappa,David Irwin, Eric Lyons and Michael Zink

IPCCC 2012

2ECE Department

Outline

Introduction

Cloud Services

CloudCast Architecture

Cloud Measurements

Nowcasting

Conclusion

3ECE Department

Motivation

• In 75 days, 90 hours of convective precipitation• Nowcast only useful during these events• Cost for dedicated hardware $4000 (does not

include power, AC, maintenance)• Idle for 95% of the time• At 45 cents/ hour, 90 hours of active use would

cost $40 per product, plus $2 per user to stream the resultant data out of the cloud

4ECE Department

Nowcast Example

5ECE Department

Introduction

Dedicating high-end servers for intermittent requirement wastes resources.

Cloud services IaaS model suits such applications.

Short term weather forecasting is one such real-time application.

Cost of operation for weather predicted last year for 75 days shows 50$ of cloud usage vs. 4000$ of dedicated hardware.

Are cloud services well suited for such applications?

6ECE Department

Introduction (Contd..)

We consider real-time application of short-term weather forecasting called “Nowcasting”.

Nowcasting produces short-term forecasts for 10s of minutes in the future.

Nowcasting has strict time constraints.

Are cloud services capable of keeping up with such strict time constraints?

7ECE Department

Cloud Services

Commercial Cloud Services: Amazon EC2, Rackspace

Research Cloud Testbeds: GENICloud, ExoGENI

Commercial cloud services provide on-demand resources based on per hour pricing.

Pricing depends on the instance requested and the amount of I/O performed per hour.

8ECE Department

Cloud Services (Contd..)

GENICloud is an open source cloud platform using Eucalyptus as a base.

GENICloud provides slice-based architecture to acquire cloud instances.

ExoGENI is open source cloud platform provisioned by ORCA control framework.

ExoGENI provides flexibility to users to upload the image of their own and extend the size of the image.

9ECE Department

NEXRAD Radars

10ECE Department

Measurements

PlanetLab measurement to mimic the NEXRAD radars.

159 NEXRAD radar sites around the country. 103 PlanetLab nodes found near to those locations.

According to our own radar testbed setup (CASA), 5 Mbps is the minimum throughput of data transfer from radars.

11ECE Department

Measurements (Contd..)

Three types of data transfer experiment.- Serial Data Transfer- Parallel Data Transfer- Distributed Data Transfer

Parallel transfer helps to understand the bottleneck in the system.

Data is transferred to nearest instance based on the subset of nodes in distributed data ingest.

12ECE Department

Distributed Data transfer

Radar2

Radar1 Radar3

Radar4

Cloud Instance

Cloud Instance1 Cloud Instance2

Radar1

Radar2

Radar3

Radar4

Parallel Data Transfer

Distributed Data Transfer

13ECE Department

Results – Serial, Commercial Cloud

14ECE Department

Results – Serial, Research Cloud

15ECE Department

Summary of Results

Measurement Type

EC2 East (Mbps)

EC2 West(Mbps)

Rackspace(Mbps)

GENICloud(Mbps)

ExoGENI(Mbps)

Serial 85.035 36.248 35.335 9.744 110.22

Parallel 3.146 1.249 14.122 7.364 17.2

Distributed 32.463 9.995 34.159 9.434 112.55

16ECE Department

Computation Time Analysis

Instance Type

Memory (GB)

Disk (GB)

CPU Cost/hr ($)

Total Cost ($)

Exec. Time (sec)

Total Time (sec)

Amzon EC2 7.5 850 4 0.34 1.13 74.34 95.08

Rackspace 8.0 320 4 0.48 1.63 96.53 120.33

GENICloud 8.0 20 4 - - 67.45 78.60

ExoGENI 8.0 20 4 - - 56.83 72.07

17ECE Department

Nowcasting

Main objective of our work is the feasibility of Cloud Services for short-term weather forecasting.

We have found from our studies that, cloud services are feasible for Nowcasting.

We performed live measurement and analysis to determine the overall time taken to generate 15-minute Nowcasts in each of the cloud services considered.

We find that it takes just about 2 minutes max for the generation of 15-minute Nowcasts and sending the results to the end user.

18ECE Department

Data Aggregation

19ECE Department

DFW Radar Deployment

NETWORK ROLLOUT:•First phase in blue, Second phase in green•UNT•UTA•Town of Addison•City of Fort Worth

20ECE Department

Installation of UTA radar

21ECE Department

Future Scenario

22ECE Department

Conclusion In our work, we look into the feasibility of cloud

services for real time scientific applications. Scientific application in our case is short-term

weather forecasting called Nowcasting. We perform a series of measurements to

determine our objective. From our measurements, we find that cloud

services perform best when there is less number of radar nodes sending data to a single instance.

We conclude that, having a subset of radar nodes sending data to a single instance works best for our real-time application.

23ECE Department

US Ignite – Ultra-high Bandwidth

Operations Control Center

Operations Control Center

Short-term Forecast in the Cloud

Short-term Forecast in the Cloud Long-term,

Large-scaleForecast

Long-term, Large-scaleForecast

End Users:Public, NWSEM,MediaIndustry

End Users:Public, NWSEM,MediaIndustry

Up to 150 Mbps

Up to n * 150 Mbps