Upload
rahul-gupta
View
372
Download
0
Embed Size (px)
Citation preview
Dr K C Gouda, ScientistCSIR -4PI, Wind Tunnel RoadBelur Campus, Bangalore-37.
Dr Shakti BanerjeeProfessor and Head of Dept, School of Statistics, Devi Ahilya Viswavidyalaya,Taksashila Parisar, Khandwa Road,Indore:452001
School of Future Studies & Planning,DAVV, Indore (M.P.)
Introduction
• The main aim of the project work entitled“DEVELOPMENT OF VISUALIZATION SOFTWAREFOR WEATHER AND CLIMATE DATA ANALYSIS” isto analyze the input file and determine if the datavalues fall within a certain range and to generatea new file of the same format using the thresholdvalue specified by the user.
• In our project we have read the netCDF files witha utility called ‘ncdump’ and generated a new file.
Organisation ProfileIn the late 1980's, the Council of Scientific and Industrial Research (CSIR) recognized the growingsignificance of Mathematical Modelling and Computer Simulation, as it became evident thatmodelling approaches were critical for illuminating the structure and evolution of complex systemsthat were invading the areas of scientific analysis and technological design.
Computing Facilities at C-MMACSIt has one of the most sophisticated and advanced computing environments in India and certainlythe best in CSIR. The highest performance computing and networking resources in C-MMACSincludes:
• SGI Altix 3700 BX2 Compute Servers• SGI Origin 3900 Compute Server• SGI Altix 350 Compute Servers• SGI Origin 200 Compute Server• High Availability IBM Storage Server• SGI Octane 2 Workstation• SGI Indigo2 Extreme Workstations• SGI O2 Workstations• SGI Indy Workstations• HP Xeon Linux Servers• HP/Compaq D330 Linux Workstations• IBM Net vista A30 Linux/Windows Desktops• IBM PC 300GL Linux/Windows Desktops• C-MMACS Local Area Network
Research Activities• C-MMACS Environment Modelling Program• Monsoon simulation• Tropical Modelling• Hydrological Modelling• Neural Network forecasting• Ocean Atmosphere coupling• Ocean Modelling• Atmospheric modelling• Lithosphere Modelling• Dynamical System Modelling• Industrial Computational MechanicsAreas of Specialization• Aerospace Propulsion• Computational Aerodynamics• Wind Tunnel Testing• Flight mechanics and control• Structural mechanics• Parallel Computing
Organisation Profile
Climate and Weather
Weather
Weather only describe the short-term conditionof the variable in the given region.
Climate
Climate is a measure of the average pattern ofvariation in temperature, humidity, atmosphericpressure, wind, precipitation, atmosphericparticle count and other meteorological variablesin a given region over long period of time.
Description of Related Study
Latitude and LongitudeLatitude is the angular distance, in degrees, minutes and seconds ofa point north and south of the Equator.Longitude is the angular distance, in degrees, minutes and secondsof a point east or west of the Prime (Greenwich) Meridian
ResolutionResolution is represented in the of number of grid points. Increasein resolution means increasing the number of grid points.
GridGrid is a pattern of horizontal and vertical lines on a chart or map,such as those representing latitude and longitude, usually crossingeach other to form squares/rectangles which help determiningabsolute location.
Description of Related Study
Meteorological Weather and climate DataMeteorology is the interdisciplinary scientific study of the atmospherethat focuses on weather process and forecasting.Meteorological phenomena are observable weather events whichilluminate and are explained by the science of meteorology.Those events are bound by the variables that exist in Earth’s atmosphere.They are temperature, air pressure, water vapour and the gradients andinteractions of each variable and how they change in time.Atmosphere and oceanographic data may be archived in several differentcomputer forms: character format, native format, packed binary or in oneof several “standard” scientific data formats. Users of datasets must beaware of how the data are stored.Architecture independent standard formats commonly used foratmosphere and oceanographic datasets include:
• NetCDF (network Common Data Format)• GrADS
Scope of the System
The proposed system aims to analyse data indifferent formats and to perform quality control ifthe data is not within certain standard ranges. Thesoftware is capable of analysing NetCDF, GRIB, ASCIIdata formats. The GUI is incorporated to performthese operations in a user friendly way.
Objective of the Proposed System
• Extracting the data values of the given parameters from the data file.
• Scanning the data and determining the maximum and minimum values in a domain.
• Development of algorithm for Climate data analysis
• Multi-format and Multi-source climate data analysis.
• Development of data display system for User
• Data Mining
• Spatial Interpolation modeling
• Spatial Statistical Analysis.
System Module
My software work as a System and thissoftware system is divided into two part asfollows :
• Perform Statistical Operation on Mined data.
• Perform Interpolation and generate spatialimage Based on mined data.
Data Flow Diagram
User
Weather & Climate Data
Display System
Fill Essential Data
Display Spatial
Maps & Chart
Shapefile(.shp)
Draw Region
Boundary
NetCDF Data
(.nc)
Analysis
the Data
SaveSpatial Image
(.jpeg)
Figure : 0th Level DFD
User
Spatial Analysis
Interpolated
Spatial Map
Fill Essential
Information
Fill Essential
Information
NetCDF Data
(.nc)
Compare the Data
Perform
Interpolation
Shapefile(.shp)Make
Boundary
Statistical Information
& Chart
Interpolated
Map
Figure : 1st Level DFD
Platforms, Languages and Tools
Hardware Configuration
• Processor Intel Pentium Dual Core and Above
• Memory: 512 MB RAM and Above
• Clock Speed: 1.72 GHz and Above
• Hard Disk Capacity: 80GB
Software Configuration
• Operating System: Ubuntu, Fedora Linux, Windows, Mac
• Language: Java
• Complier: Java Compiler (Java Virtual Machine)
• Other tools and utilities: JavaFX, Visual Paradism, many .jar files
Platforms, Languages
• JAVAThe Java platform is the name for a bundle of related programs from Sun thatallow for developing and running programs written in the Java programminglanguage. The platform is not specific to any one processor or operating system,but rather an execution engine (called a virtual machine) and a compiler with a setof libraries that are implemented for various hardware and operating systems sothat Java programs can run identically on all of them.Java Card: A technology that allows small Java-based applications (applets) to berun securely on smart card and similar small-memory devices.Java ME(Micro Edition): Specifies several different sets of libraries (known as
profiles) for devices with limited storage, display, and power capacities. Often usedto develop applications for mobile devices, PDAs, TV set-of-box, and printers.Java SE (Standard Edition): For general-purpose use on desktop PCs, servers andsimilar devices.Java EE (Enterprise Edition): Java SE plus various APIs useful for multi-tier client-server, enterprise application.
Platforms, Languages and Tools
Platforms, Languages
JAVA compilerWhen you program for the Java platform, you write source code in.java files and then compile them. The compiler checks your codeagainst the language's syntax rules, then writes out bytecodes in .classfiles. Bytecodes are standard instructions targeted to run on a Javavirtual machine (JVM). In adding this level of abstraction, the Javacompiler differs from other language compilers, which write outinstructions suitable for the CPU chipset the program will run on.
JVMAt run time, the JVM reads and interprets .class files and executes theprogram's instructions on the native hardware platform for which theJVM was written. The JVM interprets the bytecodes just as a CPUwould interpret assembly-language instructions. The difference is thatthe JVM is a piece of software written specifically for a particularplatform. The JVM is the heart of the Java language's "write-once,run-anywhere" principle.
Platforms, Languages and Tools
Tools
JavaFX ApplicationsWritten as a Java API,JavaFX applicationcode can referenceAPIs from any Javalibrary.The look and feel ofJavaFX applications canbe customized.Graphic designers caneasily customize theappearance and styleof the applicationthrough the CSS.
Platforms, Languages and Tools
EclipsePlatforms, Languages and Tools
Design Pattern
MVC PatternMVC Pattern stands for Model-View-Controller Pattern. Thispattern is used to separateapplication's concerns.
• Model - Model represents an objector JAVA POJO carrying data. It canalso have logic to update controllerif its data changes.
• View - View represents thevisualization of the data that modelcontains.
• Controller - Controller acts on bothModel and view. It controls the dataflow into model object and updatesthe view whenever data changes. Itkeeps View and Model separate.
Design Pattern
Data File Format
.shp(shapefile):• The Esri shapefile, or simply a shapefile, is a
popular geospatial vector data format forgeographic information system software. It isdeveloped and regulated by Esri as a (mostly)open specification for data interoperabilityamong Esri and other GIS software products.Shapefiles spatially describe vector features:points, lines, and polygons, representing, forexample, water wells, rivers, and lakes. Each itemusually has attributes that describe it, such asname or temperature.
• While the term "shapefile" is quite common, a"shapefile" is actually a set of several files. Threeindividual files are mandatory to store the coredata that comprise a shapefile:
• I used shapefilereader.jar. By using this jar we canread any shapefile.
Data File Format
• .shp File Structure (Header)Bytes Type Endianess Usage
0–3 int32 big File code (always hex value 0x0000270a)
4–23 int32 big Unused; five uint32
24–27 int32 big File length (in 16-bit words, including the header)
28–31 int32 little Version
32–35 int32 little Shape type (see reference below)
36–67 double little
Minimum Bounding rectangle (MBR) of all shapes
contained within the shapefile; four doubles in the
following order: min X, min Y, max X, max Y
68–83 double littleRange of Z; two doubles in the following order: min Z, max
Z
84–99 double littleRange of M; two doubles in the following order: min M,
max M
Data File Format
Data File Format• .shp file structure
Value Shape type Fields
0 Null shape None
1 Point X, Y
3 Polyline MBR, Number of parts, Number of points, Parts, Points
5 Polygon MBR, Number of parts, Number of points, Parts, Points
8 MultiPoint MBR, Number of points, Points
11 PointZ X, Y, Z, M
13 PolylineZMandatory: MBR, Number of parts, Number of points, Parts, Points, Z range, Z array
Optional: M range, M array
15 PolygonZMandatory: MBR, Number of parts, Number of points, Parts, Points, Z range, Z array
Optional: M range, M array
18 MultiPointZMandatory: MBR, Number of points, Points, Z range, Z array
Optional: M range, M array
21 PointM X, Y, M
23 PolylineMMandatory: MBR, Number of parts, Number of points, Parts, Points
Optional: M range, M array
25 PolygonMMandatory: MBR, Number of parts, Number of points, Parts, Points
Optional: M range, M array
28 MultiPointMMandatory: MBR, Number of points, Points
Optional Fields: M range, M array
31 MultiPatch
Mandatory: MBR, Number of parts, Number of points, Parts, Part types, Points, Z
range, Z array
Optional: M range, M array
Data File Format
Data File Format
.nc (NetCDF(Network Common Data Format))A platform independent format for representing multi-dimensional array-orientated scientific data.
• Self Describing - a netCDF file includes information about the data it contains.
• Direct Access - a small subset of a large dataset may be accessed efficiently, without first reading through all the preceding data.
• Sharable - one writer and multiple readers may simultaneously access the same netCDF file.
• NetCDF is new to the GIS community but widely used by scientific communities for around many years
Data File Format
Data File Format
• Multidimentional Data Representation :
XY
TZ
XY
T
Data File Format
Data File Format• Digital Value processed from a satellite data
141 241 341
131 231 331
121 221 321
111 211 311
441
431
421
411
142 242 342
132 232 332
122 222 322
112 212 312
442
432
422
412
143 243 343
133 233 333
123 223 323
113 213 313
443
433
423
413
Y
X
Time
141 241 341
131 231 331
121 221 321
111 211 311
441
431
421
411
142 242 342
132 232 332
122 222 322
112 212 312
442
432
422
412
143 243 343
133 233 333
123 223 323
113 213 313
443
433
423
413
Y
X
Time
Altitude
Data File Format
System Design
Goals of Implementation• Extracting the data values of the given parameters from the
data file.• Scanning the data and perform statistical operation.• Development of algorithm for Climate data analysis• Develop Linear Matrix Interpolation Technique.• Read and Plot shapefile(.shp) ESRI file formate.• NetCDF-format and Multi-source climate data analysis.• Analysis and visualisation of satellite data to study climate
structure• Give the better visualization to GUI it gives give more
accurate result within time.
Functional Requirement
Input RequirementsTo collect the data which may be in netCDF(.nc) and shapefiles(.shp)
• To obtain the threshold value from the user.• Save the extracted data values in the text format and Image formate.Processing Requirements
To read the information from the input file.• To calculate and store statistical result and perform charting in easy way.• Read the shapefile and plot shapefile into GUI for gives country
boundaries.• To generate the pictorial representation of the output.Output Requirements• The user can generate an image from the data values.
26
System Design
Use Case Diagram:System Design
Sequence Diagram for Statistic AnalysisSystem Design
Sequence Diagram for generate Interpolated ImageSystem Design
Package DiagramSystem Design
Class Diagram System Design
Result and DiscussionStarting Screen :
Select Year from Year List for Data Show in Table
Result and Discussion
ncDumpInformation of NetCDF files
Result and Discussion
Mining the Data from NetCDF FileResult and Discussion
Prepare Charts Based on DataResult and Discussion
Draw Shapefile in PanelResult and Discussion
How To make Shapes in Image File• I relate geo-cordinate system with screen pixel system.• In geo-cordinate system have 180(-90 to 90) Latitude and 360 (-180
to 179)Longitude and I take panel its width=730 and Height=628.Ratio(x) = 730/360 = 2.028Ratio(y) = 628/180 = 3.49
• Choose minimum ratio, if I choose max ratio than some shapes of shapefile will not show in panel.
• So if my co-ordinate is (-180, -90) = Panel Pixel(0, 0)• (int((-180 + 180 ) * 2.028)) = 0 and (int((-90 + 90) * 2.028) )= 0 is
minimum lat and lon point(0, 0).• If my co-ordinate is (40, -20) = (int((40 + 180) * 2.028)) = 446
(int((-20 + 90) * 2.028)) = 141 so pixel co-ordinate is (446, 141).
Result and Discussion
Perform InterpolationResult and Discussion
Interpolation Method
• 4-Point Matrix Interpolation :
Result and Discussion
Interpolation MethodCalculate Estimated Temperature
Result and Discussion
Interpolation Algorithm
• Calculate pixel respect to geo-cordinate.
• Set into as a Matrix.
Result and Discussion
Zoom Specific AreaResult and Discussion
Advantage
• The big advantage is that Scientist use a software GrADS(Grid Analysisand Display Design). GrADS has a scripting language. But my softwaregives GUI effects. Easy to analysis NetCDF data and perform Systemsoperation. No need to go write console level scripting or generate a Batchfile.
• The software is totally platform independent.• The whole package is easily installable in any type Operating System
and its supportive JAR(Java Application Runtime).• We can perform statistics result easily.• It’s create high resolution picture when I perform interpolation.• It can be widely used by scientist and users from various sectors like
agriculture, health etc to adopt the climate information for the crop, disease analysis and Cyclone Detection.
• It can also be used to train young researchers of various fields regarding multi-source and multi-format weather and climate data analysis and weather informatics etc.
Result and Discussion
Conclusion
The project “DEVELOPMENT OF VISULIZATION SOFTWAREFOR WEATHER AND CLIMATE ANALYSIS” aims to analyzedata in different formats and to perform quality control ifthe data is not within certain standard ranges. The softwareis capable of analyzing NetCDF data format.Almost all the objectives of the project are fulfilled.
• 1. Displaying the header information of the input file.• 2. Extracting the data values of the given variable in the
input file.• 3. Scanning the data.• 4. Generating a new interpolated image file respect of
extracted NetCDF data.
Conclusion & Future Enhancement
Future Enhancement
Some enhancements can be made in future as follows:
• An enhancement can be done so that the user can choose in which format he/she wants the output file.
• An improvement can be made so that other data formats, such as HDF, Binary etc., can also be given as input files.
• More and more data can be studied to understand the climate change and its impact at the regional scales.
• A GUI can be developed for the easy understanding of the weather and climate informatics.
Conclusion & Future Enhancement
Thanks