Intro. To GISLecture 5

Downloading and Exploring Datasets

March 4th, 2013

Reminders

• Please turn in last week’s homework

• Midterm review in 2 weeks (March 13th)

• Review Session: next Wed (March 6th)

REVIEW:“Heads-up” digitizing

• Also known as on-screen digitizing

• Scanned maps or aerial photographs used to trace features and record locations– Paper maps require a large format scanner– Images must be georeferenced– Can still be very time-consuming

REVIEW: Georeferencing• When images with unknown coordinates

are fed into GIS• 2D georeferencing: resize (rescale), rotate,

and translate to fit• Control points• Transformations:

– Polynomial– First order (affine)– spline

• Historic Map

REVIEW: New Shapefile

• Create New Shapefile– Point, polyline, polygon– Coordinate System– Empty attribute table

REVIEW: New Attributes

• Before you start to edit, add fields:– Consider the information you

need to store about the features you will be digitizing (i.e. type, name)

– Name: no spaces, characters– Choose the correct field type– For text, edit length (max =

254)

REVIEW: The Editor Toolbar

• Options greyed out depending on feature type

• Tools for creating or modifying features• Use to open the attribute window

– Allows you to edit attributes for selected feature

– Attributes can also be input into table directly

REVIEW: Geocoding• Converting street addresses to XY

coordinates

Reference Layer(Indexed Network)

Attribute Table

Results

REVIEW: Applications• Mapping restaurants in downtown

Boston• Mapping customers' addresses for your

business/education • Mapping households with high power

consumption (e.g. nstar)

REVIEW: Interpolation

Meadowlark St.

From:700

To:799

750 Meadowlark

725 Meadowlark

} Offset

REVIEW: Address Locator

• Choose locator type• Specify street data• Choose the right fields

– From address– To address– Prefix (i.e. East, West)– Street Name– Street Type (Rd./St./Ave.)– Suffix

REVIEW: Rematching• Fixing the unmatched addresses

REVIEW: Remote Sensing

03_16_Figure

REVIEW: Remote Sensing Platforms

Unmanned Airborne Vehicles

REVIEW: Earth Observing (EO)/Infrared (IR) Remote Sensing Systems

• Space borne– CORONA– IKONOS / Geoeye (high spatial res.)– Quickbird / WorldView (high spatial res.)– Landsat/ SPOT (medium spatial res.)– MODIS/VIIRS/AVHRR (low spatial res.)

• Airborne (UAV)– AVIRIS– Predator– Global Hawk

REVIEW: Concept of Resolution

• Spatial• Spectral• Temporal• Radiometric

REVIEW: Spectral Resolution

• Electromagnetic Spectrum

Pan band

REVIEW: Spectral Resolution

Refle

ctan

ce (%

)

• Electromagnetic Spectrum

REVIEW: Spectral Resolution

• Panchromatic (one single band, e.g. CORONA, old aerial photographs, IKONOS/Quickbird Pan band)

• Multispectral (several bands, e.g. Landsat, MODIS)• Hyperspectral (many bands, e.g. AVIRIS)

Courtesy of Guam Coastal Atlas

REVIEW: Trade-off between Spatial and Spectral Resolution

E (or signal)

• In order to maintain a reasonable level of energy (or signal) reaching the camera (or imaging system), the relation between the pixel size (or pixel area) and spectral bandpass (channel width) must be considered:

Pixel areaSpectral bandpassEnergy

REVIEW: Airborne Remote Sensing

• Collected by cameras mounted on planes

• Multiple passes over a short time period

• Orthorectified once images are joined

• Perspective view

Orthophoto Vs. Aerial photo/remotely sensed photo

• Bonus question: due on Wed (March 6th)• What is the difference between an

aerial photo and an orthophoto?

03_23_Figure

Very

sim

ilar

REVIEW: LiDAR• Light Detection and Ranging – laser

elevations!

Downloading Datasets

Downloading Datasets

• If somebody asked you to make a map, where would you go to find the data?

– Data often available online in digital formats

– GIS files may exist with the attributes you need

– Do some research to find out who has your data

Downloading Datasets

• Starting your web search…– Topic: environment, government, business,

health– Geography: neighborhood, city, state,

country, world– Time frame: one vs. many years; historical

data?– Sources:

• Government agencies (local, state, federal, int’l.)

• Non-profit organizations• Private corporations?

Data Sources

• Municipal GIS departments– Parcel boundaries, zoning, wards +

precincts– Street centerlines, sidewalks, building

footprints– Infrastructure

• Water supply, sewers, storm drains• Electric, gas, broadband• Municipal facilities – police, fire, DPW, schools

– Cities & towns may charge a fee for a copy of data

Data Sources

• State Agencies– MassGIS is a repository for many agencies– Political boundaries, roads, other

infrastructure– Hydrography, Wetlands, Open Space– Orthophotos, DEM, Shaded Relief

– GIS data for some other states is much harder to find!

Data Sources

• Federal Agencies– The National Atlas, U.S. Census TIGER– USGS: Nat’l. Hydro. Dataset, DEMs,

Orthophotos– NASA: Earth Observing System

Clearinghouse– NOAA: Coastal Data, Weather, Fisheries– NWS: National Wetlands Inventory– NRCS: Soil Data Mart (NATSGO, STATSGO,

SSURGO)– FEMA: Floodplains & Disaster Locations

Data Sources

• International:– United Nations –

Food & Agriculture Organization– The Nature Conservancy– OpenStreetMap

• Geofabrik extracts: http://download.geofabrik.de/

• Metro areas: http://metro.teczno.com/

Exploring the Data

• Check the metadata– “Item Description” in ArcCatalog– Details about source, attributes, date,

methods

• Make a map, play with symbology and labels– Get a sense of the range of values for

attributes– Figure out which attributes will be useful to

you

Data Structures/Models in GIS

• Vector– ???– ???

• Raster– ???– ???

Topology• How does the machine know about

relative positions of various features like point, line polygon?

• Through Topology

Vector Data and Topology• Topology

– The arrangement for how point, line, and polygon features share geometry

– Or knowledge about relative spatial positioning

• Two types of vector models exist in a GIS– Geo-relational Vector Model

• Arc Coverage (has topology) >>> format: binay• Shape files (no topology) >>>> format: *.shp, *.shx,

*dbf, etc.– Object-based Vector Model

• Includes classes and geodatabases >>> format: *.mdb

Topology• Concepts

– Adjacency– Enclosure– Connectivity

• Terms to be defined– Node– Arc– Polygon

OK….• No matter what if we have topology or

not we can ask questions from a GIS database (spatial or non-spatial) to do some quick analysis….

Query• A query is a “question” posed to a database

(attribute data)

• Examples:– Mouse click on a map symbol (e.g. road) may mean

• What is the name of road pointed to by mouse cursor ?– Typing a keyword in a search engine (e.g. google,

yahoo) means• Which documents on web contain given keywords?

– SELECT ‘FROM Senator S’ WHERE S.gender = ‘F’ means

• Which senators are female?

Non-spatial Data• Or Attributes

Record

Field (Attribute)--- It could be either numeric or text)

The Shape Field/Object ID tells about the type of vector feature (point/polygon/line)… It is where the coordinates are also stored (you do not see them here)

Organizing Attribute Data

• Flat Files • Hierarchical • Relational (databases)• Object-oriented (database)

Organizing Attribute Data

• Flat Files – Spreadsheets (e.g. excel spreadsheet)

• Hierarchical

Organizing Attribute Data

• Relational (What is commonly used in GIS)– Various tables (databases) are “linked”

through unique identifiers

Organizing Attribute Data

Query: Making Selections

• Usually interested in some subset of the data

• Selections can be made in two primary ways:– Select by Attribute – specify matching

criteria– Select by Location – based on spatial

proximity

Query: Select by Attributes

• Or Structured Query Language (SQL)

• Enter criteria for one or more fields– Numeric values =,<,>,<>– Nominal values = ‘text’

• Change criteria or narrow results based on additional criteria

Select by Attribute Tips

• Be careful with case sensitivity and spaces• Use parentheses to carefully construct a query• Use “Boolean” Operators (AND, OR, NOT, LIKE)

– AND means both criteria, OR means either– NOT allows you to exclude some criteria– LIKE lets you be more flexible, use wildcard

characters (_ for one character, % for many)– Verify your expression to make sure it works

Spatial Query: Select by Location

• Use vectors to select data from other vectors

• Same selection methods as Select by Attribute

• Choose Target & Source

• Many options for the spatial selection method

Spatial Query: Select by Location

• Spatial selection methods– Target intersects

source– …within a distance

of…– contain, completely

contain– within, completely

within– Clementini (not

boundary)– are identical to; touch

the boundary of; share a line; crossed by the outline of

Select by Location Tips

• Make sure Target and Source are correct

• Combine with Select by Attributes– Check “Use Selected Features” under

Source

• Option to apply a search distance when not using the “within a distance of” method

Joining and Relating Tables

• Many datasets are available in tabular format– Excel (.xls, .xlsx), comma-spaced values

(.csv), text

• Tables can be imported to ArcMap and linked points, lines, or polygons using a common ID

Joining Tables

• Tables downloaded as text or CSV may need to be opened and saved as Excel files first

• First row of table should contain short headers with no special characters (or spaces, ideally)

• Table must have an ID that matches geography

One-to-one relationships

• A one-to-one relationship means that each record in one table has only one matching record in another table

Many-to-one relationships

• Many-to-one means multiple records in the table match to one record in another table

Joining Tables

• Usually you will choose to “Keep All Records”

• Always Validate Join– Maybe a mismatched ID– Sometimes missing

records in the join table• Joined fields will display

in the target layer table

Relating Tables

• Relates are used when tables have a one-to-many or many-to-many relationship

• Attributes are not appended to the table, but selecting a record in one table will select all related records in another table

More on Joins and Relates

• Join field must be same format (number / text)

• To remove a join or relate, right-click the target layer again and choose the join or relate to remove, or Remove All Joins/Relates

• To preserve the attributes joined to a layer, you should export it to a new file

Homework & Lab

• Read Appendix and Ch. 5 – Q’s 3,4,6, 8, 10-11

• Lab this week: Selecting and Joining Data– Chapters 8, 9, and 10 in the lab book

• Please submit last week’s homework