Upload
martin-mcdaniel
View
218
Download
1
Tags:
Embed Size (px)
Citation preview
1
GIScience and the Big Data Age
Yihong Yuan
Department of Geography
Texas State University
2
About me
• Yihong YuanAssistant Professor
[email protected]. ELA 366, 512-245-3208
• Research Interests– Spatio-temporal data mining– Human mobility and activity patterns– Big data analytics
3
Geography and Big Data
• GIS– Not only about mapping functions
• Big Geo-data– Information and communication technologies
(ICTs)• Greater mobility flexibility• A wide range of spatio-temporal data sources• Align marketing campaigns to spatial patterns.
4
• “Geography is one of the most natural, logical and intuitive ways to discover, visualize, overlay, compare, slice, sort and apply big data to a problem”
• “GIS used to be about the analysis of relatively static
institutional data, but new data streams mean that today’s GIS problems look very much the same as today’s big data problems: extract meaningful information from a fire hose of inputs”
5
• Traditional geographic knowledge discovery– e.g., high resolution trajectories
• Incomplete Spatio-temporal datasets– Low resolution– Few individual attributes– Uncertainty?
6
Past Research• Georeferenced mobile phone data analytics
– Individual-oriented research– Activity space
» Measurements: Radius, Eccentricity, entropy» Correlation between phone usage and activity space
– Trajectory and sequence patterns» Time series analysis
– Urban-oriented studies• Spatial clusters • Spatial rhythms
• Dynamic clustering• Functional time points
7
UML Model about Geo-referenced mobile phone data
Knowledge Discovery Tasks
Generalize types of information in mobile
phone datasets
Urban-oriented research
Construct an UML model
Analyze individual activity space
Individual-oriented research
Measure trajectory similarity
Correlate activity space with phone usage
Correlate activity space with individual and
supra-individual attributes
Identify urban hotspots and clusters
Dynamic clustering and time series analysis
Extract functional time points
8
• Mobile Phone Connections in 10 cities in northeast China– Time, Duration, and Locations of Mobile
Phone Connections in 9 days– Age and Gender Attributes of the Users– Possibility of simulated data
Example Mobile Phone Dataset
9
Analysis of Activity space
• Three measurements– Radius -> Scale
• eigenvectors of trajectories
– Eccentricity -> Shape• Range [0,1]• Closer to a straight line or a circle
– Entropy->Regularity• How random the visiting patterns are
2 2
1
ˆ 1-( )
ˆ
ee
e
2e1e
21
- logN
i ii
E p p
10
Correlation between individual activity space and phone usage
11
Results
• For People with Higher Mobile Phone Usage: – Larger Activity Space– Trajectories are Closer to a Circle– Movement is More Random, Less
Predictable
12
Activity space vs Trajectory
13
Analysis of trajectory patterns
• Compare trajectories from phone records– Sequences of cell IDs
• Edit distance Method– String matching and auto-correction
14
Analysis of trajectory patterns (Cont.)
• Applications– Identify similar users
• Clustering analysis
– Identify outlier users
15
Knowledge Discovery Tasks
Generalize types of information in mobile
phone datasets
Urban-oriented research
Construct an UML model
Analyze individual activity space
Individual-oriented research
Measure trajectory similarity
Correlate activity space with phone usage
Correlate activity space with individual and
supra-individual attributes
Identify urban hotspots and clusters
Dynamic clustering and time series analysis
Extract functional time points
16
• The changing clustering of urban area
Urban hotspots and clusters
Weekdays Weekends
T2: 2pm-3pm
T1:8am-9am
T3: 7pm-8pm
17
• Mobility patterns of different population groups– Weekday 2pm-3pm
Urban clusters (Cont.)
Age: 12-17 Age: > 60
18
• Provide input for urban infrastructure planning– Are public facilities where people are??
Urban clusters (Cont.)
Age: > 60
A park
19
Dynamic Clustering
• Focus on “rhythms” instead of just “clusters”
• Various mobility patterns in urban area– How to explore? – time series analysis
CBD, Beijing Suburb, Beijing
20
Dynamic Clustering (Cont.)
• Methods– Divide study area
• Voronoi polygon (based on towers)• What to compare: 24-hour series for each
polygon based on mobility count
• Outlier detection e.g., traffic congestion
21
Outlier polygons
• 15 outliers for weekdays and 18 for weekends
Weekday Weekends
22
Mobility patterns in outlier areas• Outlier Polygon 238
– Night clubs and other leisure facilities– International trading center
• Outlier Polygon 125– Several community colleges – Not many night clubs, bars, etc.
Polygon 238 Polygon 125
23
Current and future research
24
Setting up functional time in cities
• Standardization of time – Determination of the beginning/end of a day
• The development of ICT– Real-time activity patterns– More flexibility in time management and
activity scheduling• i.e., fixed parking hour policy may not be
applicable in Central business districts
25
Setting up functional time in cities
26
Cross-country comparison for Social Media websites
• Flickr data, 100 million records and geo-tagged photos
• Similarity and dissimilarity of human mobility in various cities– “A tale of many cities”
27
Current and future research• Mobility patterns in
developing and developed countries– China as a focus
• Weibo and Twitter check-in data– Comparison study for
special time period– Holiday patterns
28
Current and future research
• Mass media and Social Media– GDELT dataset
• Geo-tagged news Events from 1970s
– Public relations and interaction between countries
29
(a) (b)
30
Big data and GIS jobs…• Traditional GIS jobs:• GIS Technician/Analyst/consultant• GIS manager/researcher• ……• Where are the positions?• Public sector… NGA, USGS, State and local Gov, DOT,
planning dept.• Private company…Oil&Gas, Mapping companies, Land
management, Utility…• Non-profit agency… Nature Conservancy, International
Crane Foundation• Consulting firms…Surveying, Remote Sensing…
31
Example: Private Sector Jobs
• Mapping Companies• Software Developers• Utilities• Land Development• Non-Profits• Others
32
Job Skills
• Project Management• Technical Support• Report Writing• Public Speaking• Research/Literature review• Programming
33
Software Skills (cont.)
• GIS software packages• ArcGIS, ENVI, GDAL• Mobile & Web Technology
– Silverlight / Flex /HTML / ASP– Android Dev
• Python / C#...• Database: Access, SQL
Server, PostgresSQL
34
Job Postings• Company Website
– ESRI summer internship program• Relevant Employment Websites
– General sites: Monster.com / Indeed.com– Linkedin.com– Glassdoor.com– GIS Jobs Clearinghouse (gjc.org)– GISjobs.com & Geojobs.org– GeoCommunity – GIS Café – WI State Cartographers Office
• http://www.sco.wisc.edu/jobs/jobs.php
35
Job Postings
• Internal Company Postings• Company Website• Relevant Employment Websites
– GIS Jobs Clearinghouse (gjc.org)– GISjobs.com & Geojobs.org– GeoCommunity – GIS Café – Monster.com
36
Job Postings
• Internal Company Postings• Company Website• Relevant Employment Websites
– GIS Jobs Clearinghouse (gjc.org)– GISjobs.com & Geojobs.org– GeoCommunity – GIS Café – Monster.com
37
Big data jobs…• Spatial data are inherently big data…• For GIS major…
– Data Scientist• This is a more “General” term• Focus on big (geo)data analytics• Highly competitive salary• Graduate degree (MA possible, PhD preferred)• Many opportunities…
• Skill set:• Strong statistical background• Strong and programming: Python, R, etc,
38
Example positions
• Data Scientist @ ESRI– http://www.simplyhired.com/job/data-scientist-agriculture-job/esri/5jjxyxjt4b?cid=n
tvzgigizsvnqhofbuscopqozjkxqugd
• Research Data Scientist– http://www.americasjobexchange.com/job-detail/job-opening-AJE-56966
1132?source=indeed&utm_source=Indeed&utm_medium=cpc&utm_campaign=Indeed
• Other potential groups: Apple geo-group, Twitter geo-group, Facebook data science group
39
Thanks!Questions and Comments?