TESLA: A Tool for Annotating Geospatial Language Corpora

Nate Blaylock and Bradley Swain and James AllenInstitute for Human and Machine Cognition (IHMC)

Pensacola, Florida, USA{blaylock,bswain,jallen}@ihmc.us

Abstract

In this paper, we present The gEoSpatialLanguage Annotator (TESLA)—a tool whichsupports human annotation of geospatial lan-guage corpora. TESLA interfaces with a GISdatabase for annotating grounded geospatialentities and uses Google Earth for visualiza-tion of both entity search results and evolvingobject and speaker position from GPS tracks.We also discuss a current annotation effort us-ing TESLA to annotate location descriptionsin a geospatial language corpus.

1 Introduction

We are interested in geospatial language under-standing— the understanding of natural language(NL) descriptions of spatial locations, orientation,movement and paths that are grounded in the realworld. Such algorithms would enable a number ofapplications, including automated geotagging of textand speech, robots that can follow human route in-structions, and NL-description based localization.

To aide development of training and testing cor-pora for this area, we have built The gEoSpa-tial Language Annotator (TESLA)—a tool whichsupports the visualization and hand-annotation ofboth text and speech-based geospatial language cor-pora. TESLA can be used to create a gold-standardfor training and testing geospatial language under-standing algorithms by allowing the user to anno-tate geospatial references with object (e.g., streets,businesses, and parks) and latitude and longitude(lat/lon) coordinates. An integrated search capa-bility to a GIS database with results presented inGoogle Earth allow the human annotator to eas-ily annotate geospatial references with ground truth.

Figure 1: A session in the PURSUIT Corpus

Furthermore, TESLA supports the playback of GPStracks of multiple objects for corpora associatedwith synchronized speaker or object movement, al-lowing the annotator to take this positional contextinto account. TESLA is currently being used to an-notate a corpus of first-person, spoken path descrip-tions of car routes.

In this paper, we first briefly describe the corpusthat we are annotating, which provides a groundedexample of using TESLA. We then discuss theTESLA annotation tool and its use in annotating thatcorpus. Finally, we describe related work and ourplans for future work.

2 The PURSUIT Corpus

The PURSUIT Corpus (Blaylock and Allen, 2008)is a collection of speech data in which subjects de-scribe their path in real time (i.e., while they are trav-eling it) and a GPS receiver simultaneously recordsthe actual paths taken. (These GPS tracks of theactual path can aide the annotator in determiningwhat geospatial entities and events were meant bythe speaker’s description.)

Figure 1 shows an example of the experimentalsetup for the corpus collection. Each session con-sisted of a lead car and a follow car. The driver of the

Figure 2: The TESLA annotation and visualization windows

lead car was instructed to drive wherever he wantedfor an approximate amount of time (around 15 min-utes). The driver of the follow car was instructed tofollow the lead car. One person in the lead car (usu-ally a passenger) and one person in the follow car(usually the driver) were given close-speaking head-set microphones and instructed to describe, duringthe ride, where the lead car was going, as if theywere speaking to someone in a remote location whowas trying to follow the car on a map. The speak-ers were also instructed to try to be verbose, andthat they did not need to restrict themselves to streetnames—they could use businesses, landmarks, orwhatever was natural. Both speakers’ speech wasrecorded during the session. In addition, a GPS re-ceiver was placed in each car and the GPS track wasrecorded at a high sampling rate. The corpus con-sists of 13 audio recordings1 of seven paths alongwith the corresponding GPS tracks. The averagesession length was 19 minutes.

3 TESLA

TESLA is an extensible tool for geospatial languageannotation and visualization. It is built on the NXTToolkit (Carletta et al., 2003) and data model (Car-letta et al., 2005) and uses Google Earth for visu-alization. It supports geospatial entity search usingthe TerraFly GIS database (Rishe et al., 2005). Cur-rently, TESLA supports annotation of geospatial lo-cation referring expressions, but is designed to beeasily extended to other annotation tasks for geospa-

1In one session, there was no speaker in the lead car.

tial language corpora. (Our plans for extensions aredescribed in Section 6.)

Figure 2 shows a screenshot of the main viewin the TESLA annotator, showing a session of thePURSUIT Corpus. In the top-left corner is a wid-get with playback controls for the session. This pro-vides synchronized playback of the speech and GPStracks. When the session is playing, audio from asingle speaker (lead or follow) is played back, andthe blue car icon in the Google Earth window on theright moves in synchronized fashion. Although thisGoogle Earth playback is somewhat analogous to avideo of the movement, Google Earth remains us-able and the user can move the display or zoom inand out as desired. If location annotations have pre-viously been made, these pop up at the given lat/lonas they are mentioned in the audio, allowing the an-notator to verify that the location has been correctlyannotated. In the center, on the left-hand side is adisplay of the audio transcription, which also movesin sync with the audio and Google Earth visualiza-tion. The user creates an annotation by highlightinga group of words, and choosing the appropriate typeof annotation. The currently selected annotation ap-pears to the right where the corresponding geospatialentity information (e.g., name, address, lat/lon) canbe entered by hand, or by searching for the entity ina GIS database.

3.1 GIS Search and VisualizationIn addition to allowing information on annotatedgeospatial entities to be entered by hand, TESLAalso supports search with a GIS database. Cur-

Figure 3: Search results display in TESLA

rently, TESLA supports search queries to the Ter-raFly database (Rishe et al., 2005), although otherdatabases could be easily added. TerraFly containsa large aggregation of GIS data from major distrib-utors including NavTeq and Tiger streets and roads,12 million U.S. Businesses through Yellow Pages,and other various freely available geospatial data.It supports keyword searches on database fields aswell as radius-bounded searches from a given point.TESLA, by default, uses the position of the GPStrack of the car at the time of the utterance as thecenter for search queries, although any point can bechosen.

Search results are shown to the user in GoogleEarth as illustrated in Figure 3. This figure showsthe result of searching for intersections with the key-word “Romana”. The annotator can then select oneof the search results, which will automatically pop-ulate the geospatial entity information for that an-notation. Such visualization is important in geospa-tial language annotation, as it allows the annotatorto verify that the correct entity is chosen.

4 Annotation of the PURSUIT Corpus

To illustrate the use of TESLA, we briefly describeour current annotation efforts on the PURSUIT Cor-pus. We are currently involved in annotating refer-ring expressions to locations in the corpus, althoughlater work will involve annotating movement andorientation descriptions as well.

Location references can occur in a number of syn-tactic forms, including proper nouns (Waffle House),

definite (the street) and indefinite (a park) refer-ences, and often, complex noun phrases (one of thehistoric churches of Pensacola). Regardless of itssyntactic form, we annotate all references to loca-tions in the corpus that correspond to types foundin our GIS database. References to such things asfields, parking lots, and fire hydrants are not anno-tated, as our database does not contain these typesof entities. (Although, with access to certain localgovernment resources or advanced computer visionsystems, these references could be resolved as well.)In PURSUIT, we markup the entire noun phrase (asopposed to e.g., the head word) and annotate thatgrouping.

Rather than annotate a location reference with justlatitude and longitude coordinates, we annotate itwith the geospatial entity being referred to, suchas a street or a business. The reasons for this aretwofold: first, lat/lon coordinates are real numbers,and it would be difficult to guarantee that each ref-erence to the same entity was marked with the samecoordinates (e.g., to identify coreference). Secondly,targeting the entity allows us to include more infor-mation about that entity (as detailed below).

In the corpus, we have found four types of en-tities that are references, which are also in ourdatabase: streets, intersections, addresses (e.g., 127Main Street), and other points (a catch-all categorycontaining other point-like entities such as busi-nesses, parks, bridges, etc.)

An annotation example is shown in Figure 4,in which the utterance contains references to two

Figure 4: Sample annotations of referring expressions togeospatial locations

streets and an intersection. Here the intersection re-ferring expression spans two referring expressions tostreets, and each is annotated with a canonical nameas well as lat/lon coordinates. Note also that ourannotation schema allows us to annotate embeddedreferences (here the streets within the intersection).

5 Related Work

The SpatialML module for the Callisto annotator(Mani et al., 2008) was designed for human anno-tation of geospatial locations with ground truth bylooking up targets in a gazetteer. It does not, how-ever, have a geographic visualization componentssuch as Google Earth and does not support GPStrack playback.

The TAME annotator (Leidner, 2004) is a simi-lar tool, supporting hand annotation of toponym ref-erences by gazetteer lookup. It too does not, asfar as we are aware, have a visualization compo-nent nor GPS track information, likely because thelevel of geospatial entities being looked at were atthe city/state/country level. The PURSUIT Corpusmostly contains references to geospatial entities ata sub-city level, which may introduce more uncer-tainty as to the intended referent.

6 Conclusion and Future Work

In this paper, we have presented TESLA—a gen-eral human annotation tool for geospatial language.TESLA uses a GIS database, GPS tracks, andGoogle Earth to allow a user to annotate refer-ences to geospatial entities. We also discussed howTESLA is being used to annotate a corpus of spokenpath descriptions.

Though currently we are only annotating PUR-SUIT with location references, future plans in-

clude extending TESLA to support the annotationof movement, orientation, and path descriptions. Wealso plan to use this corpus as test and training datafor algorithms to automatically annotate such infor-mation.

Finally, the path descriptions in the PURSUITCorpus were all done from a first-person, ground-level perspective. As TESLA allows us to replay theactual routes from GPS tracks within Google Earth,we believe we could use this tool to gather more spo-ken descriptions of the paths from an aerial perspec-tive from different subjects. This would give us sev-eral more versions of descriptions of the same pathand allow the comparison of descriptions from thetwo different perspectives.

ReferencesNate Blaylock and James Allen. 2008. Real-time path

descriptions grounded with gps tracks: a preliminaryreport. In LREC Workshop on Methodologies and Re-sources for Processing Spatial Language, pages 25–27, Marrakech, Morocco, May 31.

Jean Carletta, Stefan Evert, Ulrich Heid, Jonathan Kil-gour, Judy Robertson, and Holger Voormann. 2003.The NITE XML toolkit: flexible annotation for multi-modal language data. Behavior Research Methods, In-struments, and Computers, 35(3):353–363.

Jean Carletta, Stefan Evert, Ulrich Heid, and JonathanKilgour. 2005. The NITE XML toolkit: data modeland query language. Language Resources and Evalu-ation Journal, 39(4):313–334.

Jochen L. Leidner. 2004. Towards a reference corpusfor automatic toponym resolution evaluation. In Work-shop on Geographic Information Retrieval, Sheffield,UK.

Inderjeet Mani, Janet Hitzeman, Justin Richer, Dave Har-ris, Rob Quimby, and Ben Wellner. 2008. SpatialML:Annotation scheme, corpora, and tools. In 6th Interna-tional Conference on Language Resources and Evalu-ation (LREC 2008), Marrakech, Morocco, May.

N. Rishe, M. Gutierrez, A. Selivonenko, and S. Graham.2005. TerraFly: A tool for visualizing and dispensinggeospatial data. Imaging Notes, 20(2):22–23.