Big Data Applications from the EU FP-7 iMars and PRoViDE projects: automated DTM and Supoer-resolution restoration from NASA MRO Cameras and in future

from TGO16 CaSSiS

Yu Tao, Panos Sidiropoulos, Jan-Peter Muller Imaging group, Mullard Space Science Laboratory, University College London

BiDS 17 March 2016

An overview • Introduction to the EU FP-7 iMars and PRoViDE project • Introduction to methods (individual processing chains) developed

within the two EU projects: – Automated co-registration of HiRISE, CTX, MOC, THEMIS, HRSC (see poster from

Sidiropoulos & Muller) – Automated DTM production from HiRISE, CTX – Image quality assessment and data mining for change detection (see poster from

Sidiropoulos & Muller) – Stereo panorama processing from MER and MSL rover imagery – Super-resolution restoration from repeat-pass HiRISE images

• Systematic processing framework at MSSL’s imaging clusters. • Interactive Web-GIS interface (for data and products) visualisation and

manipulation • Handling big data at MSSL • Big data challenges and lessons learnt • Conclusions and future processing for ExoMars TGO16 CaSSiS

Page

3

Vers

ion:

30.

03.2

016

j.mul

ler@

ucl.a

c.uk

iMars • iMars (1.1.2014 – 12.31.2016 http://www.i-mars.eu) aims to explore changes on the Martian

surface since the start of robotic exploration using automated data mining techniques and crowd-sourcing from

• HRSC Ortho-rectified Images (ORIs) and Digital Terrain Models (DTMs) as base images and • Automated co-registration of NASA orbital imagery together with higher resolution DTMs

from CTX and HiRISE to these HRSC ORI/DTMs • After capturing scientific consensus (e.g. HRSC team members, EGU) on key features to

detect.

Page

4

Vers

ion:

30.

03.2

016

Ger

hard

.paa

r@jo

anne

um.a

t

www.provide-space.eu

PRoViDE • Planetary Robotics Vision Data Exploitation (PRoViDE 1.1.2013-12.31.2015

http://provide-space.eu) harvest literally all (available) planetary rover surface-captured vision data

• Process all images using 3D machine vision • Bring them into an unique and common set of geographic coordinates • Embed into 3D data base, GIS and visualisation

NASA - JPL

Automatic co-registration of orbital images

• Co-registration: Project all images from a particular dataset to a common projection system (baseline)

• Automated co-registration pipeline – Use HRSC (single strip or mosaic) as a baseline – It is fast, robust and doesn’t need (extensive)

modifications for a different input – Has been tested only on Mars high-resolution data – It shouldn’t be a problem to be transferred to other

types of products (e.g. Moon or Earth Observation images) but it hasn’t been tested yet

• Tested on the MC11-E mosaic – 1,365 CTX, 1,558 MOC and 3,629 THEMIS-VIS images – CTX median accuracy: 6.49m (X axis) 6.08m (Y axis) – MOC median accuracy: 5.33m (X axis), 4.851m (Y axis) – THEMIS median accuracy: 7.01m (X axis), 6.85m (Y axis) – Median processing time per image (single core)

• CTX: 4.5 hours, MOC and THEMIS-VIS: 25-30 minutes – Failure rate: 7.25% (CTX), 11.23% (THEMIS), 34.55% (MOC)

See the BiDS’ 16 poster of Sidiropoulos & Muller

Automated DTM production from HiRISE, CTX

The automated DTM processing chain, is called the Co-registered Ames Stereo Pipeline with Gotcha refinement and Optimization (CASP-GO). In CASP-GO, we have developed a fully automated processing system with improved performance compared to the original ASP system in terms of (a) Co-registered geo-spatial coordinates w.r.t HRSC (and MOLA) data; (b) Improved DTM completeness for unmatched areas (Fig. 2); (c) Reduced DTM artefacts; (d) Improved DTM accuracy; (e) Fully documented uncertainty for every height.

Automatic Planetary Image Quality Assessment • >400,000 Mars orbital high-resolution (≤100m)

images • A large part of the orbital data have “degraded”

visual quality – Degraded visual quality doesn’t necessarily mean of less

scientific interest

• Double Goal 1. Detect images with low visual quality 2. Discriminate between images that are of low-quality due

to acquisition failures (top panel) or due to Mars conditions, such as a dusty atmosphere (bottom panel)

• Deep learning classifier that combines statistical measures (image anisotropy), blur detection, noise detection, etc. to an SVM

• Tested on Viking Orbiter images: – Commission Errors: 6.4%, Omission Errors: 9.6% – From the correctly identified, “dusty image”

detection only • 13% commission error, 16% omission error

Demo: http://www.mssl.ucl.ac.uk/~ps2/

Mars surface change detection

• Fact: Multi-instrument high-resolution pairs of Mars images haven’t been extensively checked for changes.

• Double Goal: 1. Fast and accurate automatic detection

of surface changes 2. Classify detected changes

• Deep learning method which combines – An unsupervised learning algorithm that

models image texture – Parametric algorithm to model noise

and mis-registration errors – An edge detection algorithm – A shadow detection algorithm to

eliminate “changes” due to illumination conditions

• Still under development & testing See the BiDS’ 16 poster of Sidiropoulos & Muller

23-2-12 MY 31 LS 74

15-12-07 MY 29 LS 3

Stereo panorama processing from MER and MSL rover imagery The PRoViDE major building blocks include: (a) PRoDB holding information of source images to be used for 3D reconstruction & panorama generation; (b) PRoViP (the processing component) harvests image data from MSSL PDS mirror, process and stores the results; (c) Scheduler & Task Distribution Daemon runs on scheduling server and processing blades, connects to PRoDB, creates processing jobs and distribute over available server blades, monitors processing status of jobs, and updates database with processing results;

(d) PRoGIS gives access to rover image archives within co-registered geographical context and updated according to processing results, as a means of interacting with and exploring the products; (e) PRo3D provides the ability to directly and interactively explore the co-registered product and perform scientific analysis for geologists.

Super-resolution restoration from repeat-pass HiRISE images A novel super-resolution restoration (SRR) algorithm/pipeline, called Gotcha-PDE-TV (GPT), has been developed to be able to restore higher resolution images from the non-redundant sub-pixel information contained in multiple lower resolution raw remotely sensed images.

25cm HiRISE

25cm HiRISE

5cm SSR

5cm SSR

Automatically detected rocks (labelled green) of 25cm HiRISE image (PSP_001513_1655, top) and 5cm SRR image (bot) around an impact crater close to MER-A traverse at ~ (175.51045º, -14.58461º).

Systematic processing framework at MSSL’s imaging clusters

Example of PRoViDE MER and MSL processing system set-up at MSSL’s imaging cluster to allow external partners access to MSSL resources without threatening the integrity of software and processed data within the MSSL firewall.

Interactive Web-GIS interface for visualisation and manipulation

An interactive web-GIS system, i.e. PRoGIS 1.0 (developed in EU FP-7 PRoVisG), allows co-registered HRSC-CTX-HiRISE-SRR to be viewed with rover images and products for MER and MSL area. PRoGIS 1.0 (http://www.progisweb.eu) is currently hosted in a local desktop at MSSL. Access username: cps_apex Password: Jgkp%4Se PRoGIS 1.0 has been upgraded to PRoGIS 2.0 in PRoViDE project and will be up and running at MSSL soon, to allow analysis of key geological features and make measurement with different levels of detail and jump into a local 3D view using a 3D viewer, called PRo3D® running in a GPU server at MSSL. We plan to integrate more interesting sites to PRoGIS 2.0 for example the future ExoMars rover and CaSSiS orbital data/products.

Thanks to Jeremy Morley, James Sprinks (U of Nottingham) for PRoGIS software development

Handling big data at MSSL

• we have mirrored the HRSC, CTX, HiRISE, as well as MER, MSL EDR and RDR data volumes in a local shared storage system in order to speed up the production process

• with an option such that if data is unreachable it can be read from the original source again.

• On the other hand, developed software is installed in a shared directory, which is accessible from14 Linux processing blades (16 cores for 10 blades with 48GB and 4 with 24 cores and 96GB RAM).

• Jobs are controlled in a local desktop machine and distributed to the 14 processing blades with multiple sessions of multi-threaded processing.

• Processed results are stored in several 1TB RAID storage disks and logged back to the local controlling desktop.

• Failed jobs can be examined through detailed log files and are planned in the future will be reprocessed automatically with different processing parameters.

• Once all jobs are done, the resulting products are fetched by a PostgreSQL database and displayed in an interactive web-GIS system, called PRoGIS 2.0 (thanks to Jeremy Morley & Michele Giordano installed on a map server

• PRoGIS 2.0 is able to call Pro3D®, which is installed in a GPU server • The GPU has a K-2 GRID card with 0.5TB RAM and 24TB local disk space

and supports up to 16 simultaneous users.

Big data challenges and lessons learnt

• To address large dataset size problems in SRR, we use tiling for large HiRISE scenes and mosaic afterwards.

• To address single processing computational complexity, we load balance computationally inexpensive algorithms for initial processing and use more complex methods for result improvement.

• We plan, in the future, to port most of our existing processing chains to GPU server, to further accelerate single session processing time.

• Automated task scheduling system is also important to handle large data volumes. For example, a scheduler/task distribution daemon based system was successfully tested previously for MER and MSL stereo panorama processing.

• The automated scheduling system is able to retrieve and initialize jobs from a catalogue database, monitor the processing status of jobs on server blades and update the database with processing results.

• Moreover, an automated image quality assessment mechanism has been developed at MSSL to distinguish different levels of image quality for HiRISE, CTX and HRSC image in order to improve the job success rate and save processing time from low quality input data.

Conclusions and future processing for ExoMars TGO16 CaSSiS

• For the automated CASP-GO DTM pipeline, we have currently started processing most of the CTX and HiRISE stereo pairs accumulated to on a large linux cluster of 224 cores and are exploring possible cloud computing to be able to process the entire dataset in a sensible time period.

• We also aim to process all available image datasets in future where we have repeat multi-view imagery starting with HiRISE first and then applying these techniques to CTX after porting the GPT SRR software onto a GPU.

• We plan to develop the SRR capability for the ExoMars Trace Gas Orbiter 2016 CaSSiS instrument (from 4m up to ≤1m/pixel) including both 3D and SRR images from multiple overlapping colour stereos.