11/16/17

1

FLARECAST:Learningfromthepasttopredict thefuture

AndréCsillaghyfortheFLARECASTTeam

Imagine…

Whyautomatingsolarflarepredictions?

11/16/17

2

FLARECASTHIGH-LEVELOBJECTIVES• Develop asolarflare predictionsystembasedon

automatically extracted physicalpropertiesofsolaractiveregions,coupledwithstate-of-the-art solarflare predictionmethodsandvalidated usingthemostappropriate forecastverification measures.

• FLARECAST top-level objectives:

– Tounderstand the drivers ofsolarflare activity andimproveflare prediction

– Toprovide agloballyaccessibleflare prediction servicethat facilitates expansion

– Toengage withspace weather endusers andinformpolicymakers andthe public

FLARECASTINSIMPLETERMS

• What isthedata trying totellusaboutflareprediction?

• Input:SDOHMIimages

• Output:aflare predictionofthe kind:– Binary forecasting:FlareorNoFlare

– Probabilistic: 0<p<1

• Forthe followingcharacteristics– Within aflareclass (e.g.M1– M9.9)

– Within aforecast timewindow (e.g.24hours)&aboveathreshold

• Steps:Featurepropertyextractionà prediction learning à forecastverification

• Support:Infrastructure

THEFLARECASTSCENE

• H2020Project2015– 31.12.2017• Partners:AcademyofAthens(Georgoulis,PI),Northumbria U.(BloomfieldProjectScientist),U.Genova(Piana),CNR(Massone),CNRS(Vilmer),U.ParisSud (Buchlin),FHNW(meJ ),MetOffice(Jackson)

• Adiversegroupof~50scientistsandengineersworkingtogether

• Mixofexpertiseinflareprediction(AA,TCD,UN),mathematics(UGE,CNR),ComputerScience(UPS,FHNW),anduserperspective(MetOffice)

11/16/17

3

FLARECASTWORKFLOW

DataLoader HMIFilesFeatureProperty

Extractor

StagingArea

PropertyDB

Predic=onDatabase

Forecastverifica=onalgorithms

Flarepredic=onalgorithms(execu=on)

ManagementInfrastructure

Predic=onConfigDB

Step 1: Data acquisition

Step 3: Prediction training / execution

Step 4: Data verification

read write

Legend

Flarepredic=onalgorithms(training)

Step 2: Feature property extraction

InfraConfigDB

FLARECASTINPUTDATA

SDO/HMIdata

LOSmagnetograms(hmi.M_720s)SHARPvectormagnetograms –

definitive (hmi.sharp_720s)

SHARPvectormagnetograms –NRT

(hmi.sharp_720s_nrt)

SRSactiveregion &flaredata(YYYY_events.tar.gz )

FeatureProperties

Property Extraction AlgorithmsData Source Property Group Developer Status

SWPC catalogues (To do / In progress / Under testing /Delivered)

Solar Region Summary properties TCD Delivered

Details GOES X-ray events TCD Delivered

LOS magnetograms

SMART-derived properties ( )Ahmed et al., 2013 TCD In progress

SMART delta finder ( )Padinhatteeri et al., 2015 TCD To do

Details Effective connected magnetic field strength (B ) (eff Georgoulis & Rust,)2007

AA Delivered

Details Fractal dimension ( )Georgoulis, 2012 AA Delivered

Details Multi-fractal structure function s(q) inertial range index k (Georgoulis,)2012

AA Delivered

Details Fourier power spectral index ( )Guerra et. al., 2015 TCD Delivered

Details CWT power spectral index ( )Hewett et. al., 2008 TCD Delivered

Details Generalised correlation dimension ( )Georgoulis, 2012 AA Delivered

Details Holder exponent h ( )Conlon et al., 2010 AA In progress

Details Hausdorff dimension D(h) ( )Conlon et al., 2010 AA In progress

WTMM ( )Conlon et al., 2010 TCD Under testing (further investigated in WP6)

Details Decay index ( )Zuccarello et al. 2014 TCD Delivered

Details Magnetic polarity inversion line characteristics (Mason & Hoeksema)2010

TCD Delivered

Details 3D magnetic null point ( )Greene 1992 TCD Delivered

Details R ( )Schrijver 2007 TCD Delivered

Details LWL ( ) *SG Falconer et al. 2008 TCD Delivered

Details Ising energy ( )Ahmed et al. 2010 AA Delivered

Details WG and S ( )M l-f Korsos et al. 2015 AA Delivered

Details Magnetic helicity injection rate proxy ( )Park et al. 2013 TCD Delivered

Vectormagnetograms

Details SHARP properties ( )Bobra et al. 2014 TCD Delivered

Details Magnetic helicity injection rate ( )Berger & Field 1984 TCD Delivered

Details Magnetic energy injection rate ( )Kusano et al. 2002 TCD Delivered

Details Non-neutralized currents ( )Georgoulis et al., 2012 AA Delivered

Details Flow field characteristics ( ; )Deng et al. 2006 Wang et al. 2014 TCD Delivered

Magnetic bipolar feature characteristics TCD Under testing (further investigated in WP6)

Intensity images

Flow field scharacteristic TCD Under testing (further investigated in WP6)

Mixofinformationalreadyavailableandnewlygenerated

Link

11/16/17

4

PREDICTIONALGORITHMS

• NonML– LinearDiscriminantAnalysis– BayesianQuantileRegression

• StandardML– ClusteringandRegressionAnalysis– SimpleRecurrentNeuralNetworks

• AdvancedML– Multi-LayerPerceptron– Possibilistic C-Means

• InnovativeML– Multi-TaskLasso– PoissonRe-WeightedMultiTaskLasso– HybridMethod– SimulatedAnnealing– Reccurent NeuralNetworktrainedwithanevolutionaryalgorithm– RandomForest

Link

FORECASTVALIDATIONviastatisticalassessment

ForecastedFlare ForecastedNo-Flare

ObservedFlare True Positive False Negative

ObservedNo-Flare False Positive TrueNegative

Probabilityofdetection:POD=!"

!"#$%FalseAlarmRateFAR=

$"$"# !%

AccuracyACC= !"#!% !"#!%#$" #$%

Heidke SkillScore:HSS=' () #(* +*

*TrueSkillStatistic(Bloomfield,2012):TSS=POD-FAR

PreliminaryanalysisofHMIdata• ConsiderthefollowingpropertiesextractedwithSMART(Higginsetal2011,

Ahmedetal2013):– Minimum valueof the magnetic fieldinthe region

– Maximum valueofthe magnetic field inthe region

– Variance valueof the magnetic fieldwithin the region

– Sum ofthe magnetic field values withinthe region

– Skewness valueofthe magnetic field within the region

– Kurtos is valueof themagnetic fieldwithin the region

– Area of theregion

– Totalpos itive fluxin theregion

– Fluximbalance fraction in theregion

– Totalnegative flux inthe region

– Totaluns igned magnetic flux ofthe region

– Neutral-line length inthe region

– Falconer WLSG value

– Schrijver Rvalue (with alowerthreshold).

• TrainingData:8August2011 areusedtotrainthemodels (2,623samples)• TestData:first12hr of9August92011.(1,474samples)

11/16/17

5

SUPPORT:DOCKERCONTAINERS

Linux

DockerEngine

hmi_service

property_service

predic8on_service

db_service

algorithmalgorithmalgorithm

Management Infrastructure Data Components Algorithms

webapp luigi_service

workflow_manager_service

Theevolution of science communication

• PublicUnderstandingofScience• DialogwiththePublic

• PublicEngagementinScienceà• PublicParticipationinScience

• toResponsibleResearchandInnovationRRI.

http://flarecast.eu/outreach-resources

11/16/17

6

Experimentingwithnewcommunicationformats

FURTHERONGOINGWORK:DEEPLEARNING

Thanks totheFLARECASTteam!

Aleksandar Torbica, And re Csillaghy, Anna Maria Massone,Annalisa Perasso, Chloé Guennou, Colin Klauser, CostisGontikakis, Cristin a Campi, D. Shaun Bloomfield, Dario Vischi,David Jac kson, Douglas Biesecker, Eric Buchlin, Etienne Pariat,Federica Sciacchi tano, Federico Benvenuto, Flavio Mülle r, FraserLott, F rederic Baudin, Grah am Barnes, Hann a Sathi apal, IoannisKontogiannis, Jonas Lüthi, Jordan Guerra, Kostas Florios, ManolisGeorgoulis, Manuel Ramirez Lopez, Marco Soldati, MarkWorsfold, Michele Piana, Neal hurlburt, Nicole Vilmer, PabloAlingeri, Pascal Demoulin, Pedro Russo, Peter Gallaghe r, RomanBolzern, Sabrin a Gu astavino, Samuel von Stachelski, Silvi a Villa,Sophie masson, Sophie Murray, Stefan Mülle r, Sung-Hon g Park,Vangelis Argoudelis, Vittorio Latorre, Véronique Bommier