Upload
aaron
View
30
Download
0
Tags:
Embed Size (px)
DESCRIPTION
NIR Transient Surveys. Nicholas Cross WFAU, Edinburgh Nigel Hambly , Mike Read, Ross Collins, Eckhard Sutorius , Rob Blake, Mark Holliman. NIR Variability Science Drivers. NIR, smaller detectors, higher backgrounds and more expensive detectors than optical - PowerPoint PPT Presentation
Citation preview
NIR Transient Surveys
Nicholas CrossWFAU, Edinburgh
Nigel Hambly, Mike Read, Ross Collins, Eckhard Sutorius, Rob Blake, Mark Holliman
IAU S285, Oxford 2
NIR Variability Science Drivers
• NIR, smaller detectors, higher backgrounds and more expensive detectors than optical– Only do multi-epoch work where it is not practical for
optical detectors– Looking through the dense dusty regions of the MW
to the far side– Young Stellar Objects in star-forming regions– Low mass stars / brown dwarfs– High z galaxies / Snae– Can get better RR Lyrae / Cepheid distances in NIR
19/09/2011
IAU S285, Oxford 3
NIR Variability Surveys
• UKIRT WFCAM – UKIDSS – DXS/UDS (Deep surveys, multi-epoch), – WFCAM Transit Survey,– Calibration/Standard Stars, – Surveys of YSOs in Orion/Ophiuchus
• VISTA– VISTA Variables in Via-Lactea (VVV), (RR Lyrae,
Cepheids)– VISTA Magellanic Cloud (VMC), (RR Lyrae, Cepheids)– VIDEO (Deep Extragalactic – SNae)
19/09/2011
IAU S285, Oxford 4
WFCAM• 3.8 m UKIRT telescope
on Mauna Kea.• 4 2k x 2k Rockwell
Hawaii 2 detectors.• Spaced 94% apart. • 0.4” pixels.• 13.65’ across each side.• 60% of time on UKIRT in
2005b• 100% for 2009a
19/09/2011
IAU S285, Oxford 5
VIRCAM
19/09/2011
• 4.1m VISTA telescope at Cerro Paranel.• 16 2k x 2k Raytheon VIRGO detectors• Spaced 90% in x and 42.5% in y.• 0.34” pixels• Tile is 1.5° • VIRCAM has 100% of time. • > 3 times area WFCAM• 2 * QE
IAU S285, Oxford 6
VISTA Public Surveys
19/09/2011
IAU S285, Oxford 7
VISTA Variables in Via-Lactea (VVV)
• Very high density ~106 sources / sq. deg.– Issues with
deblending• 500 sq. deg• ~100 epochs
(currently ~10)• ~ few 1010 detections
19/09/2011
IAU S285, Oxford 8
Processing of WFCAM and VISTA data
• VDFS: VISTA Data Flow System (System for processing of UKIRT WFCAM and VISTA data.– CASU (Cambridge): Data reduction, processing of
observing blocks, photometric and astrometric calibration
– WFAU (Edinburgh): Archive, processing of multiple observing blocks – deep stacks, multi-band tables, links to external tables, MULTI-EPOCH.
– For VISTA, data goes through ESO and final products go to ESO too.
19/09/2011
IAU S285, Oxford 9
Constraints from VDFS• >=6 week time lag before data at WFAU– Data needs to be transferred to Cambridge (with VISTA
this includes disk drive to Garching and then to Cambridge)
– Accurate photometric calibration (including scattered light corrections uses 1 month of data.
– VoEvent alerts are too late from WFAU• Reprocessing of OB data requires retransfer between
CASU and WFAU and reingest of data at WFAU. – Detection tables are used by many curation processes –
reingestion into these slows later stages.
19/09/2011
IAU S285, Oxford 10
Stages of multi-epoch processing
• Stack epochs to create deep images and extract catalogues
• Create master list (Source table) from band-merged catalogues from deep images.
• Recalibrate each epoch image compared to the deep image in that filter and pointing.
• Create table linking sources to each observation • Calculate the noise properties of each pointing and filter• Calculate astrometric and photometric statistics for
each source.
19/09/2011
IAU S285, Oxford 11
Analysing Variables
• Calculate mean, rms of magnitudes.
• Bin in magnitude and calculate clipped median
• Fit empirical noise model • (m)=a+b10-0.4m+c10-0.8m
• Classify as variable or non-variable
19/09/2011
IAU S285, Oxford 12
Archival Databases• Curation of WFCAM and VISTA data occurs in a
RDBMS using Microsoft SQL Server. – Dynamic database, updated with new data,
improved calibrations and reprocessed data when necessary.
– Static releases to the science teams and world for science purposes.
• Curation controlled by comparing current state of DB with requirements
19/09/2011
IAU S285, Oxford 13
Programme Requirements• Pointing, filter and table requirements are setup by
grouping the metadata and using specifications for each survey.
• Schema updated if necessary• Stack / tile products made for a particular release
number• Source table created for particular pointings• Each stage of multi-epoch processing checks the
whether the previous table has changed in that pointing – higher curation event ID.
19/09/2011
IAU S285, Oxford 14
VISTA tiles• Most surveys require tiles to reach expected depth, and
tiles are standard ESO product.• PSF and sky vary on short time scales < integration time• Images filtered to remove large spatial variations (>30”)• Tile catalogues are inferior to pawprints:
– Not as accurate astrometry– Do not deal with saturation correctly– Extended (>30”) sources are missing or have incorrect
photometry• Catalogues from tiles and pawprints
– Need to be able to compare – multiple layers and linking tables.
19/09/2011
IAU S285, Oxford 15
Problems / bottlenecks /solutions• Reprocessing of OB data.
– 1st year of VISTA – 2 sets of full reprocessing• Ingesting new data while curating later products
– Put VVV on separate server and synchronise metadata tables• BUT foreign key constraints to vvvDetection cause major holdups if
metadata is deleted.
– Split vvvDetection into semesters / months so new data can be ingested into new semester.• Has not been implemented yet
• Users want to use both tile and pawprint detections– Produce linking tables
• BUT some queries that join these can join several tens of tables and SQL does not handle these joins well.
• Enhancements to user interface allow users to save intermediate results
19/09/2011
IAU S285, Oxford 16
Problems / bottlenecks /solutions• Checking non-detections of sources
– Using half-space method of Budavari, major improvement• Dealing with very long processing times of VVV
– Break curation into chunks with software testing to see what has already been done
– Make sure memory never exceeds ~40% – BUT this adds additional overheads at beginning of each run
• Variability table curation is dominated by DB reads (85% for VVV)– Use Query Analyser and other tools to optimise queries [OPTION
(MAXDOP 1)], adding removing indexes.– Split detection tables into parts?
• I/O limited between servers and disks– SQL Server “cluster” linked by infiniband 10Gbs-1
19/09/2011
IAU S285, Oxford 17
Other issues• Classification
– DB has simple classification (variable or not) and some other statistical quantities. VVV will have ~106 variables
– Chilean teams working on NIR templates for different types of variables
– Trend analysis (Istvan Dekany)• Accuracy
– VSA/WSA, simple ZP recalibration – rms ~0.005mag• Good enough for most variables• Planetary Transits require (prefer) ~0.001 mag.
• Confusion– Difference Imaging Analysis (Eamonn Kerins), will probably be
applied to densest 40 sq. deg of VVV bulge.
19/09/2011