A Simple Guide to NEOWISE Problems final€¦ · see the star wink out for a short period. If you time the event accurately, you can use the duration of the wink to estimate the size

ASimpleGuidetoNEOWISEDataProblems

NathanMyhrvoldMay25,2016

InapaperrecentlysubmittedtoascientificjournalandpostedasapreprintonarXiv,Ipointedoutanumberofweaknessesinresultspreviouslypublished–andwidelycited–fromtheNEOWISEasteroidmission.The110-pagemanuscriptisfullofdetailedargumentsaboutstatisticsandthephysicsofhowasteroidsabsorbandreflectsunlightandthenemitinfraredlight.It’sallprettycomplicatedstuff,writtenforanaudienceofprofessionalastronomers,physicists,andstatisticians.

ButthemostegregiousproblemswiththeNEOWISEprojectaredeadsimpletoexplain;indeed,thismightbeagoodprojectforamiddle-schoolscienceclass.Inthisguide,Ishowhowanyonecanexaminethedatathemselvesandspotthemostcriticalerrors.It’sagreatexampleofhowscientificdebatesthatseemcomplicatedsometimescomedowntoreallysimpleissues.

BackgroundLet’sbeginwithalittlebackground.NEOWISEisaNASA-fundedprojectthatspentmillionsoftaxpayerdollarstoestimatethediameterofmorethan150,000asteroids,aswellasotherphysicalpropertiesoftheobjects.Itdidthisbyanalyzingobservationsoftheinfrared(IR)lightgivenoffbytheasteroidsandpickedupbytheWISEsatellite.WISEisaspacetelescopethatisabitlikeasupersizedandmoresophisticatedversionofthethermalcamerasusedtospotheatleaksinabuilding.

TheNEOWISEprojectusedatechniquecalledthermalmodelingtoconvertthebrightnessofeachasteroidatseveraldifferentinfrared“colors”(wavelengthbands)intoanestimateoftheobject’sdiameter.ThemostwidelyusedthermalmodelisNEATM(near-Earthasteroidthermalmodel),whichdatesfromthe1990sandestimatestheinfraredemissionduetothewarmthoftheasteroiditself.TheNEOWISEteamofastronomersmodifiedtheNEATMmodeltoaddinfraredemissionsduetosunlightreflectedbytheasteroid.AccountingforreflectedsunlightismoreimportantwhenanalyzingdatafromtheWISEspacetelescopethanithasbeenforprevioustelescopes.

Inadditiontoestimatingdiameter,theNEOWISEgroupalsoestimatedotherasteroidparameters,suchasobjects’albedo,bothinvisiblelightandintheinfrared.(Albedoisascientifictermforreflectivity–howmuchofthesun’slightreflectsfromtheasteroid.)Butthemostimportantparameterisdiameterbecauseiteffectsalloftheothers.

ASimpleGuidetoNEOWISEProblems 2

Inaseriesofscientificpapers,theNEOWISEteampublishedthediameterandotherparametersforabout158,000asteroids.That’sahugecontribution.Forcomparison,thelargestpreviousstudyprovideddataonabout2,200asteroids.Thelargescopeofthesestudiesmadethemhugelyinfluential.Andthedatasetshavebeenusedbymanyotherasteroidresearchers.ThatiswhyitisreallyimportanttogettheNEOWISEdataright.

Thereareatleastthreeotherwaysthanthermalmodelingtoestimatethediameterofanasteroid.Oneistouseradarfromlargeradiotelescopestobouncesignalsofftheasteroid.It’sagreattechnique,butunfortunatelyitworksforrelativelyfewasteroids.Asecondmethodiscalledstellaroccultation,whichmeansblockedstarlight.Everynowandthen(lessoftenthanonemaythink),anasteroidwillmoveinfrontofabrightstar,andobserversonEarthseethestarwinkoutforashortperiod.Ifyoutimetheeventaccurately,youcanusethedurationofthewinktoestimatethesizeoftheasteroid.Athirdandevenmoreaccuratemethodistovisittheasteroid.Somespaceprobeshaveflowncloseenoughtoasteroidstogetnicepicturesfromwhichwecanmeasuretheirdiameters.Recently,forexample,NASAobtainedwonderfulpicturesofCeres,oneofthelargestasteroids.Ireferfrequentlyinthisguidetoradar,occultation,andspacecraftanalyses,whichIlumptogetherwiththeacronymROS.

Unfortunately,ROSestimatesareavailableforrelativelyfewasteroids.Ofthe158,000asteroidsstudiedbyNEOWISE,ROSestimateshavebeenpublishedforonlyabout150–soroughlyoneasteroidinathousand.Asaresult,themainvalueofROSestimatesisthattheyletuscheckwhethertheestimatesproducedbythermalmodelsareaccurateornot.

Theideaissimple.IfwecangainconfidencethatthermalmodelingworkswellontheROSasteroids,thenwecanfeelconfidentaboutourunderstandingofthevastlymorenumerousasteroidsforwhichallwehavearediametersestimatedfromthermalmodels.

TheProblemOnebigproblemmynewpaperidentifieswithcertainpreviousNEOWISEstudiesisthattheydidn’tjustuseROSdiametersasdescribedabove–asawaytovalidatetheirthermalmodels.InsteadtheypresentedROSdiametersasestimatesproducedbytheirthermalmodels.Thediameterswereexactlycopied.

Thefirstpapertoconsideris(Masieroetal.,2011),whichhasthetitle“MainBeltAsteroidswithWISE/NEOWISE.I.PreliminaryAlbedosandDiameters.”Fortunately,TheAstrophysicalJournal,in


whichtheNEOWISEgrouppublisheditsresults,isopenaccess,soyoucandownloadthepaperhere:http://iopscience.iop.org/article/10.1088/0004-637X/741/2/68.

Theabstractsumsupthepurposeofthispaperprettywell:“UsingaNEATMthermalmodelfittingroutine,wecomputediametersforover100,000MainBeltasteroidsfromtheirIRthermalflux,witherrorsbetterthan10%.”

Table1inthepaperpresentsanexcerptofthemostimportantresults,presentedat“ThermalModelFits.”Thisfulltableincludesmorethan100,000entriesandcanbedownloadedfromhttp://iopscience.iop.org/0004-637X/741/2/68/suppdata/apj398969t1_mrt.txtordirectlyavailablefromCaltech/JPLathttp://wise2.ipac.caltech.edu/staff/bauer/NEOWISE_pass1/.

HereisasnapshotofTable1,whereIhaveaddedsomecoloredgraphicstomakeiteasiertoseetherelevantpartofthetable.The“Object”columnsliststheidentifyingnumberforeachasteroid,andtheDcolumnliststhediametergeneratedbytheirmodel–orsotheyclaim.

Ihavedrawnaredandgreenboxesaroundsomeoftheasteroidentries.Intheredboxes,notethatthediameters,whichareinunitsofkilometers,allendin“.000.”Incontrast,thediametersforasteroidshighlightedbythegreenboxincludenon–zerodigitsallthewayouttothenearest0.001km,whichisthenearestmeter.

Notealsothatsomeasteroids(suchasasteroids00002and00009)appearonmultiplerows.ThatisbecausetheNEOWISEteambrokeupthedatainanon-standardwayandperformedseparatecurvefitson3-dayto10-daysegmentsratherthansimplyfittingallofthedataavailable.Itdoesn’tmakeanysensetodothat,butthat’saseparateandmorecomplicatedissuethatIdiscussinmypaper.


Hereistheprobleminanutshell.Thediametersboxedinredabove–aswellasmorethan100othersnotshownhere–areexactlyequal,tothenearestmeter,toROSdiameterspublishedinpaperswellbeforetheNEOWISEstudies.

SomeofthesupposedNEOWISEresultsaren’tNEOWISEresultsatall–theyweredirectlycopiedfromtheworkofothers.

Youcancheckthisyourself.Asteroid2(alsoknownasPallasor00002)hasadiameterof544.000kminthetableabovefromMasieroetal.,2011.CompareittotheentryforthatasteroidinShevchenkoandTedesco,2006,oneoftheROSpapers,whichisavailableathttp://www.sciencedirect.com/science/article/pii/S001910350600128X.

InTable1oftheShevchenkopaperwefindthis:

Here,column𝐷"##givesthediameterdeterminedbytheoccultationmethod.ItmatchestheMasierovalueforasteroid2Pallas(aka00002)exactly.TurningtoanotherROSsource,Durechetal.2011,availableathttp://www.sciencedirect.com/science/article/pii/S0019103511001072orhttp://arxiv.org/abs/1104.4227,wefindthefollowingtable.

Checkoutthevalueforthediameterofasteroid5Astraea(aka00005):itis115km,exactlyequaltothediameterlistedintheMasieroTable1above.Youmayalsonoticethatasteroid2Pallasisalsointhistable,withadifferentdiameterthanintheShevchenkoorMasierotables.Thatistypical;onealmostnevergetsthesamenumberfromdifferentROSstudies,mainlyasaresultofmeasurementerror.

Youcanalsoseethatasteroid00002intheMasierotablematchesdiameterreportedbytheShevchenkoandTedescoratherthantheestimategivenbyDurechetal.ItisnotclearwhytheNEOWISE


authorscopiedfromonesourceversusanotherineachcase.Butitisclearthatthesenumberswerecopied,notcomputedasclaimed.

TheexcerptfromMasieroTable1includessixexamplesofcopyingforfiveasteroids.Butifyoucomparethediametersgiveninthecompletetableofmorethan100,000asteroidstothoseintheROSreferences(seelinksbelow),youwillfind,asIdid,exactmatchesin117cases,involving102asteroids.

ThesameproblemoccursaswellinanotheroneofthemainNEOWISEpapers:Mainzer,Grav,Bauer,etal.,2011,“NEOWISEObservationsofNearEarthObjects:PreliminaryResults.”Thepaperisathttp://iopscience.iop.org/article/10.1088/0004-637X/743/2/156,anditsdatatableisavailableathttp://iopscience.iop.org/0004-637X/743/2/156/suppdata/apj408731t1_mrt.txt.Thispaperpresents428diameters,againstatingthattheyaretheresultsofthermalmodelfits.Butsixofthediameters,forthreeasteroids,exactlymatchROSnumberpreviouslypublished.

BetweenthesetwoMasieroandMainzerpapers,Ifound123exactmatchestoROSdiametersfor102asteroids.(ThethreeasteroidsthathavetheexactmatchesinMainzeralsoappearinMasiero–eventhoughthatmakesnosensebecauseoneisaboutmainbeltasteroidsandtheotherisaboutNEOs.)Thetableattheendofthisguidelistsall123cases,alongwithreferencessothatanyonecancheckthemeasily.(ThislistisalsoincludedinmypaperasTable4.)YoucancheckthemyourselfbecauseIalsolistthereferencesandtheURLs.

Thematchesarecertainlynotcoincidence.AlloftheROSpaperspre-datethefirstNEOWISEpapers,andinfactarereferencedbyoneofthem.TheNEOWISEteamdefinitelyknewaboutthem.NorisitcredibletothinkthattheNEOWISEmodelfittingjusthappenedtoarriveatexactlythesameestimates,downtothemeter.Insection4.3ofmypaper,Icalculatetheoddsofthisoccurringbychance.Evenunderincrediblygenerousassumptions,thoseoddsarelessthan1in10&'(,whichisprettymuchthesameassayingitisimpossible.

Youmightwonderwhetherthe“.000”isimportanttotheeffect.Maybetheyjustroundedsomediametersdifferently?Thisseemslikeapoorexplanationtome.Estimatesfortheotherasteroidsinthesamepapersarecarriedouttothenearestmeter.Thereisnovalidreasontoselectivelyroundsomeasteroidsandnotothers.

Moreover,theoddsagainstjustthefirstthreedigitsmatching(i.e.matchingatthekilometerlevelratherthantothemeter)arestillastronomical(punintended).Youjustcan’tgetthatnumberofexactmatchesbycoincidence.


Itisinterestingthatwhilethediameterswerecopiedtheestimatederrors(thenumberswith±infrontofthem)werenot.ThatsuggeststhattheNEOWISEteamdidtheirownerroranalysisaftercopyingthediameters,whileholdingdiameterfixed.

RememberthatthescientificpurposeofcomparingdiametersfromthermalmodelfitstoROSdiametersistotestwhetherthethermalmodelsareabletocomeupwithestimatesthatareclosetotheROSdiameters.Theonlypossiblewaytoperformsuchatestistofirstcomputediametersfromyourthermalmodels,thenasaseparatestep,comparethemtotheROSdiameters.

Previousstudieshavedoneexactlythat.Ryan&Woodward(2010),forexampleapplytwodifferentthermalmodels–STMandNEATM–toasteroidsandthenpresentatablecomparingtheresultstoROSmeasurements.Hereisanexcerptfromthatpaper,whichisavailableathttp://iopscience.iop.org/article/10.1088/0004-6256/140/4/933:

WhenMasieroetal.claimintheirabstractthatthediameterstheyobtainbyfittingthermalmodelsareaccurateto“betterthan10%,”thatimpliestheyhavecomparedtheircomputeddiameterstosomegold-standardmethodofmeasuringdiameters.TheonlystandardsavailablearetheROSdiameters.TheyoughttoincludeatablejustliketheonebyRyanandWoodward.Onecouldthencomputethepercentageaccuracyforeachasteroid,andtallyitupcollectivelyforagroup.ButtheMasieroetal.paperincludesnosuchcomparison,sothereisnowaytochecktheirclaimofhighaccuracy.

WhatismuchworseisthatcopyingthediameteroftheROSexamplesexactlycompletedefeatstheabilityforanyoneelsetomakethatcomparison.


Itstrikesmeascompletelyillogicaltoreportthermal-modelingdiametersforeverythingexcepttheonlyasteroidsthatonecouldpossiblyverifythemodelswith.Yetthat’sexactlywhatthesetwoNEOWISEpapersdid.

TheRedHerringExplanationWhenmypaperwasreleasedinpreprintform,itwasimmediatelycoveredbyseveralmediaoutlets.ThereportersofcourseaskedtwooftheprincipalWISEandNEOWISEastronomers,Dr.AmyMainzerandProfessorNedWright,aboutthesecuriousmatchestotheROSdiameters.Unfortunately,thescientistsskirtedthetopicbyraisingoneredherringafteranother.ThatiswhyIhavewrittenthisguide:tocounterthisfeint,inthehopethatotherswillpressthemtoproviderealanswerstothesubstantivequestionsraisedbymystudy.

InawrittenstatementaboutmypapertoareporterforTheNewYorkTimes,Dr.Mainzersaid:

InMainzeretal.2011AstrophysicalJournal736,100,radarmeasurementswereusedasgroundtruthinordertocomputemodelWISEandvisiblefluxes,whichwerecomparedtothemeasuredfluxesandfoundtobeingoodagreement(Figure3).ThisishowtheWISEfluxcalibrationforasteroidswasvalidated.Thepaperreportedtheradardiameters,becausethesewereheldfixedsothatthepredictedWISEfluxescouldbecomputed.ThecaptionforTable1ofthepaperstates"ThediametersandHvaluesusedtofiteachobjectfromtherespectivesourcedata(eitherradar,spacecraftimaging,oroccultation)aregiven."

AndinastatementgiventoareporterforScience,Dr.Mainzerreiterated,referringtosection4.3ofmypaper,whichdiscussesthecopyingofROSdiameters:

Thepapermischaracterizestheuseoftheradar/occultation/flybydiameters.InMainzeretal.2011ApJ736,100,theNEOWISEteamusesthesediametersascalibratortargetstocomputemodelbrightnessesandcomparethemtothemeasuredbrightnessesfortheobjects.Theyareinexcellentagreement.Theradar/occultation/flybysourcesarecitedinthispaper;laterpapersreferencethiscalibrationpaper.

These“explanations”areactuallyadodge,intwowaysatonce.

First,it’sadodgebecauseitgivesamisleadingimpressionaboutapaperwehaven’tyetdiscussed,Mainzer,Grav,Masiero,etal.,2011,“ThermalModelCalibrationforMinorPlanetsObservedwithWide-FieldInfraredSurveyExplorer/NEOWISE,”availableathttp://iopscience.iop.org/article/10.1088/0004-637X/736/2/100/pdf.Inthispaper,Mainzerandhercoauthorsdo


arguethatitislegitimatetousetheROSdiameters–inthatpaperalone–to“calibrate”thecolorsandotheraspectsoftheirthermalmodel.

Onemightthinkthat“calibration”meanswhatittypicallydoesinscience–thatonemeasuresanempiricalvalueofoneormoreparametersthatthenguaranteesthatyou’llgetthesameresult–atleastforthecalibrationset,andhopefullymore.

Thisisemphaticallynotthecasehere.TheNEATMmodelhasnocalibrationparameters.WhatMainzerandcoworkersmeanby“calibration”isdebatable,buttheyappeartomeanmoreinthesenseofvalidation–thatwhentheycalculatetheircolorcorrection(toadjusttothepropertiesoftheWISEsensor),theygetroughlythesameobservedIRfluxfromthetestobjectsusingtheROSdiametersastheyseefromtheasteroidstheyrepresent.

Itwouldbesurprising(suspicious,even)ifthermalmodelfittingthenproduceddiameterestimatesthatexactlymatchROSdiameterstosixdigitsofprecision.

Dr.Mainzer’sexplanationswereadodgeinasecond,evenmoreimportantway.Theydon’tbearatallonthetwostudiesthatIshowunjustifiablycopiedROSdiameters,namelytheMasieroetal.MainBeltstudyandtheMainzeretal.NEOstudymentionedabove.Inthosepapers,thediametersmatchROSfiguresfor102asteroids–farmorethanthe47thatmatchinthecalibrationpaperthatDr.Mainzercited.Thereisnoexplanationinthosepapersthatdiameterswerecopied,letaloneajustificationforwhyitwasdone.NorcanIimagineanyjustificationthatwouldpassscientificmuster.

Theproperprocedurethesetwopapersshouldhavetakenisclear:

1. Fitthermalmodelstoalloftheasteroidsinthesamemanner,regardlessofwhetheranROSdiameterisavailableornot.

2. ThenforasteroidswhichalsohaveaROSdiameter,comparetheunbiasedthermalmodelresulttotheROSdiameter.

RyanandWoodwardfollowedthissimpleandlogicalapproach,ashasnearlyeveryotherresearchgroupworkinginthisareaexceptfortheNEOWISEteam.It’sthestandardbecauseitistheonlywaytogainconfidencethatathermalmodelgeneratesgoodestimatesforthe150,000orsoasteroidsthatlackROSdiametermeasurements.

Howcouldthishappen?TheNEOWISEgroupincludesmanyhighlyrespectedscientists,whodeservethebenefitofthedoubt.OnewouldexpectthecorrespondingauthorsonthestudytocorrespondwithmeandanswerthequestionsIputtothemasIuncoveredthese


irregularitiesintheirpublishedwork.Instead,withoneexception,theyflatlyrefusedtocorrespondwithmethemomentIaskedasimplephysicsquestionaboutNEOWISE(aboutKirchhoff’slaw,butthat’saseparatetopic.)

TheonlypersonconnectedwithWISE/NEOWISEwhowouldcommunicatewasProfessorWright,whoistheprincipalinvestigatorofWISEbutisnotformallypartoftheNEOWISEgroup.Heofferedmesomeveryimportantpointers.Yethetoodeclinedtoanswermostofmyquestionsorgavefrustrating,crypticresponseslike“youstartoutcorrectbutthengoastray.”

Deniedthehelpofthosewhocouldmosteasilyaddressmyconcerns,IemailedotherasteroidresearcherswhohadpublishedpapersusingWISE/NEOWISEdata.Theyallsaidthesamething:first,thattheycouldnotreplicatetheresults,andsecond,thattheNEOWISEteamdidnotanswertheiremailedquestionseither.Anumberofthem,citedintheacknowledgementssectionofmypaper,lookedovermyresultsandprovidedhelpfulfeedback.

ClearlytheNEOWISEteam,asafederallyfundedresearchgroup,shouldbemoretransparentandcooperativewithotherscientistsintheasteroidresearchcommunity.AndclearlytheyshouldofferfullandforthrightexplanationsaboutproblemsthatIandothershavepointedoutintheirpublishedwork.

Absentthatcooperation,wearelefttospeculateaboutwhatwentwrong.Ihavewrackedmybrainaboutthisissueandtalkedwithastronomicalcolleaguesaboutit.SofarIhavebeenabletocomeupwithjustafewpossibilities:

• Colossalerror.Itispossiblethatasoftwarebug,filecorruptionorsomeotheraccidentalsourcesomehowcopiedthediametersoftheROSsourcesoverthediametersthatNEOWISEcalculated.

• Fraud.Whileitisuncomfortableforscientiststoconfrontthis,thefactthatisthatinallwalksoflifepeoplemakethingsup.Scientificmisconductisunfortunatelyawell-knownphenomenon.

• Somethingelse.Thisisacomplexareaofscience,anditispossiblethatthereissomeinnocentexplanation.

LetmebeclearthatIdon’tknowwhichitis–onlypeopleinsidetheNEOWISEgroupknowthat.Theiranswerstodatearequitedisappointing,butperhapstheywillcomeclean,oraninvestigativebodycangetthemto.It’salsopossiblethatnotallofthemknowthereason,evenifittrulywasahiddenerror.

Let’sconsidereachofthepossibilitiesinturn.


Errorsoccurinsciencejustastheydoinanyhumanendeavor.Someerrorsareminor,andsome–thecolossalvariety–fundamentallyundermineastudy.

ProfessorWrighthaspointedoutsometyposinmyequations.Hehasusedthesetoarguethatmyentirepapershouldbedisregarded.Butinfacttheyareminorandeasilycorrected.

WrighthasalsoarguedininterviewsthatIhaveahistoryofbeingassociatedwithbadsoftware,citingmyyearsaschieftechnologyofficeratMicrosoft.SoIguessitiswithhisendorsementthatIcanoffermyexpertopiniononsoftwarebugs.

Everybodywhoprogramscomputerscreatessoftwarebugs.Whatmattersinsoftwaredevelopmentisnotjusthowgoodyouareatavoidingbugs,butalsohowgoodyouareattestingtofindthem.

Itcertainlyispossiblethatabugorerrorcouldhavecausedthecopyingofdiameters.ButwhatbothersmeisMasieroTable1.Evenacausalinspectionrevealsthatallofthecasesintheredboxeshavediametersthatendin“.000,”andnoneoftheothersdo.Ididawidersearch.Outofabout120,000diameterestimatesinthatpaper,fordiametersbiggerthan9km,allbutfiveofthemhaveanexactmatchinaROSsource.Thatmakesmewonderwhetherthosefiveasteroidshaveanexactmatchaswell,justinsomeotherROSstudyIhaven’tlocated.

The000endingsarearedflagtoanybodycheckingtheresults.Theyaren’thiddendowninthegianttextfilewithmorethan100,000entries–theyareinasmalltablepublisheddirectlyinahighlyregardedjournal,shouldhavedrawnattentioniftheyweresimpleaccidentalerrors.Thefactthattheyweren’tcaughtandcorrectedeitherbeforeorafterpublicationdoesn’treflectwellonhowmuchcaretheNEOWISEteamtooktoinsuregoodresults.

Anotherlessonfromsoftwaredevelopmentisthatwhenyoufindonebug,anotheralmostcertainlyliessomewhereundetected.ThismajorerrorinthecentralresultsoftwoNEOWISEpapers,ifitisduetoabug,throwsalloftheNEOWISEresults,inalloftheirpapers,intoquestion.

It’shardtoseehowtheaccidental-errortheorycouldaccountfortheerrorestimatesthataccompanythecopieddiameters.Thosearethenumberswith±infrontoftheminsomeofthetablesabove.Theestimatederrorinthediameterisjustasimportantasthediameteritself,sothetwonumbersgotogether.

But,withonexception,thoseerrorsweren’tcopiedfromtheROSsourceslikediameterswere–theyappeartohavebeencomputedbytheNEOWISEauthors.Thatdifferenttreatmentstronglysuggests


thattheNEOWISEteamsetthediameterequalandthendidtheirerroranalysis(theyuseMonteCarlosimulation),whilekeepingthediameterfixed.

Ifabugcausedthis,thenitisaverytrickyandstrangebugindeed.

Thisbringsustoamoretroublingpossibility:thattheerrororiginatednotwithamistake,butfromdeliberatefraudormisconduct.Ihavenowaytoknowwhetherthishappened.Icertainlyhopeitdidnot.Butitwouldbeunwisenottoconsiderthepossibilitythatitwasdeliberate.

OneclearmotiveformisrepresentationwouldbetomaketheNEOWISEresultsmeetacertainstandard,eveniftheactualresultsdidn’tworkoutthatway.PriortoNEOWISE,severalotherstudies–allmuchsmaller–hadconcludedthatthermalmodelingwasaccuratetoaround10%.

WhetherthosestudiesactuallyachievedthatdegreeofaccuracyornotishardertosaybecauseROSdiametersarethemselvesnotperfectlyaccurate.SowhenonecomparesdiametersderivedfrommodelswiththosefromROSstudies,youcan’tpossiblybemoreaccuratethantheROSdiameters.Nevertheless,theearlierstudiesprimedtheconventionalwisdominthefieldtoexpect±10%accuracyfromNEOWISE.

Itwasalwaysgoingtobehard,perhapsevenimpossible,forNEOWISEtomeetthathighexpectation,forseveralreasons.OneisthatNEOWISEanalyzedahugenumberofasteroids,mostofthemrathersmall.Theasteroidsthatpreviousstudieslookedattendedtobeeitherlargeandroughlyspherical,orclosetotheearthandeasiertosee,orboth.Smallerandmoredistantasteroids–themajorityofNEOWISEcases–tendtobemoreirregularinshape.Thattranslatesintobiggererrorsontheestimatesoftheobjects’diameters.

Mypapershowsthatit’sjustnotpossiblewiththeWISEdatatoachieveagoalof±10%accuracy.Inmostcases,thedataaretooscattered,almostcertainlybecausesomanyasteroidsareshapedlikepotatoesorducksorotherirregularshapes,notspheres.Moreover,someobjectsthatlooktotheWISEtelescopelikeasinglepointoflightareactuallytwoasteroidsorbitingeachother.Caseslikethataddtothemarginoferror.

YetNEOWISEcamebackwiththeanswereverybodywanted;that“wecomputediameters…witherrorsbetterthan10%”intheMasieroMainBeltpaperdiscussedabove.

Thiscertainlysuggeststhattheycalculatedtheaccuracy.Asdiscussedabove,theonlywaytodothisistoestimatethediameterfortheasteroidsthathaveROSdiameters,thencomparetothat


diameter.Yet,shockingly,thatisnotdoneinanyoftheNEOWISEpapers.

Instead,theNEOWISEpapersallreferbacktotheMainzeretal.“ThermalModelCalibration”paperthatisdiscussedintheprevioussection.Thatpaperdoesn’tdothecalculationeither.Insteaditpresentsanindirect,hand-wavingargumentaboutwhattheminimumerrorindiametermightbebasedonfluxesfromthetestobjectswithROSdiameters.Whynotjustdotheobviousthing:comparethethermalmodeldiameterstotheROSdiameters?Goodquestion.

It’sdeeplydisingenuousforMasieroetal.toclaimthattheycomputediametersto“betterthan10%”whentheirdatasetincludesmorethan100mainbeltasteroidsthathavecopiedROSdiameters.Obviouslytheydidnotcheckagainstthem,andneithercanwe.

Whenonefindscitationslikethisthatdon’tbackupwhatisclaimed,itisawarningsign.That’sparticularlythecasewhenthecitedclaimsarecrucialtomeetaveryhighstandard,onethatcouldaffecthowthefieldviewsthesuccessoftheproject.

Irreproducibleresultsareanotherwarningsign.NASA,asapublicentity,isobligedtomakeallofthedataandresultsitproducespublicallyaccessible.Inthiscase,theobservationaldatafromWISEisavailable.TheNEOWISEresultsareavailable.Buthowtheyarrivedatthoseresultsisnot.

Scientistslikemyselfwhodon’treceiveNASAfundingoughttobeabletousetheWISEdataandourownimplementationoftheNEOWISEdataanalysismethodstoreproducethediameterestimatesthattheNEOWISEstudiespublished.

Itriedmightilytodothis,butitisnotpossible.TheNEOWISEresultsareirreproduciblebecausetheydon’tdescribehowtheygotthem.Theanalyticalprocedurestheyappliedareunnecessarilycomplicatedbyallkindsofad-hocrulesthatmakenosensefromastatisticsstandpoint.

Thestudiesalsoreferencestepsinotherpapersthatdon’texist.HereisoneexamplefromtheMainzeretal.NEOpaper:

AsdescribedinMainzeretal.(2011b,hereafterM11B)andCutrietal.(2011),weincludedobservationswithmagnitudesclosetoexperimentally-derivedsaturationlimits,butwhensourcesbecamebrighterthanW1=6,W2=6,W3=4andW4=0,weincreasedtheerrorbarsonthesepointsto0.2magnitudesandappliedalinearcorrectiontoW3(seetheWISEExplanatorySupplementfordetails).


Here’stheproblem:thatMainzeretal.paperciteddoesnotprovideanyfurtherdetailsasclaimed.AndtheCutrietal.2011paper,alsoknownastheWISEExplanatorySupplement(orWES,availableherehttp://wise2.ipac.caltech.edu/docs/release/allsky/expsup/),containsno“linearcorrection”toW3.

Iemailedthecorrespondingauthorsforthedetailsbutreceivednone.Prof.Wright,aftermanyrequests,atleastdidadmitinemailthathecouldn’tfindthelinearcorrectiontoW3either.HereferredmetoamuchmorerecentpaperbyChineseastronomerslookingatstars,notasteroids,thatfoundonlytheveryslightestcorrectiontotheW3band.

Mypaperenumeratesmanyothersuchexamples.Somearemoreseriousthantheoneabove.Thegroupneverexplains,forexample,howtheydocurve-fitting.Thereareseveraldifferentwaystodothis,andyoucangetenormouslydifferentresultsdependingonwhichapproachyoutake.

Tosumup,alargeresearchgroup,generouslyfundedbythegovernmentanddominantinitsfield,haspublishedwhatappeartobeobviouslyerroneousresults,haverepeatedlyfailedtofollownormsofrigorousstatisticalanalysis,havedescribedtheirmethodsmisleadinglyandincompletely,andhaverefusedstandardrequestsbyotherscientistsfortheinformationneededtoreplicatetheirwork.

Allthatisleft,then,foranindependentresearcherwantingtoassesstheaccuracyoftheNEOWISEresultsistocomparethemtoROSresults.Yetthisisnotpossiblebecause,inalmosteverycase,NEOWISEsettheirresultsequaltotheROSresults.Couldthisstephavebeentakentopreventanyonesecond-guessingtheiraccuracy?

IstressyetagainthatthisiscircumstantialandIaminnowayproposingthatthisconstitutesproofthatthediametersweredeliberatelycopied.ButtheNEOWISEgroupcertainlyhasdisplayedverybadbehaviorbytheprofessionalstandardsofscience,andthatcertainlydoesnotbodewell.Atthispointitseemsclearthatthepossibilityofmisconductmustbeinvestigated–eveniftheNEOWISEgroupclaimsitwasallamistake.Bugshappen,buttheyaren’talwaysaccidental.

Theanswercouldalsobenoneoftheabove.Perhapswhathappenedwasdeliberatebuthasaninnocentexplanation.Acolleaguesuggested,forexample,thatmaybetheNEOWISEteamusedtheROSdiameterssothattheycouldprovidebetterestimates.Afterall,ifpartoftheNEOWISEmissionistocomeupwiththebestpossibleestimates,whynotusetheROSvaluewheretheyexist?


Unfortunately,thatdoesn’treallywork.IftheROSdiametersactuallyaremuchmoreaccuratethanthoseestimatedbyNEOWISEthermalmodeling,thentherightthingtodoistoreportthatsothatweknowhowtoevaluatetheestimatesforthe150,000asteroidsthathavenoROSdata.JustadoptingtheROSnumbersfor102asteroids,atthecostofmakingitimpossibletochecktheaccuracyoftherest,isfundamentallydeceptive–notinnocent.

ButmaybeIamjustnotthinkingcreativelyenough.Supposethatthereisaninnocentexplanation.Thenwhydidtheyforgettotellus?TheMasieroandMainzerarticlesassertrepeatedlythattheresultsareNEOWISEthermal-modelingresults.That’sveryclear–butalsoclearlyincorrect.

CopyingdiametersfromtheROSdatasetisahugeexceptiontowhattheydescribe.Noscientistcouldpossiblyconsiderthatsomethingthatdoesn’twarrantclearandcarefulexplanation.Ifthereissomeinnocent,orevendesirablereason,thenitshouldhavebeendisclosedalongwiththecopying.

QuestionsforNEOWISEMypaper,obviouslycontroversialand110pageslong,willtakesometimetogothroughpeerreview.Inthatprocess,expertswilljudgewhetherthedetailedanalysesIpresentareright,wrong,orneedadditionaljustification.

Butmyresearchisn’ttheissuehere.Asthisguideshows,anyonecanseetheirregularitiesintheNEOWISEdata.Thatisn’tadeepquestionforexperts–theseexactmatchesinthedataarequiteobvious.

TheNEOWISEpaperswentthroughpeerreviewthemselves,butweareconstantlyremindedofpublishedscientificdiscoveriesthatwerewrong.The“discovery”offaster-than-lightparticlesatCERNturnedouttobeduetoamiscalibratedGPStimestandard.SolidstatephysicistHendrikSchönfakedhisresearch–including17peerreviewedpapers.

TheburdenofexplainingthisclearandobviousproblemfallssquarelyonDr.Mainzerandherteam.

Dr.Mainzerhasimportantquestionstoanswer.Whatistheexplanationforthecopieddiameters?Isitcolossalerror,fraud,orsomethingelsewhichisdeliberate-but-innocent?

Weneeddetails.Asimplebrush-offordiversionisnotsufficient.

ForProf.Wright,whowasinitiallyhelpfulbuthassincebecomeanoutspokencriticandapologistforNEOWISE,thequestionis:Did


youknowaboutthisissueandgoalongwithit?Ifnot,whyaggressivelydefendtheprojectbeforeyouknowallthedetails?

Asaprivatecitizen,allIcandoisraisethesequestionsandpursuemyownresearch.Ican’tmakeanyoneansweranything;indeedmytrackrecordofgettingtheNEOWISEteamtorespondisquitepoor.ButmyhopeisthatresearchmanagersatNASA,JPL,TheAstronomicalJournal,andelsewherewillbeabletogetanswerstothesequestionsquickly.

ReferencestothermalmodelingpapersOpenaccessreprintsareavailablefromtheURLsprovided.

Mainzer,A.,Grav,T.,Bauer,J.M.,Masiero,J.R.,McMillan,R.S.,Cutri,R.M.,…Wasserman,L.H.(2011).NeowiseObservationsofNear-EarthObjects:PreliminaryResults.TheAstrophysicalJournal,743(2),156.doi:10.1088/0004-637X/743/2/156http://iopscience.iop.org/article/10.1088/0004-637X/743/2/156/pdf

Mainzer,A.,Grav,T.,Masiero,J.,Bauer,J.M.,Wright,E.L.,Cutri,R.M.,…Eisenhardt,P.R.M.(2011).ThermalModelCalibrationforMinorPlanetsObservedWithWide-FieldInfraredSurveyExplorer/Neowise.TheAstrophysicalJournal,736(2),100.doi:10.1088/0004-637X/736/2/100http://iopscience.iop.org/article/10.1088/0004-637X/736/2/100/pdf

Masiero,J.R.,Mainzer,A.,Grav,T.,Bauer,J.M.,Cutri,R.M.,Dailey,J.,…Wilkins,A.(2011).MainBeltAsteroidswithWISE/NEOWISE.I.PreliminaryAlbedosandDiameters.TheAstrophysicalJournal,741(2),68.doi:10.1088/0004-637X/741/2/68http://iopscience.iop.org/article/10.1088/0004-637X/736/2/100/pdf

Ryan,E.L.,&Woodward,C.E.(2010).RectifiedasteroidalbedosanddiametersfromIRASandMSX.TheAstronomicalJournal,140(4),933.doi:10.1088/0004-6256/140/4/933http://iopscience.iop.org/article/10.1088/0004-6256/140/4/933

ReferencestoROSpapersURLstofreeoropenaccessreprintsareprovidedwhereavailable.

Durech,J.,Sidorin,V.,Kaasalainen,M.,2010.DAMIT:adatabaseofasteroidmodels.Astron.Astrophys.513,A46.doi:10.1051/0004-6361/200912693http://www.aanda.org/articles/aa/full_html/2010/05/aa12693-09/aa12693-09.html

Durech,J.,Kaasalainen,M.,Herald,D.,Dunham,D.,Timerson,B.,Hanuš,J.,Frappa,E.,Talbot,J.,Hayamizu,T.,Warner,B.D.,Pilcher,F.,Galád,A.,2011.Combiningasteroidmodelsderivedbylightcurveinversionwithasteroidaloccultationsilhouettes.Icarus214,652–670.doi:10.1016/j.icarus.2011.03.016http://www.sciencedirect.com/science/article/pii/S0019103511001072http://arxiv.org/abs/1104.4227

Magri,C.,1999.MainbeltAsteroids:ResultsofAreciboandGoldstoneRadarObservationsof37Objectsduring1980–1995.Icarus140,379–407.doi:10.1006/icar.1999.6130http://www.sciencedirect.com/science/article/pii/S0019103599961304http://echo.jpl.nasa.gov/asteroids/magri+1999_37_survey.pdf

Keller,H.U.,Barbieri,C.,Koschny,D.,Lamy,P.,Rickman,H.,Rodrigo,R.,Sierks,H.,A’Hearn,M.F.,Angrilli,F.,Barucci,M.A.,Bertaux,J.-L.,Cremonese,G.,DaDeppo,V.,Davidsson,B.,DeCecco,M.,Debei,S.,Fornasier,S.,Fulle,M.,Groussin,O.,Gutierrez,P.J.,Hviid,S.F.,Ip,W.-H.,Jorda,L.,Knollenberg,J.,Kramm,J.R.,Kührt,E.,Küppers,M.,Lara,L.-M.,Lazzarin,M.,LopezMoreno,J.,Marzari,F.,Michalik,H.,Naletto,G.,Sabau,L.,Thomas,N.,Wenzel,K.-P.,Bertini,I.,Besse,S.,Ferri,F.,Kaasalainen,M.,Lowry,S.,Marchi,S.,Mottola,S.,Sabolo,W.,Schröder,S.E.,Spjuth,S.,Vernazza,P.,2010.E-typeasteroid(2867)SteinsasimagedbyOSIRISonboardRosetta.Science327,190–193.doi:10.1126/science.1179559http://www.astro.umd.edu/~sbesse/REPRINTS/Keller_steins_2010.pdf


Ostro,S.J.,Campbell,D.B.,Hine,A.A.,Shapiro,I.I.,Chandler,J.F.,Werner,C..,Rosema,K.D.,1990.RadarImagesofAsteroid1627Ivar.Astron.J.99http://adsabs.harvard.edu/full/1990AJ.....99.2012O

Veeder,G.J.,Hanner,M.S.,Matson,D.L.,Tedesco,E.F.,Lebofsky,L.A.,Tokunaga,A.T.,1989.Radiometryofnear-earthasteroids.Astron.J.97,1211–1219.doi:10.1086/115064http://articles.adsabs.harvard.edu/cgi-bin/nph-iarticle_query?1989AJ.....97.1211V&classic=YES

Busch,M.W.,2010.Shapesandspinsofnear-Earthasteroids(DoctoralDissertation).CaliforniaInstituteofTechnology.https://books.google.com/books?hl=en&lr=&id=3NLLj8hQHXoC&oi=fnd&pg=PR8&dq=%22shapes+and+spins+of+near-Earth+asteroids%22&ots=ZrCLeTTUCY&sig=2jCM6HPsn19mJG3_8dceqTjkwyc

Benner,L.A.M.,Nolan,M.C.,Ostro,S.J.,Giorgini,J.D.,Pray,D.P.,Harris,A.W.,Magri,C.,Margot,J.L.,2006.Near-EarthAsteroid2005CR37:Radarimagesandphotometryofacandidatecontactbinary.Icarus182,474–481.doi:10.1016/j.icarus.2006.01.016http://www.sciencedirect.com/science/article/pii/S0019103506000315http://echo.jpl.nasa.gov/asteroids/2005CR37/benner.etal.2006.2005cr37.pdf

Mahapatra,P.R.,Benner,L.A.M.,Ostro,S.J.,Jurgens,R.F.,Giorgini,J.D.,Yeomans,D.K.,Chandler,J.F.,Shapiro,I.I.,2002.Radarobservationsofasteroid7335(1989JA).Planet.SpaceSci.50,257–260.doi:DOI:10.1016/S0032-0633(02)00002-8http://www.sciencedirect.com/science/article/pii/S0032063302000028http://echo.jpl.nasa.gov/asteroids/mahapatra.etal.2002.1989ja.pdf

Magri,C.,Nolan,M.C.,Ostro,S.J.,Giorgini,J.D.,2007.Aradarsurveyofmain-beltasteroids:Areciboobservationsof55objectsduring1999-2003.Icarus186,126–151.doi:10.1016/j.icarus.2006.08.018http://www.sciencedirect.com/science/article/pii/S001910350600282Xhttp://echo.jpl.nasa.gov/asteroids/MBAs/magri.etal.2007.mbas.pdf

Shepard,M.K.,Clark,B.E.,Ockert-Bell,M.,Nolan,M.C.,Howell,E.S.,Magri,C.,Giorgini,J.D.,Benner,L.A.M.,Ostro,S.J.,Harris,A.W.,Warner,B.D.,Stephens,R.D.,Mueller,M.,2010.AradarsurveyofM-andX-classasteroidsII.Summaryandsynthesis.Icarus208,221–237.doi:10.1016/j.icarus.2010.01.017http://www.sciencedirect.com/science/article/pii/S0019103510000278http://echo.jpl.nasa.gov/asteroids/MBAs/shepard.etal.2010.mxmbas.pdf

Shevchenko,V.G.,Tedesco,E.F.,2006.Asteroidalbedosdeducedfromstellaroccultations.Icarus184,211–220.doi:10.1016/j.icarus.2006.04.006http://www.sciencedirect.com/science/article/pii/S001910350600128X

TableofexactmatchesBelowisTable4frommypaper,showingalloftheexactmatches.

NEOWISEPublishedResults PriorRadar/Occultation/SpacecraftReferences

Asteroid 𝑫 𝝈𝑫 Paper 𝑫 𝝈𝑫 Paper

2

544000 60714 Masiero/MB:Pre 544000 N/A Shevchenkoetal.,2006

522000 N/A Shevchenkoetal.,2006

539000 28000 Durechetal.,2011

5

115000 9353 Masiero/MB:Pre115000 6000 Durechetal.,2011

115000 12000 Mainzer/TMC

108290 3700 Masiero/MB:NIR

120000 14000 Magrietal.,1999

6

185000 10688 Masiero/MB:Pre 185000 10000 Magrietal.,1999


180000 40000 Durechetal.,2011

8




138000 9000 Magrietal.,1999

141000 10000 Durechetal.,2011


13227000 38000 Mainzer/TMC

227000 30000 Magrietal.,2007227000 25948 Masiero/MB:Pre



19



210000 10000 Timmersonetal.,2010

36103000 11451 Masiero/MB:Pre

103000 11000 Magrietal.,2007103000 1000 Mainzer/TMC



39





46 124000 9641 Masiero/MB:Pre124000 9000 Magrietal.,1999

124000 9000 Magrietal.,2007

47

138000 13000 Mainzer/TMC 138000 N/A Shevchenkoetal.,2006












54

142000 14758 Masiero/MB:Pre142000

9000 Durechetal.,2011


135000 20000 Durechetal.,2011

165000 19000 Magrietal.,2007



64



51000 10000 Durechetal.,2011

52000 10000 Durechetal.,2011

80

79000 9125 Masiero/MB:Pre79000 10000 Magrietal.,1999




67000 11000 Durechetal.,2011

83




81000 2000 Shepardetal.,2010

84 79000 4867 Masiero/MB:Pre 79000 13000 Magrietal.,1999

85

163000 18648 Masiero/MB:Pre 163000 15000 Durechetal.,2011

163000 16000 Mainzer/TMC 163000 19000 Magrietal.,2007




94

187500 7256 Masiero/MB:Pre187500 N/A Shevchenkoetal.,2006



97

83000 6000 Mainzer/TMC 83000 5000 Shepardetal.,2010









109




11089000 6336 Masiero/MB:Pre 89000 9000 Shepardetal.,2010




114







128

188000 9002 Masiero/MB:Pre188000 29000 Mainzer/TMC



141000 37000 Magrietal.,2007

129



113000 12000 Shepardetal.,2010

118000 19000 Durechetal.,2011

134 112200 10798 Masiero/MB:Pre 112200 N/A Shevchenkoetal.,2006

135

77000 8000 Mainzer/TMC77000 7000 Shepardetal.,2010





139

164000 19000 Mainzer/TMC164000 22000 Magrietal.,1999






145









182





192





95000 13000 Magrietal.,1999

198




208


45000 10000 Durechetal.,201145000 5000 Mainzer/TMC






211




216




124000 15000 Shepardetal.,2010



230





238











306



49000 5000 Durechetal.,2011

53000 5000 Durechetal.,2011

308

144400 13864 Masiero/MB:Pre144400 N/A Shevchenkoetal.,2006





324


229000 12000 Magrietal.,2007




198770 5600 Grav/Hilda



99000 N/A Shevchenkoetal.,200699000 11469 Masiero/MB:Pre



350


99500 N/A Shevchenkoetal.,200699500 6354 Masiero/MB:Pre





















444







488





40000 8000Shepardetal.,2010



84000 9000 Mainzer/TMC 84000 9000 Mainzer/TMC

566

134000 15000 Mainzer/TMC134000 N/A Shevchenkoetal.,2006







654






704
















771

29000 2544 Masiero/MB:Pre29000 2000 Shepardetal.,2010









45000 2000 Shepardetal.,2010

45000 6000 Magrietal.,1999


914




925

58000 6000 Mainzer/TMC58000 16000 Durechetal.,2011




951


12200 813 Masiero/MB:Pre 12200 N/A Thomasetal.,1994




1512

65000 7000 Mainzer/TMC65000 N/A Shevchenkoetal.,2006


79870 770 Grav/Hilda

1627

9000 817 Masiero/MB:Pre9000 900 Ostroetal.,1990


8370 75 Mainzer/NEO:Pre

10200 N/A Veederetal.,1989

1866

8700 1000 Mainzer/TMC8700 1000 Mainzer/TMC





7335




1000 N/A Mahapatraetal.,2002




68216







164121







1717 550 Mainzer/PP:3

1738 577 Mainzer/PP:3

2005CR37

1000 123 Masiero/MB:Pre1000 N/A Benneretal.,2006



Documents

A Simple Guide to NEOWISE Problems final€¦ · see the star wink out for a short period. If you time the event accurately, you can use the duration of the wink to estimate the size