39
Sasi Parthasarathy [email protected]

Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 2: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

•  WhySpam?

•  Defini8onofSpam•  TypesofSpam:PageLevel&LinkLevel•  Newerthreats•  SecurityandSpam–hijackedsites,scareware•  Low‐Qualitypages•  ChallengingProblems

Page 3: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

•  Aspampageisonethatusesoneormorespamtechniquestoinflateitsrankinginsearchresults,inawaythataddsnovaluetotheuser.

•  AtBing,wetrytoneutralizetheeffectofspamtechniquesonsearchranking

•  AtBing,wetakebothmanualandalgorithmicac8ononspam

Page 4: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

•  Searchengines,ataveryhighlevel,careabouttwo(2)thingswhenranking:–  Contentofthepage(page‐level)–  Linkspoin8ngtothepage(link‐level)

•  Spammerstargetthecontentandthelinkstogainunfairadvantageoverothersinranking

Page 5: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

•  Page‐levelspam

•  Linklevelspam

Page 6: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

PageLevel

•  KeywordStuffing•  ParkedDomain

•  HiddenContent•  MachineGeneratedContent

•  SocialMediaSpam

LinkLevel

•  LinkFarms•  LinkExchanges

Page 7: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

Keywordsaretypicallyaroundpopulartheme,buttheremaybemanythatareunrelatedandusedsimplytosurfacepageinunrelatedsearches

Page 8: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

Domainparkingreferstotheregistra8onofaninternetdomainnamewithoutthatdomainbeingassociatedwithanyservices.Parkeddomainpagesarefullofads,highlyrepe88ve,andrarelyprovideinforma8vecontent.

Page 9: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

HidingcontentfilledwithkeywordstopromotethepageTypes

•  HiddenTextonthepage•  Hiddenlinksonthepage–canberesultofhackingHow•  Use8nyfontswithsamecoloraspagebackground

•  Use<invisible>,<hidden>tags

Page 10: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)
Page 11: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)
Page 12: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

•  <p><fontsize=1color=white>•  Rivertongymnas8cs,Lehigymnas8cs,Jordangymnas8cs,Herrimangymnas8cs,SaltLakeCitygymnas8cs,gymnas8cssaltlake,gymcatsut,gymcats

ut,parentsnightoff,parentsnightoutgymcats,gymcatsdraper,gymcatsdraper,gymnas8cscamp,•  backhandsprings,gymcatsofftrack,UtahGymcats,Gymcatsgymnas8cs,Utahgymcats,gymcats,gymcatsatthepoint,gymcat,gymcat•  Drapergymnas8cs,Bluffdalegymnas8cs,Rivertongymnas8cs,SaltLakegymnas8cs,Utahgymnas8cs,•  Bluffdaletumbling,Rivertontumbling,Drapertumbling,Draper,Bluffdale,Riverton,SaltLakeCity,Lehi,Herriman,SouthJordan,WestJordan,•  Utah,gymnas8cs,Drapergymnas8cs,BluffdaleGymnas8cs,Bluffdaletumbling,Rivertontumbling,Drapertumbling,LehiTumbling,Herriman

Tumbling,EagleMountaingymnas8cs,JordanTumbling,•  Rivertongymnas8cs,Lehigymnas8cs,Jordangymnas8cs,Herrimangymnas8cs,SaltLakeCitygymnas8cs,gymnas8cssaltlake,gymcatsut,gymcats

ut,parentsnightoff,parentsnightoutgymcats,gymcatsdraper,gymcatsdraper,gymnas8cscamp,•  backhandsprings,gymcatsofftrack,UtahGymcats,Gymcatsgymnas8cs,Utahgymcats,gymcats,gymcatsatthepoint,gymcat,gymcat•  Drapergymnas8cs,alpinegymnas8cs,Highlandgymnas8cs,AmericanForkgymnas8cs,Lehigymnas8cs,•  Bluffdaletumbling,Rivertontumbling,Drapertumbling,Draper,84065,84095,84020,84043,Herriman,SouthJordan,WestJordan,•  Utah,gymnas8cs,Drapergymnas8cs,BluffdaleGymnas8cs,Bluffdaletumbling,Rivertontumbling,Drapertumbling,LehiTumbling,Herriman

Tumbling,EagleMountaingymnas8cs,JordanTumbling,•  Rivertongymnas8cs,Lehigymnas8cs,Jordangymnas8cs,Herrimangymnas8cs,SaltLakeCitygymnas8cs,gymnas8cssaltlake,gymcatsut,gymcats

ut,parentsnightoff,parentsnightoutgymcats,gymcatsdraper,gymcatsdraper,gymnas8cscamp,•  backhandsprings,gymcatsofftrack,UtahGymcats,Gymcatsgymnas8cs,Utahgymcats,gymcats,gymcatsatthepoint,gymcat,gymcat•  Drapergymnas8cs,Bluffdalegymnas8cs,Rivertongymnas8cs,SaltLakegymnas8cs,Utahgymnas8cs,•  Bluffdaletumbling,Rivertontumbling,Drapertumbling,Draper,Bluffdale,Riverton,SaltLakeCity,Lehi,Herriman,SouthJordan,WestJordan,•  Utah,gymnas8cs,Drapergymnas8cs,BluffdaleGymnas8cs,Bluffdaletumbling,Rivertontumbling,Drapertumbling,LehiTumbling,Herriman

Tumbling,EagleMountaingymnas8cs,JordanTumbling,•  Rivertongymnas8cs,Lehigymnas8cs,Jordangymnas8cs,Herrimangymnas8cs,SaltLakeCitygymnas8cs,gymnas8cssaltlake,gymcatsut,gymcats

ut,parentsnightoff,parentsnightoutgymcats,gymcatsdraper,gymcatsdraper,gymnas8cscamp,•  backhandsprings,gymcatsofftrack,UtahGymcats,Gymcatsgymnas8cs,Utahgymcats,gymcats,gymcatsatthepoint,gymcat,gymcat•  Drapergymnas8cs,Bluffdalegymnas8cs,Rivertongymnas8cs,SaltLakegymnas8cs,Utahgymnas8cs,•  Bluffdaletumbling,Rivertontumbling,Drapertumbling,Draper,Bluffdale,Riverton,SaltLakeCity,Lehi,Herriman,SouthJordan,WestJordan,•  Utah,gymnas8cs,Drapergymnas8cs,BluffdaleGymnas8cs,Bluffdaletumbling,Rivertontumbling,Drapertumbling,LehiTumbling,Herriman

Tumbling,EagleMountaingymnas8cs,JordanTumbling,•  </p>

Page 13: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)
Page 14: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

<divstyle="width:150px;height:1px;padding:0px;font:11pxTahoma;overflow:hidden;">

•  <ahref="hlp://medicalpioneers.com/images/thumbs/v9/viagra‐ivf‐frozen‐embryo.php">viagraivffrozenembryo</a><ahref="hGp://www.snugjazz.com/siteold/log/viagra‐or‐cialis‐which‐is‐healthier.php">viagraorcialiswhichishealthier</a><ahref="hGp://medicalpioneers.com/images/thumbs/v9/viagra‐user‐staNsNcs.php">viagrauserstaNsNcs</a><ahref="hlp://www.seafdec.org/cms/language/v8/suppository‐viagra‐100mg.php">suppositoryviagra100mg</a><ahref="hlp://www.snugjazz.com/siteold/log/viagra‐patent‐length‐cialis‐levitra.php">viagrapatentlengthcialislevitra</a><ahref="hlp://www.seafdec.org/cms/language/v8/gm‐spends‐millions‐on‐viagra‐salaried‐employees.php">gmspendsmillionsonviagrasalariedemployees</a><ahref="hlp://www.seafdec.org/cms/language/v8/viagra‐m‐portugal.php">viagramportugal</a><ahref="hlp://www.seafdec.org/cms/language/v8/laughing‐gas‐viagra‐and‐lipitor.php">laughinggasviagraandlipitor</a><ahref="hlp://www.snugjazz.com/siteold/log/women‐who‐take‐cialis‐heart‐alack.php">womenwhotakecialisheartalack</a><ahref="hlp://medicalpioneers.com/images/thumbs/v9/mmm‐viagra‐t‐shirts.php">mmmviagratshirts</a><ahref="hlp://www.snugjazz.com/siteold/log/so0‐sale‐in‐europe‐viagra‐cialis‐levitra.php">so0saleineuropeviagracialislevitra</a><ahref="hlp://medicalpioneers.com/images/thumbs/v9/viagra‐developed.php">viagradeveloped</a><ahref="hlp://www.snugjazz.com/siteold/log/instruc8ons‐for‐taking‐viagra‐cialis‐levitra.php">instruc8onsfortakingviagracialislevitra</a><ahref="hlp://medicalpioneers.com/images/thumbs/v9/women‐viagra‐study‐low‐testosterone.php">womenviagrastudylowtestosterone</a><ahref="hlp://www.snugjazz.com/siteold/log/cialis‐costume.php">cialiscostume</a><ahref="hlp://medicalpioneers.com/images/thumbs/v9/how‐to‐make‐viagra‐and‐alcohol.php">howtomakeviagraandalcohol</a><ahref="hlp://medicalpioneers.com/images/thumbs/v9/take‐too‐much‐viagra‐stuffy‐nose.php">taketoomuchviagrastuffynose</a><ahref="hlp://www.seafdec.org/cms/language/v8/what‐happens‐iif‐a‐woman‐takes‐a‐viagra.php">whathappensiifawomantakesaviagra</a><ahref="hlp://www.snugjazz.com/siteold/log/ii‐bathtubs‐cialis.php">iibathtubscialis</a><ahref="hlp://www.seafdec.org/cms/language/v8/footwear‐sell‐viagra.php">footwearsellviagra</a><ahref="hlp://www.seafdec.org/cms/language/v8/viagra‐lisinopril.php">viagralisinopril</a><ahref="hlp://www.snugjazz.com/siteold/log/bri8sh‐cialis.php">bri8shcialis</a><ahref="hlp://medicalpioneers.com/images/thumbs/v9/viagra‐for‐female‐2008.php">viagraforfemale2008</a><ahref="hlp://www.snugjazz.com/siteold/log/discussion‐groups‐viagra‐vs‐cialis.php">discussiongroupsviagravscialis</a><ahref="hlp://www.seafdec.org/cms/language/v8/viagra‐dose‐speed.php">viagradosespeed</a><ahref="hlp://medicalpioneers.com/images/thumbs/v9/viagra‐and‐iraq.php">viagraandiraq</a><ahref="hlp://www.seafdec.org/cms/language/v8/viagra‐clinical‐trials‐phosphodiesterase‐inhibitors.php">viagraclinicaltrialsphosphodiesteraseinhibitors</a><ahref="hlp://www.snugjazz.com/siteold/log/suppliers‐of‐viagra‐cialis‐levitra.php">suppliersofviagracialislevitra</a><ahref="hlp://medicalpioneers.com/images/thumbs/v9/viagra‐from‐beijing.php">viagrafrombeijing</a><ahref="hlp://www.snugjazz.com/siteold/log/uk‐online‐viagra‐cialis.php">ukonlineviagracialis</a><ahref="hlp://medicalpioneers.com/images/thumbs/v9/low‐testosterone‐viagra‐gel‐group.php">lowtestosteroneviagragelgroup</a><ahref="hlp://medicalpioneers.com/images/thumbs/v9/viagra‐changes‐in‐vision‐ischemic‐op8c‐neuropathy.php">viagrachangesinvisionischemicop8cneuropathy</a><ahref="hlp://www.seafdec.org/cms/language/v8/how‐long‐effec8ve‐viagra‐cialis‐levitra.php">howlongeffec8veviagracialislevitra</a><ahref="hlp://medicalpioneers.com/images/thumbs/v9/coldwell‐banker‐viagra‐board.php">coldwellbankerviagraboard</a><ahref="hlp://www.seafdec.org/cms/phocadownload/SP/v/viavra.php">viavra</a><ahref="hlp://www.snugjazz.com/siteold/log/hot‐dogs‐and‐cialis.php">hotdogsandcialis</a><ahref="hlp://medicalpioneers.com/images/thumbs/v9/template‐of‐viagra.php">templateofviagra</a><ahref="hlp://www.snugjazz.com/siteold/log/buy‐pfizer‐viagra‐cialis‐levitra.php">buypfizerviagracialislevitra</a><ahref="hlp://medicalpioneers.com/images/thumbs/v9/viagra‐side‐effects‐depression.php">viagrasideeffectsdepression</a><ahref="hlp://medicalpioneers.com/images/thumbs/v9/viagra‐for‐free‐in‐usa.php">viagraforfreeinusa</a><ahref="hlp://www.seafdec.org/cms/language/v8/viagra‐banned.php">viagrabanned</a><a

</div>

Page 15: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

•  Contentiss8tchedwithkeywordstakenfromquerylogstorankhighforthosewords

•  Nologicalmeaningwhentryingtoreadthecontentonthepage

hlp://kessebohmer.t35.com/orz.4chan.org‐e‐res‐621947.html.html

hlp://www.ipci‐comurnat.org/page/2hlp://www.predrepka6.bezsternika.pl/movado‐museum‐piece‐watches.html

Page 16: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)
Page 17: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

Searchenginecrawlerseescontentfilledwithkeywordswhiletheusersarere‐directedtoadifferentpagefromsearchresults

Canbeinten8onalorcanbeduetothesitebeinghijacked

SourcePage:hlp://www.oie.pl/posturas‐lesbicas.html

Des8na8onPage:hlp://sexnet.sexyweb.es/

Page 18: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)
Page 19: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

R‐ratedcontent

Page 20: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

Pageswithlilleornousefulcontentlinktoeachother.Mostofthelinkfarmsarecreatedasnetworksofsitesthatcontainnumerouslinkstooneanother.

•  3e2v.org‐>eazishopping.com•  3e2v.org‐>5olde.com•  5olde.com‐>eazishopping.com•  netprivacypower.org‐>eazishopping.com•  netprivacypower.org‐>5olde.com•  angelcityoutcasts.com‐>netprivacypower.org•  angelcityoutcasts.com‐>eazishopping.com

Page 21: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

•  Aspam‐orientedlinkexchangeoccurswhencompletelyunrelatedwebsitesreciprocallyexchangelinks.Thisisdonetoinflateranking

•  Alllinksintendedtomanipulatesearchengineresultswillbediscounted

Page 22: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

•  Hijackedsites•  Scareware•  LowQualityusergeneratedcontent

Page 23: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

•  Partofthesiteishijackedtohostadultcontentorsellinteres'ng()products

•  EDUsites(studentpages,departmentpages)andsocialdiscussioncomponentsofsiteslikeforums,blogsarethemostvulnerable

hlp://ltss.edu/shared/tmpl_c/FBEF8076/buy‐viagra‐on‐line.htmlhlp://www.math.rutgers.edu/mdnews/cheapest‐place‐to‐buy‐viagra‐online.html

hlp://forums.plexapp.com/index.php?/user/13257‐bbviagra/

Page 24: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)
Page 25: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

•  Targetspikyqueriesthatfollowpopularevents{gaddafilibyanprotests},{racheloberlinpictures}•  Hijackpartofpopularsitestogetauthority/popularityfor

free(no8mefororganicspamminghere)

•  Stuffthehijackedpagewithkeywordstarge8ngthelatestevents–searchenginebotsseethispage

•  Redirecttoscarewarepagewhenclickedasasearchresult

Page 26: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)
Page 27: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)
Page 28: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

WhatisContentFarm?

“Contentfarmsarenetworksofprovidersaimedatgenera'nglargeamountsofsubjectspecific,butgenerallysuperficialcontentforthepurposeofsearchenginemone'za'on”

Mo8va8on?

Syndica8on‐basedmone8za8on

Ad‐basedmone8za8on

Page 29: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)
Page 30: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)
Page 31: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

Algorithmicandmanualac8on

Detec8onmechanismperspamtechniqueLookforawaytoaggregate

Page 32: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

•  Neutralizingspamwithouthur8ngusersisnotaneasyproblem–  Bignamesitesbuyingpaidlinks–  Dealingwithlowqualitycontentpagesfromsitesthathavegood

pagestoo

Page 33: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)
Page 34: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

•  Certaintypesofspamishardtodetectthanothers–linklevelspam

Page 35: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

•  Certainspamtechniquesarehardtodetectbyrelevancejudges–linklevelspam,lowqualitycontent–  Weakcorrela8onbetweenspamandrelevance–  Determiningtheactualimpactofspamonrelevanceishard

Page 36: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)
Page 37: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

hlp://likedrop1.info/

hlp://www.likedatstuff.com/like.php?id=479hlp://www.coolslikes.com/like.php?id=28

Withsocialsignalsbecomingpopular,itisonlynaturalthattheyaregoingtobetargetedbyspammersmore

Page 38: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

•  Eggs,bacon,spam,spam,andspam(SEM101)•  Theperniciousperfidyofpage‐levelwebspam(SEM101)

•  Theliabilityofloathsome,link‐levelwebspam(SEM101)

Page 39: Sasi Parthasarathy sasip@microso - USENIX · • Search engines, at a very high level, care about two (2) things when ranking: – Content of the page (page‐level)

Four part series on website security:

•  http://www.bing.com/community/blogs/webmaster/archive/2009/09/11/the-merciless-malignancy-of-malware-part-1-sem-101.aspx

•  http://www.bing.com/community/blogs/webmaster/archive/2009/09/18/the-merciless-malignancy-of-malware-part-2-sem-101.aspx

•  http://www.bing.com/community/blogs/webmaster/archive/2009/09/24/the-merciless-malignancy-of-malware-part-3-sem-101.aspx

•  http://www.bing.com/community/blogs/webmaster/archive/2009/10/01/the-merciless-malignancy-of-malware-part-4-sem-101.aspx