Embed Size (px)
DESCRIPTION
Selenium scraping of other people's fun and profit
Citation preview
MAN IN THE MIDDLE ATTACK ON BANKS
Selenium scraping of other people's fun and profit
WHO? WHERE? WHAT?
Marko Elezović @melezov
tech lead at Instantor AB
Swedish bank aggregator
Alice Bob
IN A NUTSHELL
Alice BobstEve
IN A NUTSHELL
IN A NUTSHELL
LEGAL, SALES & TECH
LEGAL, SALES & TECH
LEGAL, SALES & TECH
[ ] I have read and agree to the EULA
LEGAL, SALES & TECH
[ ] I have read and agree to the EULA
LEGAL, SALES & TECH
[x] I have read and agree to the EULA
LEGAL, SALES & TECH
[x] I have read and agree to the EULA
LEGAL, SALES & TECH
[x] I have read and agree to the EULA
identity (KYC)
LEGAL, SALES & TECH
[x] I have read and agree to the EULA
identity (KYC)
LEGAL, SALES & TECH
[x] I have read and agree to the EULA
identity (KYC)
cashflow (accounts / txns)
stEve says: “You cannot afford to take that loan at this rate”
stEve says: “OK, that will work”
LEGAL, SALES & TECH
[x] I have read and agree to the EULA
identity (KYC)
cashflow (accounts / txns)
budget tool
LEGAL, SALES & TECH
[x] I have read and agree to the EULA
identity (KYC)
cashflow (accounts / txns)
budget tool
TECH
2010 – ???
2010 – ???
2010 – POST
2010 – POST
2010 – POST
2010 – POST
2010 – POST
2010 – POST
def login(number: String, otp: String) = { val req = Post( "https://www.zaba.hr/ebank/gradjani/Prijava" , "command" -> "Prijava" , "linkId" -> "446" , "AppIdentifikator" -> "0" , "KioskVersion" -> "0" , "br_tokena" -> number , "otp" -> otp ) sendAndLog(req, "Login POST")}
2010 – POT OF GOLD
2010 – POT OF GOLD
<xml/>
.csv
.xlsx .html
2010 – PO(S)T OF GOLD
def getTransactions(account: ZabaAccount, dates: Interval) = { val req = Post( "https://www.zaba.hr/ebank/gradjani/Gradjani" , "command" -> "PrometiPoRacunu" , "action" -> account.kind , "download" -> "N" , "cboBrojRacuna" -> account.number , "fieldDatumOd" -> dates.start , "fieldDatumDo" -> dates.end , "commandAction" -> "Prijava" ) sendAndLog(req, "Transactions POST for " + account.number)}
2010 – PO(S)T OF GOLD
2010 – PO(S)T OF GOLD
<div class='main'><div class="naslov"> <div class="title">Prometi</div> <div class='podnaslov'> <div class='title'>Prometi po računu <span style='font-weight:normal;'>HR602360000</span>1234567890 (tekući račun) za razdoblje od 05.10.2013. do 05.10.2014.</div> </div> <div id='prometiDospijeli'/><noscript language='JavaScript'><!--var prometiDospijeli=new Array();prometiDospijeli[0]=new Array('07/01/2013','1234567890123456','Pasivna kamata',0.01,null,78.82,'HRK');prometiDospijeli[1]=new Array('08/14/2013','1234567890123451','Uplata redovitog primanja',2677.83,null,4756.65,'HRK');prometiDospijeli[2]=new Array('08/19/2013','1234567890123452','Isplata',null,4750.00,6.65,'HRK');prometiDospijeli[3]=new Array('08/19/2013','1234567890123453','Uplata',20.00,null,26.65,'HRK');prometiDospijeli[4]=new Array('09/06/2013','1234567890123454','Naknada za korištenje - p.a. moderan',null,20.00,6.65,'HRK');prometiDospijeli[44]=new Array('04/01/2014','1234567890123455','Zatezna kamata po nedopuštenom prekoračenju',null,0.10,9.31,'HRK');prometiDospijeli[46]=new Array('04/14/2014','1234567890123456','Osobno primanje isplaćeno u cijelosti',2672.59,null,2661.90,'HRK');prometiDospijeli[57]=new Array('05/26/2014','1234567890123457','E-zaba prijenos - super sport - uplata na račun',null,2.20,0.12,'HRK');createDataTablePrometi('prometiDospijeli',prometiDospijeli);// --></noscript></div><noscript src='./JavaScript/InitPrometiValidation.js?v=1.18.00' language='JavaScript'></noscript>
<br /><br /></div></div></div>
2010 – PO(S)T OF GOLD
<div class='main'><div class="naslov"> <div class="title">Prometi</div> <div class='podnaslov'> <div class='title'>Prometi po računu <span style='font-weight:normal;'>HR602360000</span>1234567890 (tekući račun) za razdoblje od 05.10.2013. do 05.10.2014.</div> </div> <div id='prometiDospijeli'/><noscript language='JavaScript'><!--var prometiDospijeli=new Array();prometiDospijeli[0]=new Array('07/01/2013','1234567890123456','Pasivna kamata',0.01,null,78.82,'HRK');prometiDospijeli[1]=new Array('08/14/2013','1234567890123451','Uplata redovitog primanja',2677.83,null,4756.65,'HRK');prometiDospijeli[2]=new Array('08/19/2013','1234567890123452','Isplata',null,4750.00,6.65,'HRK');prometiDospijeli[3]=new Array('08/19/2013','1234567890123453','Uplata',20.00,null,26.65,'HRK');prometiDospijeli[4]=new Array('09/06/2013','1234567890123454','Naknada za korištenje - p.a. moderan',null,20.00,6.65,'HRK');prometiDospijeli[44]=new Array('04/01/2014','1234567890123455','Zatezna kamata po nedopuštenom prekoračenju',null,0.10,9.31,'HRK');prometiDospijeli[46]=new Array('04/14/2014','1234567890123456','Osobno primanje isplaćeno u cijelosti',2672.59,null,2661.90,'HRK');prometiDospijeli[57]=new Array('05/26/2014','1234567890123457','E-zaba prijenos - super sport - uplata na račun',null,2.20,0.12,'HRK');createDataTablePrometi('prometiDospijeli',prometiDospijeli);// --></noscript></div><noscript src='./JavaScript/InitPrometiValidation.js?v=1.18.00' language='JavaScript'></noscript>
<br /><br /></div></div></div>
2010 – PO(S)T OF GOLD
<div class='main'><div class="naslov"> <div class="title">Prometi</div> <div class='podnaslov'> <div class='title'>Prometi po računu <span style='font-weight:normal;'>HR602360000</span>1234567890 (tekući račun) za razdoblje od 05.10.2013. do 05.10.2014.</div> </div> <div id='prometiDospijeli'/><noscript language='JavaScript'><!--var prometiDospijeli=new Array();prometiDospijeli[0]=new Array('07/01/2013','1234567890123456','Pasivna kamata',0.01,null,78.82,'HRK');prometiDospijeli[1]=new Array('08/14/2013','1234567890123451','Uplata redovitog primanja',2677.83,null,4756.65,'HRK');prometiDospijeli[2]=new Array('08/19/2013','1234567890123452','Isplata',null,4750.00,6.65,'HRK');prometiDospijeli[3]=new Array('08/19/2013','1234567890123453','Uplata',20.00,null,26.65,'HRK');prometiDospijeli[4]=new Array('09/06/2013','1234567890123454','Naknada za korištenje - p.a. moderan',null,20.00,6.65,'HRK');prometiDospijeli[44]=new Array('04/01/2014','1234567890123455','Zatezna kamata po nedopuštenom prekoračenju',null,0.10,9.31,'HRK');prometiDospijeli[46]=new Array('04/14/2014','1234567890123456','Osobno primanje isplaćeno u cijelosti',2672.59,null,2661.90,'HRK');prometiDospijeli[57]=new Array('05/26/2014','1234567890123457','E-zaba prijenos - super sport - uplata na račun',null,2.20,0.12,'HRK');createDataTablePrometi('prometiDospijeli',prometiDospijeli);// --></noscript></div><noscript src='./JavaScript/InitPrometiValidation.js?v=1.18.00' language='JavaScript'></noscript>
<br /><br /></div></div></div>
2010 – POST
2011 – POST
2011 – POST MORTEM
2011 – SELENIUM
2011 – SELENIUM
def doLogin(userCode: String, password: String) = { val UserCode = By.xpath("//input[@id and @name='username']") val Password = By.xpath("//input[@name='password']") val ButtonOk = By.xpath("//button[@name='loginButton']")
findElement(UserCode).sendKeys(userCode) findElement(Password).sendKeys(password) findElement(ButtonOk).click()}
2012 – SELENIUM (34SE)
2012 – SELENIUM (34SE)
Selenium 1.xSelenium
+WebDriver
(2.x)
2012 – SELENIUM 2.X
• Non – JavaScript based
• Dismiss dialogs & alerts
• Upload / Download files (Save as…)
• Firefox, Chrome, Opera, IE, …
2012 – SELENIUM 2.X
• Non – JavaScript based
• Dismiss dialogs & alerts
• Upload / Download files (Save as…)
• Firefox, Chrome, Opera, IE, …
Missing remote session support!
2012 – SELENIUM 2.X
• Non – JavaScript based
• Dismiss dialogs & alerts
• Upload / Download files (Save as…)
• Firefox, Chrome, Opera, IE, …
Missing remote session support!https://github.com/tferega/selenate
2012 - SELENATE
• Runs on Akka remote
• Session support through GUIDs
• M-N session connectivity
2012 - SELENATE
• Runs on Akka remote
• Session support through GUIDs
• M-N session connectivity
Client
Servers(no session IDs)
2012 - SELENATE
• Runs on Akka remote
• Session support through GUIDs
• M-N session connectivity
Client
Servers(no session IDs)
Production client Debug
client
session #3FCArunning on node 4
session #2898running on node 2
2014+ - SELENATE 3.0
• Akka cluster support
• (gossip protocol)
2014+ - SELENATE 3.0
• Akka cluster support
• (gossip protocol)
• Robot & Sikuli support
• OCR through Tesseract
2013 – PATTERN MATCHING
• approx. hundred “lines” for what was previously a simple POST
2013 – PATTERN MATCHING
• approx. hundred “lines” for what was previously a simple POST
• multiple selectors and failovers
• ID -> Name -> Title -> Regex
2013 – PATTERN MATCHING
• approx. hundred “lines” for what was previously a simple POST
• multiple selectors and failovers
• ID -> Name -> Title -> Regex
• countless bugs & special cases
2013 – PATTERN MATCHING
• approx. hundred “lines” for what was previously a simple POST
• multiple selectors and failovers
• ID -> Name -> Title -> Regex
• countless bugs & special cases
Refactoring special cases is DIFFICULT
PHILOSOPHICAL YAMMER
I have seen things you people wouldn't believe...Submit buttons, disabled for days…I watched broken TLS implementations break 20% of all requests.All those bugs will be lost in time, because I didn’t take screenshots.
- Replicator node 7, Selenium Runner
SECURITY THROUGH OBSCURITY
SECURITY THROUGH OBSCURITY
SECURITY THROUGH OBSCURITY
SECURITY THROUGH OBSCURITY
Pro tip:Virtual Frame Buffer
(Xvfb)
SECURITY THROUGH OBSCURITY
SECURITY THROUGH OBSCURITY
SECURITY THROUGH OBSCURITY
SECURITY THROUGH OBSCURITY
LiveConnect?
LIVECONNECT ._.
LIVECONNECT ._.
Legend:
JRE version
Firefox version
Point in time
6u45
7u15
7u51
7u45
7u25
8u20
8u11
FF18 FF21
2011
2012
FF15
2013
FF29
FF40
DANID PSYCHOLOGICAL OPERATIONS DIVISON SECRET WEAPON
DANID PSYCHOLOGICAL OPERATIONS DIVISON SECRET WEAPON
Wuddlecakes
DANID PSYCHOLOGICAL OPERATIONS DIVISON SECRET WEAPON
WuddlecakesFoofieface
DANID PSYCHOLOGICAL OPERATIONS DIVISON SECRET WEAPON
WuddlecakesFoofiefaceWoogycute
LoverschnookumloveSchmoopiecake
WooglecakesCuddlypooPoofcuddle
MoopsiewookieWookumdarlingSnookieKissie
PLENTY MORE WHERE THAT CAME FROM!
CATS ARE USELESS
Alice Bob
CATS ARE USELESS
Alice Bob
CATS ARE USELESS
Alice Bob
Alice BobstEve
CATS ARE USELESS
Alice BobstEve
CATS ARE USELESS
Alice BobstEve
CATS ARE USELESS
LESS RANTS,HIGER SECURITY BY 2015
F.Q.A.
F.Q.A.(Faked Questions from the Audience)
def doLogin(userCode: String, password: String) = { val UserCode = By.xpath("//input[@id and @name='username']") val Password = By.xpath("//input[@name='password']") val ButtonOk = By.xpath("//button[@name='loginButton']")
findElement(UserCode).sendKeys(userCode) findElement(Password).sendKeys(password) findElement(ButtonOk).click()}
F.Q.A.(Faked Questions from the Audience)
def doLogin(userCode: String, password: String) = { val UserCode = By.xpath("//input[@id and @name='username']") val Password = By.xpath("//input[@name='password']") val ButtonOk = By.xpath("//button[@name='loginButton']")
findElement(UserCode).sendKeys(userCode) findElement(Password).sendKeys(password) findElement(ButtonOk).click()}
F.Q.A.(Faked Questions from the Audience)
PhantomJS
CasperJS
SlimerJS
THANX
Questions?
We’re hiring!
