Upload
others
View
19
Download
1
Embed Size (px)
Citation preview
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 1
Roll Your Own Cybersecurity
Tools with Python
George Markowsky
School of Computing & Info Science, UMaine
Department of Computing Security, RIT
http://DrGM.us
Outline
• Introduction
• Why Python?
• Resources
• Testing for Services
• Password Cracking
• Scanning
• Forensics
• Geolocation
• Social Engineering
• Generating SPAM
What This Tutorial Will Do?
• Introduce Python
• Show what a valuable tool Python is in cybersecurity
• Cover some basic concepts from cybersecurity and illustrate them with working code
• Will cover some of the ground that I have covered in previous tutorials, but from a different perspective
• Try to give a "self-contained" presentation
• Give you an appreciation of the power of some of the open source cybersecurity programs
• Give you 10 simple tools that you can experiment with and which can serve as building blocks for future work
10 Programs
1. Very Simple Scanner
2. Simple Password
Cracker
3. Zip File Password
Cracker
4. More Advanced
Scanner
5. Embedding Nmap into
Python
6. Windows Recycle Bin
Viewer
7. Skype Profile Viewer
8. Geolocating IP
addresses
9. Snooping on Hotel
Guests
10. Generating SPAM
Prerequisites
• Some familiarity with computers
• Some familiarity with programming
What This Tutorial Won't Do?
• Make you a hacker!
• Make you a cybersecurity expert
• Teach you all you need to know about security
• Teach you all about cybersecurity law
• Give you common sense
• If it seems that you should not do something,
don't do it
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 2
WARNINGS AND DISCLAIMERS
• You need to be careful when
experimenting with these tools
• You can damage your own computers
and networks
• You can damage other people's
computers and networks
• You can get into serious legal trouble!
WARNINGS AND DISCLAIMERS
• I assume no responsibility for
any damage to your computer or
problems that may result from
your participation in this
workshop
• Please HACK responsibly!
Downloading Stuff
• If you want to download files, please connect
to: http://DrGM.us – there will be a link on
the homepage to the tutorial materials
• No password is needed
• The directories you will see are docs and
programs
• Some other pieces will be available on other
websites
Additional Warnings!
• Some of the things I will tell you about will not work
properly on some of your computers
• The reason for this is that many of you are running
various anti-virus, anti-malware or other protective
software
• This software often recognizes some of these tools
and will not let you download or install them
• There are slight differences among the various
versions of Python, various OSes, etc. so you may
need to play with the code to have it run on a
particular system
Additional Warnings!
• Firewalls, etc. might not let you run some of
the programs
• At the same time, I do not want to instruct
you to disable your protective software,
especially for such an exercise
• You assume all responsibility for participating
in the tutorial session and pursuing this
knowledge further
How Can We Teach Such Stuff?
• The hackers know it
• It's time for the good guys to know it
• Just because people know how to damage
someone or something does not mean that
they will
• People are surprisingly well-behaved – each of
us resists the temptation to wreak havoc many
times a day
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 3
Why Python?
• I like to say that Python is executable pseudo-code
• I contend that many people can read Python programs without "knowing" Python
• We will, of course, be testing this claim tonight
• Python has resources for just about any type of computing and is very readable
• "Ugly code" will not run! Indentation is required!
Hello World!
• The first program in a language is often the
“Hello World” program
• In Python this is very simple
• print “Hello World!”
About Python
Versions
• Python now is generally 2.X or 3.X
• We will use 2.X because everything works with
it
• There is no big advantage to 3.X for beginners
and serious disadvantages
• Many Macs already have Python 2.7x installed
• Can install it, if you don't have it
Introduction to Python
• Go to www.python.org for much
documentation and downloads of Python
• Please use the versions from Python.org or
which are already installed on your computer
• The following material comes from python.org
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 4
About Python
• Python is a remarkably powerful dynamic
programming language that is used in a wide
variety of application domains.
• Python is often compared to Tcl, Perl, Ruby,
Scheme or Java.
Python Plays Well With Others
• Python can integrate with COM, .NET,
and CORBA objects.
• For Java libraries, use Jython, an
implementation of Python for the Java Virtual
Machine.
• For .NET, try IronPython , Microsoft's new
implementation of Python for .NET, or Python
for .NET.
Python Plays Well With Others
• If you find something that Python cannot do, or if you need the performance advantage of low-level code, you can write extension modules in C or C++, or wrap existing code with SWIG or Boost. Python.
• Wrapped modules appear to your program exactly like native Python code. You can also go the opposite route and embed Python in your own application, providing your users with a language they'll enjoy using.
Other Points
• Python runs everywhere
• Python is friendly... and easy to learn
• Python is Open
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 5
Who Uses Python?
• http://www.python.org/about/quotes/
• Python is used successfully in thousands of
real-world business applications around the
world, including many large and mission
critical systems. Here are some quotes from
happy Python users:
YouTube.com
• "Python is fast enough for our site and
allows us to produce maintainable
features in record times, with a minimum
of developers,"
• Cuong Do
• Software Architect
Industrial Light & Magic
• "Python plays a key role in our production pipeline. Without it
a project the size of Star Wars: Episode II would have been
very difficult to pull off. From crowd rendering to batch
processing to compositing, Python binds all things together,"
• Tommy Burnette, Senior Technical Director,
• "Python is everywhere at ILM. It's used to extend the
capabilities of our applications, as well as providing the glue
between them. Every CG image we create has involved
Python somewhere in the process,"
• Philip Peterson, Principal Engineer,
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 6
• "Python has been an important part of Google
since the beginning, and remains so as the
system grows and evolves. Today dozens of
Google engineers use Python, and we're
looking for more people with skills in this
language."
• Peter Norvig
• Director of Search Quality
Python Resources
• Most contemporary Macs and Linux systems
have some version of Python installed
• You can get additional versions of standard
Python and much documentation at
http://python.org
• My website, http://DrGM.us has entire books
on Python that you can download. Look for
the tutorial link on the homepage
Installing & Running Python
• You install Python with an integrated
development environment called IDLE
• Will give you a quick overview of what is
involved
IDLE's TWO WINDOWS
Shell or Command Window –DO NOT WRITE YOUR
PROGRAMS IN THIS TYPE OF WINDOW!
Text Window – WRITE YOUR PROGRAMS IN THIS TYPE
OF WINDOW!
Shell Window Menus
Click on Line to Get Menu to Stay on
Screen in a Separate Window!
Shortcuts
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 7
Text Window Menus
The File, Edit, Windows and
Help Menus are the same for both Windows.
The new menus for text windows
are on the next slide.
You Can Use Python Without Using
IDLE
• On Unix systems and Apple's, some version of
Python is installed
• It is typically installed without IDLE
• You can run it from the terminal window
• Apple OS X is a variant of Unix and you can
access its power through the terminal
application and the command line interface
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 8
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 9
The Command Shell in Windows
• Click on Start in the lower left corner of your
main screen
• A text entry box appears at the bottom of the
pop-up menu
• Enter cmd.exe in it and hit enter and you will
get a shell
• You close shells with the command exit
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 10
Hacking Resources
• Violent Python: A Cookbook for Hackers, Forensic Analysts, Penetration Testers and Security Engineers by TJ. O'Connor, Syngress Press, 2013 – the foundation for this tutorial
– If program names have GM means modified by me, else from this book
• Gray Hat Python: Python Programming for Hackers and Reverse Engineers by Justin Seitz, No Starch Press, 2009
• Various resources on the web
Modifications
• Much of the material comes from Violent Python
with some modifications
• Background material has been supplied
• Programs adapted to Windows
• Programs adapted to GUI rather than command line
• Programs stripped down so they are easier to follow
• The book builds toward more complex programs so
will focus on the simpler examples or parts of
programs
Violent Python
• I highly recommend the book
• Have a special discount from Syngress press courtesy of Steve Eliot of Morgan Kaufman and Syngress
• Get the flyer here or from the docs folder on my website
Anatomy of a Hack (based on a
diagram in Hacking Exposed)
Footprinting
(Google)
Scanning
(Nmap)
Enumeration
(Nmap, Netcat)
Gaining Access
(Exploits, Cain, Netcat)
Escalating Privilege
(Cain, ophcrack,
Wireshark, Netcat)
Pilfering (Cain,
Wireshark, Netcat)
Covering Tracks
(Cain, Netcat)
Creating Back Doors
(Netcat)
Denial of
Service (Cain, Netcat)
Hacking is a process
and not a single event!
Footprinting
• This means to collect information about a
target
• Google is often very useful here
• Social engineering also useful
• Some of the tools can be used for data
collection
Scanning
• Huge topic with many different tools and different
types of scans for each tool
• This is the process of finding out which computers
are active on a network, what services are they
running, what operating system are they running,
etc.
• This information can be researched and exploits
selected for this scanning
• There are all sorts of scanners available
• We will focus on some simple ideas and then
introduce Nmap, probably the most widely used
scanner
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 11
Are Scanners Legal?
• Yes. Scanners are most often designed, written, and distributed by security personnel and developers. These tools are usually given away, via public domain, so that system administrators can check their own systems for weaknesses. However, although scanners are not illegal to possess or use, employing one if you are not a system administrator would meet with brutal opposition from the target host's administrator. Moreover, certain scanners are so intrusive in their probing of remote services that the unauthorized use of them may violate federal or state statutes regarding unauthorized entry of computer networks. This is a matter of some dispute and one not yet settled in law. Therefore, be forewarned. – From Maximum Security
WARNING:
• Do not take scanning activity lightly. If you intend to scan wide ranges of domains, check the laws in your state. Certain states have extremely particular legislation. The wording of such statutes is (more often than not) liberally construed in favor of the prosecution. For example, the state of Washington has provisions for computer trespass. (Wash. Rev. Code Sec. 9A.52 110-120.) If you deploy a scanner that attempts to steal the passwd file (a password file on the UNIX platform located in the directory /ETC), you might actually have committed an offense. – From Maximum Security
Nmap – created by Fyodor
• Nmap (“Network Mapper”) is an open source tool for
network exploration and security auditing. It was
designed to rapidly scan large networks, although it
works fine against single hosts. Nmap uses raw IP
packets in novel ways to determine what hosts are
available on the network, what services (application
name and version) those hosts are offering, what
operating systems (and OS versions) they are
running, what type of packet filters/firewalls are in
use, and dozens of other characteristics.
IP Addresses
• All computers on the Internet must have an IP
address so they can be found
• Some computers are directly on the Internet and
others are connected through intermediate
computers
• The old style of IP addresses are called IPv4 because
it was the fourth version of the addressing schemed
• Addresses consist of 4 numbers running from 0 to
255
• Example: 157.160.255.19 (cnn.com)
IP Addresses
• At one time, it was thought that there were
more than enough IP addresses, but this
scheme "only" gives 4,294,967,296 IP
addresses, fewer than 1 IP address per person
• Since the early folks on the Internet, in
particular US concerns, got very large blocks
of IP addresses, this has caused an IP address
squeeze
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 12
IP Addresses
• The new IPv6 has 2128≅ 3.40 x 1038 distinct
addresses: this is like 3 followed by 38 zeroes
• If we assume a maximum world population of
10 billion people this would give each person
3.4 x 1028 IP addresses
• This is believed to be enough IP addresses
• Most systems still run IPv4, but we are
transitioning – will ignore IPv6 in this tutorial
IP Addresses
• Addresses that look like
– 10.*.*.*
– 172.*.*.*
– 192.*.*.*
• Are not directly on the Internet
• There are some other reserved addresses
Ports
• Services run on a "port" – this is really just a
number to distinguish services on a computer
(e-mail, web, etc.)
• It is easier to use a number than words
especially with different naming conventions
and languages
Common Ports
Port Number Service
20 File Transfer Protocol (FTP)
21 File Transfer Protocol (FTP)
22 Secure Shell (SSH)
23 Telnet Remote Login Service
25 Simple Mail Transfer Protocol (SMTP)
69 Trivial File Transfer Protocol (TFTP)
79 Finger
80 Hypertext Transfer Protocol (HTTP)
110 Post Office Protocol (POP3)
119 Network News Transfer Protocol (NNTP)
143 Internet Message Access Protocol (IMAP)
161 Simple Network Management Protocol (SNMP)
443 Secure HTTP (HTTPS)
Source: RFC 1700 and Descendants
Acknowledgement
• Great textbook on
networking
• Have used some slides
from this book
Protocol Hierarchies
Tanenbaum
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 13
Everyday Protocol Hierarchy
CEO – Proposal
Secretary – Dictation,
Typing
Mailroom
Clerk – Envelope
Post Office,
FedEx, UPS, etc.
Post Office,
FedEx, UPS, etc.
Mailroom
Clerk – Envelope
Secretary – Scanning,
Delivery
CEO – Proposal
Protocol Hierarchies (3)
• Example information flow supporting virtual
communication in layer 5.
Tanenbaum
Banner Grabbing
• Banner grabbing means connecting to remote
applications and observing the output
• Can be done by automated tools, but can also
be done manually with netcat, telnet or web
browser
• Commonly practiced against limited set of
ports such as the ones on the previous slide
More on Banner Grabbing
• Often generate errors deliberately to see how the
machine will respond
• Programmers are often sloppy about error
handling and programs break or give away the
store
• The banners you get can vary quite a bit from
system to system
• Hacking is a lot like detective work – you patiently
collect the clues and formulate a plan of action
Banner Grabbing
• Let's look at a Python program that begins to
get at nmap functionality!
• Very primitive program that just gets us
started in communicating with a server
• In Python # is used to indicate a comment
• Once Python sees # it ignores the rest of the
line
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 14
P01-ScanGM.pyimport socket # Imports networking library
def retBanner(ip, port):
try: # program will not crash if the following fails
socket.setdefaulttimeout(2) # set time limit
s = socket.socket() # prepare for communication
s.connect((ip, port)) # communicate
banner = s.recv(1024) # see if you get anything
return banner
except: # action in case of failure
return "Exception"
portList = [21,22,25,80,110,443]
ipList = [21,22,23]
for byte in ipList:
ip = ‘130.111.197.' + str(byte) # Find something that works
for port in portList: # Some UMaine IP addresses
banner = retBanner(ip, port)
print ip, port, banner
P01-ScanGM.py Output
XXX.XXX.XXX.21 21 220 ProFTPD 1.3.2a Server (FTP server) [::ffff:XXX.XXX.XXX.21]
…
XXX.XXX.XXX.21 22 SSH-2.0-Sun_SSH_1.1.1
XXX.XXX.XXX.21 25 Exception
XXX.XXX.XXX.21 80 Exception
XXX.XXX.XXX.21 110 +OK POP3 gandalf 2007e.104 server ready
…
XXX.XXX.XXX.21 443 Exception
XXX.XXX.XXX.22 21 Exception
XXX.XXX.XXX.22 22 SSH-2.0-Sun_SSH_1.1.1
XXX.XXX.XXX.22 25 Exception
XXX.XXX.XXX.22 80 Exception
XXX.XXX.XXX.22 110 Exception
XXX.XXX.XXX.22 443 Exception
XXX.XXX.XXX.23 21 Exception
XXX.XXX.XXX.23 22 SSH-2.0-OpenSSH_5.6
…
XXX.XXX.XXX.23 25 Exception
XXX.XXX.XXX.23 80 Exception
XXX.XXX.XXX.23 110 Exception
XXX.XXX.XXX.23 443 Exception
Comments on P01
• Your results will depend on the machines you
scan
• If they do not have services available you will
not get anything interesting
• Do not do this on machines you do not own
• Much depends on fire walls, etc.
• Just wanted to convey the main ideas
Password Cracking
• Most of security is handled through passwords
• Having a password is like having a key to a
door
• Some passwords are like having the keys to
the kingdom
• I will now cover some background dealing
with passwords
Implementing Passwords
• The most straightforward implementation of
passwords would be to have a file containing the
passwords along with the user ID
• Initially, these were in plaintext
• The big problem with that is once someone got to
the file, that person would be able to use the
passwords and get into people's accounts
• System administrators would know all passwords
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 15
Implementing Passwords
• The next step was to hash the passwords and
keep only the hash around
• MOST MODERN SYSTEMS DO NOT KEEP THE
PASSWORD AND THE SYSTEM ADMINISTRATOR
CANNOT TELL YOU YOUR PASSWORD
• When you log in, the system takes your password
and hashes it
• It compares the hash from your login to the
stored hash – if they are the same you are in!
What is a Hash?
• A hash is a way of assigning a number (string)
to arbitrary strings
• For example, if we work just with upper case
letters, we can assign the numbers 1..26 to
the letters A..Z and 27 to a blank
• Our hash could be adding up the values for
the letters in a word, dividing by 100 and using
the remainder as a hash
A Simple Hash
• For example, HELLO would hash to 52
• H → 8
• E → 5
• L → 12
• L → 12
• O → 15
• Sum = 52
• 52 divided by 100 gives a remainder of 52
Why Is This Hash Bad?
• If you put the letters in any order you get the same hash: ELLOH, HELOL, LEHOL, etc. all hash to 52
• Only possible values are 0..99, so not hard to try all hashes. Easy to generate words that have a particular hash
• The phrase A WORLD hashed to 0, so HELLO AWORLD would hash to 52
• Many collisions, i.e., people entering the wrong password might still get in
Some Other Hash Values
• The hash of HELLO WORLD is: 51
• The hash of WHAT A FUNNY PERONS is: 1
• The hash of WHAT A FUNNY PERSON is: 1
• The hash of WORLD HELLO is: 51
• The hash of WHY ARE YOU DOING THIS is: 54
• The hash of WHO ARE YOU is: 85
• The hash of WHAT HAPPENED is: 48
• The hash of WHAT HAPPENED TO ME is: 55
• The hash of LET ME HELP YOU is: 38
• The hash of SEE THE HIPPO is: 80
• The hash of SEE THE ELEPHANT is: 97
• The hash of SEEC THE ELEPHANT is: 0
• The hash of WORLD is: 72
• The hash of A WORLD is: 0
Password Hash Functions
• Good hash functions are hard to construct
• Some look good, but eventually people figure
out how to attack many of them
• Note that you do not have to recover the
password, just find something that has the
same hash!
• Some widely used hashes are MD5, SHA1,
SHA2, DES, LM and NTLM
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 16
Setting Up A User Account• User is given a user ID by the system
administrator
• User is asked to input a regular password either at the time the account was set up and after using the account with a temporary password
User Password Hasher Password Hash
Discarded Stored
SALT
Logging In
• The user enters a
password, the
system hashes it
and compares to
the password hash
stored for that user
• If the new hash
matches the stored
hash the user is in
User ID
Password
IDs Salts Passwo
rd
Hashes
… … …
IDn Sn Hashn
IDn+1 Sn+1 Hashn+1
IDn+2 Sn+2 Hashn+2
IDn+3 Sn+3 Hashn+3
IDn+4 Sn+4 Hashn+4
… … …
Hasher
Hash Match?
Where do the IDs, the salts
and hashes live?
Where Do Passwords Live?
Windows
• Security Accounts Manager
(SAM) database:
c:\win????\system32\config
• Active Directory database
file – that can be stored
locally or distributed
• Could be in backup of SAM
file, repair disks and
emergency disks, or
c:\win????\repair,
Linux (Unix)
• /etc/passwd – readable by everyone on the system –no longer contains password hashes
• /etc/shadow – readable by system & root – contains password hashes
• /etc/security/passwd – file name may vary – sys & root
• Other files are possible
Where Do Passwords Live?
Mac
• Depends on the version
• Earlier implementations of
OS X used the netinfo
database
• Ignored many lessons
learned
• Later versions improved
• Need to do research on
version interested in
Other Operating Systems
• You need to study the particular operating system
• Remember that Google search is one of your primary hacking tools
• Hackers are constantly getting information from the web
• You need to get information about your own systems!
Some Assumptions
• The system needs to store the salt somewhere
• The salt needs to be stored in unencrypted form, since if you encrypt it you will need to store another password, etc.
• In short, something needs to be stored unencrypted
• Often the salt is stored together with the password hash, so you must protect the password file
Sample Files
passwords.txtvictim: HX9LLTdc/jiDE: 503:100:Iama Victim:/home/victim:/bin/sh
root: DFNFxgW7C05fo: 504:100: Markus Hess:/root:/bin/bash
Salts Password Hashes
dictionary.txtapple
orange
egg
lemon
grapes
secret
strawberry
password
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 17
#!/usr/bin/python #runs Python on unix systems
import crypt # Only works for unix systems
def testPass(cryptPass): # Single parameter cryptPass
salt = cryptPass[0:2] # First two characters are salt
print cryptPass, salt
dictFile = open('dictionary.txt', 'r') # get words
for word in dictFile.readlines(): # test all words
word = word.strip('\n') # remove new line chars
cryptWord = crypt.crypt(word, salt) # call crypt
if cryptWord == cryptPass: # if found print & halt
print '[+] Found Password: ' + word + '\n'
return # stop function testPass
print '[-] Password Not Found.\n' # announce failure
return # stop function testPass
P02-passwordCrack.py
def main():
passFile = open('passwords.txt') # get password hash
for line in passFile.readlines(): # run through file
if ':' in line: # process if the line has data
user = line.split(':')[0] # first item is user
cryptPass = line.split(':')[1].strip(' ')#hash
print '[*] Cracking Password For: ' + user
testPass(cryptPass)# try to crack it
if __name__ == '__main__': # Standalone or module
main()
P02-passwordCrack.py
[*] Cracking Password For: victim
HX9LLTdc/jiDE HX
[+] Found Password: egg
[*] Cracking Password For: root
DFNFxgW7C05fo DF
[-] Password Not Found.
Output
P03-Zipfile Password Cracker
• You can protect many types of individual files, with passwords
• This includes zip files and pdf files
• You can apply the same password cracking algorithm to these
• The next program also illustrates the use of threads to distribute the work
• Can distribute across cores, clusters or botnets!
Setting the Stage
The ResultC:\>3-zipCrack.py
usage %prog -f <zipfile> -d <dictionary>
C:\>3-zipCrack.py
-f evil.zip -d dictionary.txt
[+] Found password secret
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 18
P03-zipCrack.py
#!/usr/bin/python
import zipfile # package for dealing with zip files
import optparse # used for passing command line options
from threading import Thread # for creating threads
def extractFile(zFile, password):# try password on file
try: # guard against failure
zFile.extractall(pwd=password)# try to unzip
print '[+] Found password ' + password + '\n'
except:
pass
P03-zipCrack.py
def main():
parser = optparse.OptionParser("usage %prog "+\#cont.
"-f <zipfile> -d <dictionary>") # set up parser
parser.add_option('-f', dest='zname', type='string',\
help='specify zip file') # add an option & help
parser.add_option('-d', dest='dname', type='string',\
help='specify dictionary file') # add option & help
(options, args) = parser.parse_args()
if (options.zname == None) | (options.dname == None):
print parser.usage # give correct usage
exit(0) # exit with error code 0
else:
zname = options.zname # filespec of zip file
dname = options.dname # filespec of dictionary
P03-zipCrack.py
zFile = zipfile.ZipFile(zname)# open zip file
passFile = open(dname) # open dictinary
for line in passFile.readlines(): # read through words
password = line.strip('\n')# get rid of \n
t = Thread(target=extractFile,\
args=(zFile, password))# set thread up
t.start() # run it
if __name__ == '__main__': # usual for modules
main()
Cain – a more serious password cracker
Note options for using the
dictionary attack.
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 19
Penetration Testing
• The idea here is to try to get into a system and
see how good the defenses are
• One of the chief tools here is scanning, which
we already discussed in connection with our
first program
• We now quickly present two additional
enhancements for scanning
P04-Improved ScannerC:\>P04-portScan.py -H 157.166.255.17 -p 80
[+] Scan Results for: 157.166.255.17
[+] 80/tcp open
[+] <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>501 Method Not Implemented</t
C:\>P04-portScan.py –H cnn.com -p 80
[+] Scan Results for: 157.166.255.19
[+] 80/tcp open
[+] <!DOCTYPE html >
<html>
<head>
<title>Page Not Found</title>
<meta name="viewport" content="widt
CNN.COM
CNN.COM
P04-Improved Scannerimport optparse
from socket import *
from threading import *
screenLock = Semaphore(value=1)# set up for screen share
def connScan(tgtHost, tgtPort):
try:
connSkt = socket(AF_INET, SOCK_STREAM)# set up
connSkt.connect((tgtHost, tgtPort)) # connect
connSkt.send('ViolentPython\r\n') # transmit
results = connSkt.recv(100) # receive limited
screenLock.acquire() # lock screen
print '[+] %d/tcp open' % tgtPort # display
print '[+] ' + str(results) # display
except:
screenLock.acquire() # lock screen
print '[-] %d/tcp closed' % tgtPort # display
finally:
screenLock.release() # unlock screen
connSkt.close() # close socket
P04-Improved Scannerdef portScan(tgtHost, tgtPorts):
try:
tgtIP = gethostbyname(tgtHost)# IPv4 address
except:
print "[-] Can’t resolve '%s': Unknown host" % tgtHost
return
try:
tgtName = gethostbyaddr(tgtIP)# get more info
print '\n[+] Scan Results for: ' + tgtName[0]
except:
print '\n[+] Scan Results for: ' + tgtIP
setdefaulttimeout(1)
for tgtPort in tgtPorts:
t = Thread(target=connScan,\ #set up thread
args=(tgtHost,int(tgtPort)))
t.start() # start thread
P04-Improved Scannerdef main():
parser = optparse.OptionParser('usage %prog '+\
'-H <target host> -p <target port>')
parser.add_option('-H', dest='tgtHost', type='string',\
help='specify target host')
parser.add_option('-p', dest='tgtPort', type='string',\
help='specify target port[s] separated by comma')
(options, args) = parser.parse_args()
tgtHost = options.tgtHost
tgtPorts = str(options.tgtPort).split(',')
if (tgtHost == None) | (tgtPorts[0] == None):
print parser.usage
exit(0)
portScan(tgtHost, tgtPorts)
if __name__ == '__main__':
main()
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 20
Nmap
• Most likely this is the most widely used
scanner
• Very powerful and very good
• Has its own scripting language
Scanning the
Local network
Computers on
the Network
Many
Different
Views
Details for
Selected Host
Nmap & Python
• Who wants to learn yet another language?
• Can call Nmap from Python
• Must have Nmap installed and accessible!
• The Python package you need to enable using
Nmap can be found at:
http://xael.org/norman/python/python-
nmap/
• Be sure to get the correct version
P5-Nmap
C:\>P05-nmapScan.py -H 10.0.0.115 -p 80
[*] 10.0.0.115 tcp/80 open
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 21
P5-Nmap
import nmap
import optparse
def nmapScan(tgtHost,tgtPort):
nmScan = nmap.PortScanner()
nmScan.scan(tgtHost,tgtPort)
state=nmScan[tgtHost]['tcp'][int(tgtPort)]['state']
print "[*] " + tgtHost + " tcp/"+tgtPort +" "+state
def main():
# parser stuff omitted …
for tgtPort in tgtPorts:# These come from the cmd line
nmapScan(tgtHost, tgtPort)
if __name__ == '__main__':
main()
Sequences
• Sequences of objects:
– strings
– lists
– tuples
• Many of the properties that strings have are
also possessed by the other sequences
Sequences• A string is a sequence of characters
– Designated by matching quote marks (' or ")
– Cannot be modified
• A list is a sequence of objects
– Designated by square brackets
– Can be modified
– [1, 2, '1', [3, 4], 'Hello']
• A tuple is like a list, but cannot be modified
– Designated by parentheses
– (1, 2, '1', (3, 4), 'Hello')
Strings
• Will focus on strings
• A string is a sequence of characters
• You can access the characters one by one as follows
• st = "Hello, World!"
• st[0] = "H", st[1] = "e", …, st[12] = "!"
• The number inside the square brackets is called the index
• Python always starts the numbering at 0!
The String Data Type
• The most common use of personal computers
is word processing.
• Text is represented in programs by the string
data type.
• A string is a sequence of characters enclosed
within quotation marks (") or apostrophes (').
The String Data Type
>>> str1="Hello"
>>> str2='spam'
>>> print str1, str2
Hello spam
>>> type(str1)
<type 'str'>
>>> type(str2)
<type 'str'>
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 22
The String Data Type
• We can access the individual characters in a string through indexing.
• The positions in a string are numbered from the left, starting with 0.
• The general form is <string>[<expr>], where the value of expr determines which character is selected from the string.
The String Data Type
>>> greet = "Hello Bob"
>>> greet[0]
'H'
>>> print greet[0], greet[2], greet[4]
H l o
>>> x = 8
>>> print greet[x - 2]
B
H e l l o B o b
0 1 2 3 4 5 6 7 8
The String Data Type
• In a string of n characters, the last character is at position n-1 since we start counting with 0.
• We can index from the right side using negative indexes.
>>> greet[-1]'b'>>> greet[-3]'B'
H e l l o B o b
0 1 2 3 4 5 6 7 8
The String Data Type
• Indexing returns a string containing a single
character from a larger string.
• We can also access a contiguous sequence of
characters, called a substring, through a
process called slicing.
The String Data Type
• Slicing:
<string>[<start>:<end>]
• start and end should both be integers
• The slice contains the substring beginning at
position start and runs up to but doesn’t
include the position end.
The String Data Type
>>> greet[0:3]'Hel'>>> greet[5:9]' Bob'>>> greet[:5]'Hello'>>> greet[5:]' Bob'>>> greet[:]'Hello Bob'
H e l l o B o b
0 1 2 3 4 5 6 7 8
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 23
The String Data Type
• If either expression is missing, then the start or the end of the string are used.
• Can we put two strings together into a longer string?
• Concatenation “glues” two strings together (+)
• Repetition builds up a string by multiple concatenations of a string with itself (*)
len
• The built-in function len gives the length of a
string or other sequences as we will see
• len("Hello, World!") = 13
• Remember that the length of a string is always
one more than the last index since indices
start at 0
• Negative indices start at the back of the string
st[-1] is the last character
The String Data Type
• The function len will return the length of a string.
>>> "spam" + "eggs"
'spameggs'
>>> "Spam" + "And" + "Eggs"
'SpamAndEggs'
>>> 3 * "spam"
'spamspamspam'
>>> "spam" * 5
'spamspamspamspamspam'
>>> (3 * "spam") + ("eggs" * 5)
'spamspamspameggseggseggseggseggs'
Empty String
• The empty string is the only string of length 0
• It is commonly written as "" or ''
• Do not confuse two consecutive single quotes
with one double quote
• Note that " " and ' ' are not the empty string,
they are the one character string consisting of
a single space character
Exercise
• What is len("COS 125")?
• 7
• If A = "Hello, World!", what
is A[-3]?
• "l"
• What is len("COS 125"+A)?
• 20
• If B = "COS 125" + A, what
is B[-15]?
• "2"
• What is A[0] + A[7]?
• "HW"
String Slices
• st[[start]:[stop+1][:step]]
• What does this mean?
• If st is a string, you can select some substring
by picking a starting point
• one more than the stopping point (like range)
• and an increment if you want one
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 24
Exercise
• F = 'watermelon'
• F[3:8] =
• 'ermel'
• F[-4:-2] =
• 'el'
• F[:5] =
• 'water'
• F[5:] =
• 'melon'
• F[::-1] =
• 'nolemretaw'
The String Data Type
Operator Meaning
+ Concatenation
* Repetition
<string>[] Indexing
<string>[:] Slicing
len(<string>) Length
For <var> in <string> Iteration through characters
Strings are Immutable
• That means that you cannot change a string
once you define it
• In particular, the following is an error
• H = 'hello'
• H[0] = 'j'
Strings and Lists
• It turns out that strings are really a special kind of sequence, so these operations also apply to lists!
>>> [1,2] + [3,4]
[1, 2, 3, 4]
>>> [1,2]*3
[1, 2, 1, 2, 1, 2]
>>> grades = ['A', 'B', 'C', 'D', 'F']
>>> grades[0]
'A'
>>> grades[2:4]
['C', 'D']
>>> len(grades)
5
Strings and Lists
• Strings are always sequences of characters,
but lists can be sequences of arbitrary values.
• Lists can have numbers, strings, or both!
myList = [1, "Spam ", 4, "U"]
• Lists can have other lists:
myList = [1, 3.14, [1, 2] ]
Tuples
• Like lists, but not mutable
• Will have more to say about sequences later in
the course
• Numbering for strings, lists and tuples begins
with 0
• Can use negative indices for them as well
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 25
range Generates a List
>>> range(10)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> range(1,10)
[1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> range(1,10,2)
[1, 3, 5, 7, 9]
>>>
• Note different
numbers of
parameters
• [(start),limit,(step)]
• NEVER
REACHES limit
• Default start is
0
Forensics
• Often this means exploring a cyber environment to find items of legal interest
• Here we will give a few examples of some Python programs that can find material of interest on a computer
• First we show how to search through the recycle bin (cyber dumpster diving)
• Best to run this as an administrator so you can go anywhere
C:\ RECYCLER > reg query “HKEY_LOCAL_MACHINE\ SOFTWARE\ Microsoft\ Windows NT\
CurrentVersion\ ProfileList\ S-1-5-21-1275210071-1715567821-725345543-1005” /v
ProfileImagePath
! REG.EXE VERSION 3.0
HKEY_LOCAL_MACHINE\ SOFTWARE\ Microsoft\ Windows NT\ CurrentVersion\ ProfileList \S-1-
5-21-1275210071-1715567821-725345543-1005 ProfileImagePath REG_EXPAND_SZ
%SystemDrive%\ Documents and Settings\ alex
P06 – Windows Recycle Bin Reader[*] Listing Files For User: markov
[+] Found File: $I3QNQA6.log
[+] Found File: $I4HH8OA.git
[+] Found File: $I4PQWO6.zip
[+] Found File: $IJI4LHQ.git
[+] Found File: $IKTD9DS.log
[+] Found File: $ILOLXQO
[+] Found File: $IQEQN8A
[+] Found File: $IXSJG8E.log
[+] Found File: $R3QNQA6.log
[+] Found File: $R4HH8OA.git
[+] Found File: $R4PQWO6.zip
[+] Found File: $RJI4LHQ.git
[+] Found File: $RKTD9DS.log
[+] Found File: $RLOLXQO
[+] Found File: $RQEQN8A
[+] Found File: $RXSJG8E.log
[+] Found File: desktop.ini
Traceback (most recent call last):
File "C:/Users/markov/Dropbox/WorldComp/2013/Tutorial/Web/programs/P06-
dumpRecycleBin.py", line 36, in <module>
main()
File "C:/Users/markov/Dropbox/WorldComp/2013/Tutorial/Web/programs/P06-
dumpRecycleBin.py", line 33, in main
findRecycled(recycledDir)
File "C:/Users/markov/Dropbox/WorldComp/2013/Tutorial/Web/programs/P06-
dumpRecycleBin.py", line 25, in findRecycled
files = os.listdir(recycleDir + sid)
WindowsError: [Error 5] Access is denied: 'C:\\$Recycle.Bin\\S-1-5-21-973107526-
766114380-1998031910-1001/*.*'
Run as non-
administrator
before emptying
recycle bin
P06 – Windows Recycle Bin Reader
Run as administrator
after emptying recycle
bin
C:\>P06-dumpRecycleBin.py
[*] Listing Files For User: markov
[+] Found File: desktop.ini
[*] Listing Files For User: S-1-5-21-973107526-766114380-1998031910-1001
[+] Found File: desktop.ini
[*] Listing Files For User: S-1-5-21-973107526-766114380-1998031910-500
[+] Found File: desktop.ini
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 26
P06 – Windows Recycle Bin Readerimport os
from _winreg import * # Library for Windows
def sid2user(sid):
try:
key = OpenKey(HKEY_LOCAL_MACHINE,
"SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList"
+ '\\' + sid) # Registry key
(value, type) = QueryValueEx(key, 'ProfileImagePath')
user = value.split('\\')[-1]
return user
except:
return sid
def returnDir():
dirs=['C:\\Recycler\\','C:\\Recycled\\','C:\\$Recycle.Bin\\']
for recycleDir in dirs:
if os.path.isdir(recycleDir):
return recycleDir
return None
What’s with \\?
P06 – Windows Recycle Bin Reader
def findRecycled(recycleDir):
dirList = os.listdir(recycleDir)
for sid in dirList:
files = os.listdir(recycleDir + sid)
user = sid2user(sid)
print '\n[*] Listing Files For User: ' + str(user)
for file in files:
print '[+] Found File: ' + str(file)
def main():
recycledDir = returnDir()
findRecycled(recycledDir)
if __name__ == '__main__':
main()
Databases on Disks
• Many programs create profiles of one sort or
another
• For example, Skype and Firefox create profiles
• Python has the ability to read these databases
through its SQL3lite module
• We will show a brief example of the sorts of
information that can be found in a Skype
profile
SQL3lite Database Viewer
• This might be more comprehensible if you first
download a SQL3lite database viewer
• A free viewer is available at http://sourceforge.net/projects/sqlitebrowser/
• Here is partial view of what is stored in the profile
• As you can imagine this could be a rich source of
information
Looking at a Skype Profile
Output of Skype Profile ViewerC:\>P07-skypeParse.py -p skype_profile
[*] -- Found Account --
[+] User : Victor Python
[+] Skype Username : victor.python
[+] Location : Normandy,FR
[+] Profile Date : 2012-06-20 18:53:49
[*] -- Found Contact --
[+] User : Victor Python
[+] Skype Username : victor.python
[+] Birthday : 19760704
[*] -- Found Contact --
[+] User : Echo / Sound Test Service
[+] Skype Username : echo123
[*] -- Found Calls --
[+] Time: 2012-06-20 18:56:55 | Partner: +118008828880
[*] -- Found Messages --
Time: 2012-06-20 18:55:19 To echo123: Testing, 1.2.3....
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 27
P07 – Skype Profile Viewer
P07 – Skype Profile Viewer
P07 – Skype Profile Viewer
P07 – Skype Profile Viewer
P07 – Skype Profile Viewer
P07 – Skype Profile Viewer
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 28
Analyzing Network Traffic
• Python can be used to analyze network traffic
both for offense and defense
• One problem that we can discuss is finding the
physical location of an IP address
• Used by companies and news services to
target delivery
• Used when GPS data is not available
• For this we need some data
http://dev.maxmind.com/geoip/legacy/geolite/
Gives two large CSV files which are basically somewhat
structured text files – (Comma, Separated, Values)
The files are
GeoLiteCity-Blocks.csv (66.223 MB -2,020,169 blocks) and
GeoLiteCity-Location.csv (21.518 MB – 421,274 locations)
Structure of GeoLiteCity-Blocks.csvCopyright © 2011 MaxMind Inc. All Rights Reserved.
startIpNum endIpNum LocId
"16777216" "16777471" "17"
"16777472" "16778239" "49"
"16778240" "16778751" "14409"
Structure of GeoLiteCity-Location.csv
Copyright (c) 2012 MaxMind LLC. All Rights Reserved.
locId Country Region City Postal Code Latitude Longitude Metro
Code
Area Code
1 O1 0.0000 0.0000
2 AP 35.0000 105.0000
3 EU 47.0000 8.0000
… … … … … … … … …
Idea Behind the Program
• Use the first table to find the block that the IP
address belongs to and then use the second
table to find its country, latitude and longitude
• The problem is that the tables are so large
• In principle could use a spreadsheet program,
but many can’t handle such large files reliably
• Database version would be faster, but less
transparent
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 29
P08-geolocateIpGM.py
Please enter an ip address. 157.166.226.25
2644959769
"30348" "Atlanta" "GA" "US"
Metro Code = 524 Area Code = 678
Longitude = -84.3880 Latitude = 33.7490
****************************
Please enter an ip address. 130.111.197.21
2188363029
"04469" "Orono" "ME" "US"
Metro Code = 537 Area Code = 207
Longitude = -68.6334 Latitude = 45.0028
P08-geolocateIpGM.py
P08-geolocateIpGM.py
P08-geolocateIpGM.py
Faster Algorithms such as Binary
Search are Possible
P08-geolocateIpGM.py
Result for
CNN.com
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 30
Result for
UMaine
Correct
Location
Of course,
this can be
automated
Wireless Eavesdropping
• In our earlier discussions we mentioned
various protocols
• Many protocols do not use encryption so
when then send information it is not encoded
• Anyone who intercepts the transmission can
read
• Some protocols even send passwords in the
clear
P09 – Hotel Snooper
• This program uses other programs that we
don't have time to discuss, so I will just show
it to you and highlight some of the features
• TJ. O'Connor talks about a hotel that
transmitted guest names and room numbers
in the clear – you could go to the restaurant
and use that information to charge a meal to
someone else's room
P09 – Hotel Snooper
P09 – Hotel Snooper
P09 – Hotel Snooper
Roll Your Own Cybersecurity Tools With
Python
7/23/2013
George Markowsky WorldComp'13 Tutorial
[email protected] http://DrGM.us 31
Social Engineering
• A serious problem
• If you fool people you can get all sorts of
information out of them
• Social engineering is often done via e-mail
P10 – Generating SPAM
import smtplib
from email.mime.text import MIMEText
def sendMail(user,pwd,to,subject,text):
msg = MIMEText(text)
msg['From'] = user # You can change this to anything!
msg['To'] = to # Also not that important
msg['Subject'] = subject
try:
smtpServer = smtplib.SMTP('smtp.gmail.com', 587)
print "[+] Connecting To Mail Server."
smtpServer.ehlo()# Hackers don't use gmail
print "[+] Starting Encrypted Session."
smtpServer.starttls()
smtpServer.ehlo()
print "[+] Logging Into Mail Server."
smtpServer.login(user, pwd)# Need real ID & Pwd
print "[+] Sending Mail."
smtpServer.sendmail(user, to, msg.as_string())
smtpServer.close()
print "[+] Mail Sent Successfully."
except:
print "[-] Sending Mail Failed."
P10 – Generating SPAM
user = 'username' # Use ID that will work
pwd = 'password' # Use pwd that will work
sendMail(user, pwd, '[email protected]',\
'Re: Important', 'Test Message') # create real address
Wrap-Up
• I hope this tutorial has given you some insight into cybersecurity and how you can explore it using Python
• I strongly recommend Violent Python by TJ. O’Connor if you want to learn more about this area
• You can get a discount from the publisher if you user the flyer available here and on the website
Thanks for Attending
Contact Information:
George Markowsky
+1-207-581-3940