Perl/DBI - accessing databases from Perl

  • Published on
    11-Jan-2016

  • View
    46

  • Download
    2

Embed Size (px)

DESCRIPTION

Perl/DBI - accessing databases from Perl. Dr. Andrew C.R. Martin martin@biochem.ucl.ac.uk http://www.bioinf.org.uk/. Aims and objectives. Understand the need to access databases from Perl Know why DBI? Understand the structure of DBI - PowerPoint PPT Presentation

Transcript

  • Perl/DBI - accessing databases from PerlDr. Andrew C.R. Martinmartin@biochem.ucl.ac.ukhttp://www.bioinf.org.uk/

  • Aims and objectivesUnderstand the need to access databases from PerlKnow why DBI?Understand the structure of DBIBe able to write a Perl/DBI script to read from or write to a databasePRACTICAL: write a script to read from a database

  • Why access a database from Perl?

  • Why access a database from Perl?Web browserWeb serverCGI can extract parameters sent with the page request

  • Why access a database from Perl?Populating databasesNeed to pre-process and re-format data into SQLIntermediate storage during data processingReading databasesGood database design can lead to complex queries (wrappers)Need to extract data to process it

  • Why use DBI?

  • Why Perl/DBI?Many relational databases available.Commercial examples:OracleDB/2SQLServerSybaseInformixInterbaseOpen source examples:PostgreSQLmySQL

  • Why Perl/DBI?Databases use a common query language: structured query language (SQL)

    Queries can easily be ported between different database softwareMinor variations in more advanced featuresProprietary extensions

  • Why Perl/DBI?Can call command-line interface from within your program.

    $result = `psql -tqc SELECT * FROM table `;

    @tuples = split(/\n/, $result);foreach $tuple (@tuples){ @fields = split(/\|/, $tuple);}Inefficient: new process for each database access

  • Why Perl/DBI?Databases generally provide own APIs to allow access from programming languagese.g. C, Java, PerlProprietary APIs all differVery difficult to port software between databases Standardized APIs have thus become available

  • Why Perl/DBI?Perl/DBI is the standardized API for PerlEasy to port Perl scripts from one database to another

  • DBI and ODBCODBC (Open DataBase Connectivity)Consortium of vendors in early 1990sSQL Access GroupOctober 1992 & 1993, draft standard:Call Level Interface (CLI - an API)Never really adoptedMicrosoft embraced and extended it to create ODBC

  • DBI and ODBCDBPerl designed as database interface specifically for Perl4September 1992 (i.e. pre-ODBC)Just before release, Perl5 announced with OO facilitiesDBPerl modified to support OO and loosely modelled on CLI standardThis became DBI

  • DBI and ODBC

    ODBCDBI

  • DBI and ODBCThere is an ODBC driver for DBI DBI can be used to access any ODBC database

  • The structure of DBI

  • DBI ArchitectureDBI

  • DBI ArchitectureMulti-layer designPerl scriptDBIDBDDatabase APIRDBMSDatabaseIndependentDatabaseDependent

  • DBI ArchitectureDBD::OracleDBD::OracleDBD::OraclePerlScriptPerlScriptPerlScriptDBIDBD::mysqlDBD::OracleDBD::PgOraclemySQLPostgreSQL

  • DBI ArchitectureReturns a list of installed (DBD) drivers@drivers = DBI->available_drivers();

  • DBI ArchitectureDBD::OracleDriver HandleDBD::pgDriver HandleDatabaseHandleDatabaseHandleDatabaseHandleStatementHandleStatementHandleStatementHandleStatementHandleStatementHandleStatementHandleStatementHandle

  • DBI ArchitectureDriver Handles

    References loaded driver(s)One per driver

    Not normally referenced in programsStandard variable name $drh

  • DBI ArchitectureDatabase HandlesCreated by connecting to a database

    References a database via a driverCan have many per database (e.g. accessing different user accounts)Can access multiple databasesStandard variable name $dbh$dbh = DBI->connect($datasource, ... );

  • DBI ArchitectureStatement HandlesCreated by preparing some SQL

    Can have many per database handle (e.g. multiple queries)Standard variable name $sth

    $sth = $dbh->prepare($sql);

  • DBI ArchitectureDBD::pgDriver HandleDatabaseHandleStatementHandle$dbh = DBI->connect($datasource, ... );$sth = $dbh->prepare($sql);

  • SQL Preparationprepare()Perl scriptDBI and DriverDatabase$sthfetchrow_array()ExecuteStatement

  • Writing Perl/DBI scripts

  • Accessing DBI from PerlMust have the DBI package and appropriate DBD package installedDBD::Oracle, DBD::Pg, DBD::mysql, etc.use DBI;

  • Data sources

    $dbname = mydatabase;$dbserver = dbserver.cryst.bbk.ac.uk;$dbport = 5432;

    $datasource = dbi:Oracle:$dbname;$datasource = dbi:mysql:database=$dbname;host=$dbserver;$datasource = dbi:Pg:dbname=$dbname;host=$dbserver;port=$dbport;

    $dbh = DBI->connect($datasource, $username, $password);

  • Username and password$username and $password also optionalOnly needed if you normally need a username/password to connect to the database.Remember CGI scripts run as a special web-server user. Generally, nobody or apache. Database must allow access by this useror specify a different username/password$dbh = DBI->connect($datasource, $username, $password);

  • SQL commands with no return valueSQL commands other that SELECT dont return values may return success/failure flagnumber of entries in the database affected

    For example: creating a tableinserting a row modifying a row

  • SQL commands with no return valueFrom Perl/DBI:INSERT INTO idac VALUES (LYC_CHICK, P00698)e.g. insert a row into a table:$sql = INSERT INTO idac VALUES (LYC_CHICK, P00698);$dbh->do($sql);

  • SQL commands that return a single rowSometimes, can guarantee that a database query will return only one rowor you are only interested in the first row $sql = SELECT * FROM idac WHERE ac = P00698;@values = $dbh->selectrow_array($sql);

  • SQL commands that return a multiple rowsMost SELECT statements will return many rows

    Three stages must be performed: preparing the SQLexecuting itextracting the results

  • SQL commands that return a multiple rows(Can also obtain array or hash reference) NB: statement handle / fetchrow_array rather than db handle / selectrow_array$sql = SELECT * FROM idac;$sth = $dbh->prepare($sql);if($sth->execute){ while(($id, $ac) = $sth->fetchrow_array) { print ID: $id AC: $ac\n; }}

  • SQL commands that return a multiple rowsIf you need to stop early you can do:$sql = SELECT * FROM idac;$sth = $dbh->prepare($sql);if($sth->execute){ for($i=0; $ifetchrow_array) { print ID: $id AC: $ac\n; } } $sth->finish;}

  • SQL commands that return a multiple rowsA utility method is also available to print a complete result set:$sql = SELECT * FROM idac;$sth = $dbh->prepare($sql);if($sth->execute){ $nrows = $sth->dump_results;}(Mostly useful for debugging)

  • Repeated SQL callsOften want to repeat essentially the same query, but with some different value being checked. For example:foreach $ac (P00698, P00703){ $sql = SELECT * FROM idac WHERE ac = $ac; @values = $dbh->selectrow_array($sql); print @values\n;}(using special option for 1-row returns)

  • Repeated SQL callsCould also be do:foreach $ac (P00698, P00703){ $sql = SELECT * FROM idac WHERE ac = $ac; $sth = $dbh->prepare($sql); $sth->execute; while(@values = $sth->fetchrow_array) { print @values\n; }}i.e. dont use special option for 1-row returns $dbh->selectrow_array($sql)

  • Repeated SQL callsIncrease in performance by binding a variable:$sql = SELECT * FROM idac WHERE ac = ?;$sth = $dbh->prepare($sql);

    foreach $ac (P00698, P00703){ $sth->bind_param(1, $ac); $sth->execute; while(@values = $sth->fetchrow_array) { print @values\n; }}

  • Repeated SQL callsNOTE:Performance increase depends on database and driverAlthough strings normally enclosed in single inverted commas, the bound variable is not quoted. If you have a number which you need to be treated as a string, then you do:

    $sth->bind_param(1, 42, SQL_VARCHAR);

  • SummaryDBI provides a standard APIIt does not standardize the SQLDBI is an older standard than ODBCThey can be used together and they are both evolvingBasic 3-step process:prepare / execute / fetchShortcut calls for no return or 1-row returnMany other functions available