62

Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

  • Upload
    cili

  • View
    50

  • Download
    0

Embed Size (px)

DESCRIPTION

Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005. Dragoslav Ogar SC Akademija. Architectural Benefits. SQL Server 2005 provides all the "big company" benefits all the latest technology while minimizing total cost of ownership - PowerPoint PPT Presentation

Citation preview

Page 1: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005
Page 2: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Dragoslav OgarDragoslav OgarSC AkademijaSC Akademija

Page 3: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Architectural BenefitsArchitectural Benefits

SQL Server 2005 provides SQL Server 2005 provides all the "big company" benefitsall the "big company" benefits all the latest technologyall the latest technology while minimizing total cost of ownershipwhile minimizing total cost of ownership

When you invent "the next big thing"When you invent "the next big thing" your database systems can grow with your database systems can grow with

your business without a total system your business without a total system rewriterewrite

don't re-architect when you outgrow don't re-architect when you outgrow hardwarehardware

Scaling is incrementalScaling is incremental pay as you gopay as you go

What features make this happen?When can I use them?

Page 4: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Take Advantage When?Take Advantage When?How much work to leverage the How much work to leverage the technology?technology?

User/Schema User/Schema Separation Separation

Cache SyncCache Sync SQLCLR SQLCLR

ProceduresProcedures LOB Data TypesLOB Data Types T-SQL TRY/CATCHT-SQL TRY/CATCH New T-SQL New T-SQL

StatementsStatements Data PagingData Paging XML ProcessingXML Processing

Upgrade Immediate

Minimal Work to Leverage

Design and Architect

Security by Security by PolicyPolicy

Secure Secure MetadataMetadata

Granular Granular PermissionsPermissions

Support for Support for Advanced Advanced OS/Hardware OS/Hardware featuresfeatures

Relational Relational Engine Speed-Engine Speed-upsups

Notification Notification Services Services IntegrationIntegration

Service BrokerService Broker Web ServicesWeb Services Data Encryption Data Encryption

and Key and Key ManagementManagement

Execute Context Execute Context for Proceduresfor Procedures

XML TypeXML Type UDTs/UDTs/

UDAggregatesUDAggregates

Improving Security and Scalability from Installation to Design

Availability in Layers to allow re-architecting in stages

Page 5: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Start Small...Start Small...Secure By Default, now...Scale Up?Scale Out?Caching?More Data Formats?More Robust?

Secure By Default, now...Scale Up?Scale Out?Caching?More Data Formats?More Robust?

Web Serverin DMZ

Web Serverin DMZ

Database Server

Database Server

Your Internal Network

Your Internal Network

Secure Connections

Secure Connections

Secure Data And Metadata

Secure Data And Metadata

Page 6: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Or Start Big...Or Start Big... Secure By DefaultScale UpScale UpScale Out – MessagingPerformance – Caching More Data FormatsMore Robust

Secure By DefaultScale UpScale UpScale Out – MessagingPerformance – Caching More Data FormatsMore Robust

Web ServerWeb Server

Database Server

Database Server

SQL Server 2005 supports advanced hardware and OS features

New, Bigger

Database Server

New, Bigger

Database Server

Page 7: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

As You GrowAs You Grow Secure By DefaultScale UpScale Out – MessagingPerformance – Caching More Data FormatsMore Robust

Secure By DefaultScale UpScale Out – MessagingPerformance – Caching More Data FormatsMore Robust

Inv

Order

Bill

Reliable transactionalmessaging with

SQL Server Service Broker

Page 8: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

As You GrowAs You Grow Secure By DefaultScale UpScale Out – MessagingPerformance – Caching More Data FormatsMore Robust

Secure By DefaultScale UpScale Out – MessagingPerformance – Caching More Data FormatsMore Robust

Inv

Order

BillMaster data management

and better performancewith Query Notifications

Page 9: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

As You GrowAs You Grow Secure By DefaultScale UpPerformance – Caching Scale Out – Messaging More Data FormatsMore Robust

Secure By DefaultScale UpPerformance – Caching Scale Out – Messaging More Data FormatsMore Robust

<invoice/><invoice/>

Inv

Order

BillNative XML storageImproved LOB types

Custom data typesand aggregates

Page 10: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

As You GrowAs You Grow Secure By DefaultScale UpPerformance – Caching Scale Out – Messaging More Data FormatsMore Robust

Secure By DefaultScale UpPerformance – Caching Scale Out – Messaging More Data FormatsMore Robust

<invoice/><invoice/>

Inv

Order

Bill

BEGIN TRY ...BEGIN TRY ...END TRYEND TRYBEGIN CATCH ...BEGIN CATCH ...END CATCHEND CATCH

Exception Handlingin Transact-SQL

Page 11: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Improved Security Improved Security From the Ground Up From the Ground Up

Page 12: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

SecuritySecurity

SQL Server is part of the Trustworthy SQL Server is part of the Trustworthy Computing initiativeComputing initiative Whether your company is small, Whether your company is small,

medium, or large, security is not optionalmedium, or large, security is not optional Data is your company's view of "reality"Data is your company's view of "reality" You must be secure for accurate pictureYou must be secure for accurate picture

Database security consists ofDatabase security consists of Security by design - integrated with Security by design - integrated with

policypolicy Security by defaultSecurity by default Secure deployment and maintenanceSecure deployment and maintenance Secure communications and storageSecure communications and storage

Page 13: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Off by DefaultOff by DefaultSQL Server Surface Area ConfigurationSQL Server Surface Area Configuration

Page 14: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Secure Data & MetadataSecure Data & Metadata

User-schema separation - database User-schema separation - database objects need not be tied to usersobjects need not be tied to users

Fixes "user leaves company" problemFixes "user leaves company" problem Allows DBA to allow installation of packages Allows DBA to allow installation of packages

with owners other than DBOwith owners other than DBO Allows separation of database object owners Allows separation of database object owners

even within a single databaseeven within a single database

Secure MetadataSecure Metadata You can only see what you can accessYou can only see what you can access Consolidation without seeing others' Consolidation without seeing others'

datadata All Permissions GrantableAll Permissions Grantable

Granular permissionsGranular permissions

Page 15: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Encryption and PrivilegeEncryption and Privilege

Some industries require encryptionSome industries require encryption Encryption keys securely stored in Encryption keys securely stored in

databasedatabase Instance key protected by DPAPIInstance key protected by DPAPI Logins are always encryptedLogins are always encrypted

Procedures can be signed or run as Procedures can be signed or run as certain accountscertain accounts Principle of least privilegePrinciple of least privilege Original login always available for Original login always available for

auditingauditing Proxy accounts for SQL Agent jobsProxy accounts for SQL Agent jobs

Page 16: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

CryptographyCryptography

1234-5678-1234-56781234-5678-1234-5678

Symmetric Key Encryption Symmetric Key Encryption

0x0088840517080E4FA2…0x0088840517080E4FA2…

EncryptionEncryption

DecryptionDecryption

1234-5678-1234-56781234-5678-1234-5678 0x0088840517080E4FA2…0x0088840517080E4FA2…

Encryption with public keyEncryption with public key

Decryption with private keyDecryption with private keyPublic KeyPublic Key Private KeyPrivate Key

Asymmetric Key Encryption Asymmetric Key Encryption

Page 17: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

SQL Server EncryptionSQL Server Encryption

Good Scenario:Good Scenario:

Encrypting secrets Encrypting secrets during loginduring login

Using asymmetric keys Using asymmetric keys to generate session keysto generate session keys

Using symmetric keys for Using symmetric keys for data encryptiondata encryption

Using SQL Server Using SQL Server certificates from trusted certificates from trusted sourcessources

Encrypting data as Encrypting data as required by lawrequired by law

Bad Scenario:Bad Scenario:

Encrypting all network Encrypting all network traffic inside an traffic inside an organizationorganization

Using asymmetric keys Using asymmetric keys for data encryption (slow) for data encryption (slow)

Using symmetric keys for Using symmetric keys for main key distribution main key distribution mechanismmechanism

Using SQL Server as a Using SQL Server as a certificate servercertificate server

Encrypting all data Encrypting all data (SLOW, and data can't be (SLOW, and data can't be used for indexes and used for indexes and joins)joins)

Page 18: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Encapsulating EncryptionEncapsulating Encryption

low-privlow-priv Credit ViewCredit View Credit Card #Credit Card #

Low-privneeds accessto both keys

Has access to view

Page 19: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Execute As and EncryptionExecute As and Encryption

low-privlow-priv Credit ViewCredit View Credit Card #Credit Card #

low-privlow-priv Credit ViewCredit View

Credit Card #Credit Card #Decrypt HelperDecrypt Helper

Has access to viewLow-priv

needs accessto both keys

Has access to view

EXECUTE AS DBO Low-priv hasno access to keys

Page 20: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Data SecurityData Security

Page 21: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Defense in DepthDefense in Depth Using a layered approach:Using a layered approach:

Increases an attacker’s risk of detection Increases an attacker’s risk of detection Reduces an attacker’s probability of Reduces an attacker’s probability of

successsuccess

Policies, Procedures, & Awareness

Policies, Procedures, & Awareness

SQLOS/SQLCLR hardeningSQLOS/SQLCLR hardening

Firewalls, packet filtersFirewalls, packet filters

Guards, locks, tracking Guards, locks, tracking devices, HSM, tamper-devices, HSM, tamper-evident labelsevident labels

SSL, session keys, cert SSL, session keys, cert securitysecurity

Execute As, signed procs, Execute As, signed procs, schemasschemas

Permissions, encryption, Permissions, encryption, secure metadata secure metadata

Password policies, off by Password policies, off by defaultdefault

Physical SecurityPhysical Security

PerimeterPerimeter

Internal NetworkInternal Network

HostHost

ApplicationApplication

DataData

Page 22: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Summary: SecuritySummary: SecurityTechnologyTechnology ImprovesImproves WhenWhen

Off by Default &Off by Default &

Password Password PoliciesPolicies

Greater security at install timeGreater security at install time Integrated Windows/SQL policies Integrated Windows/SQL policies

Upgrade Upgrade ImmediateImmediate

Metadata Metadata securitysecurity

All permissions All permissions grantablegrantable

Less exposed surface area Less exposed surface area Permissions easier to manage Permissions easier to manage

Upgrade Upgrade ImmediateImmediate

User/Schema User/Schema SeparationSeparation

No recoding when staff change No recoding when staff change Separate DBO and developers Separate DBO and developers

Upgrade Upgrade ImmediateImmediate

Keys and Keys and EncryptionEncryption

Compliance with privacy Compliance with privacy requirementsrequirements

Secure communications Secure communications

Minimal Work Minimal Work to Leverageto Leverage

Execution Execution ContextContext

Signed ProcsSigned Procs

Principle of least privilegePrinciple of least privilegeAuditabilityAuditability

Design And Design And ArchitectArchitect

Page 23: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

SQL Server 2005 Scales SQL Server 2005 Scales

With Hardware and Operating SystemWith Hardware and Operating System With Database FeaturesWith Database Features With Application DesignWith Application Design

Page 24: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Scaling - Hardware Scaling - Hardware OptionsOptions SQL Server optimized for hardware & SQL Server optimized for hardware &

OSOS Known as the "SQLOS" abstractionKnown as the "SQLOS" abstraction

This enables better support onThis enables better support on 64 bit architectures64 bit architectures Non-Uiform Memory Access (NUMA) Non-Uiform Memory Access (NUMA)

systemssystems Threads managed as tasksThreads managed as tasks

Enables SQL Server use of new OS Enables SQL Server use of new OS featuresfeatures Windows Server 2003Windows Server 2003

Dual core supportat no extra cost

Page 25: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Windows Server 2003 Windows Server 2003 EnabledEnabled Password policy check for SQL Password policy check for SQL

passwordspasswords Hot add memoryHot add memory Dynamic AWEDynamic AWE Native 64 bit support Native 64 bit support SOAP supportSOAP support Instant file initializationInstant file initialization 8 node SQL Server failover cluster8 node SQL Server failover cluster

Page 26: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Scaling Data with ServicesScaling Data with Services

Functionality built-in to SQL ServerFunctionality built-in to SQL Server Asynchronous Operations - Service Asynchronous Operations - Service

BrokerBroker Cache coherency - Cache SyncCache coherency - Cache Sync Request-response - Web ServicesRequest-response - Web Services Service Programs can be T-SQL or Service Programs can be T-SQL or

SQLCLRSQLCLR

Page 27: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

SQL Service BrokerSQL Service Broker Platform for building reliable, Platform for building reliable,

asynchronous, loosely coupled database asynchronous, loosely coupled database applicationsapplications Queues are database objectsQueues are database objects

Input in one transaction/context, execute in a Input in one transaction/context, execute in a different onedifferent one

Queue locking reduces conflicts and deadlocksQueue locking reduces conflicts and deadlocks Locks are based on dialogs (point-to-point Locks are based on dialogs (point-to-point

conversation)conversation)

Dialogs give unprecedented message orderingDialogs give unprecedented message ordering Reliable, durable, sequenced Reliable, durable, sequenced

communications session between servicescommunications session between services Ordering even across transactionsOrdering even across transactions

New DDL and DML for messagingNew DDL and DML for messaging Use the same API’s and tools as vanilla SQLUse the same API’s and tools as vanilla SQL

Activation - the right number of readers Activation - the right number of readers runningrunning To service the queuesTo service the queues

Page 28: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Database BDatabase A

DialogsDialogs DialogsDialogs provide two-way messaging between two services provide two-way messaging between two services Dialogs offer:Dialogs offer:

Guaranteed deliveryGuaranteed delivery Exactly-once deliveryExactly-once delivery In-order deliveryIn-order delivery Secure communicationsSecure communications

Dialogs:Dialogs: May be long-lived (years) or short-lived (seconds)May be long-lived (years) or short-lived (seconds) Are light-weightAre light-weight Are persistent sessionsAre persistent sessions

Customer Service

Dialog

Travel Service

Page 29: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Messaging with Service Messaging with Service BrokerBroker Inbound messages arrive on protocol pipeInbound messages arrive on protocol pipe Message is:Message is:

AuthenticatedAuthenticated Dispatched to appropriate queueDispatched to appropriate queue

Service Programs:Service Programs: Pick up work from queuePick up work from queue Run in different context than Run in different context than

incoming messageincoming message May run inside or outside serverMay run inside or outside server May send additional messagesMay send additional messages

Service QueueService Queue

SharedSharedStuffStuff

PrivatePrivateStateState

MessageMessageMessageMessage

Service ProgramService Program(decrement_inventory)(decrement_inventory)

System continues to runif service program

or queue is unavailable!

X

Page 30: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Service BrokerService Broker

Page 31: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Query NotificationsQuery Notifications

master data

multiplegranularreplicas

Notify Caches When Notify Caches When Master Data ChangesMaster Data Changes built into SQL Server built into SQL Server

20052005 based on indexed view based on indexed view

notificationsnotifications built into ADO.NETbuilt into ADO.NET

cache listeners can be cache listeners can be scaled to multiple scaled to multiple machines using SQL machines using SQL ExpressExpress

delivery via Service delivery via Service BrokerBroker

built into ASP.NETbuilt into ASP.NET automatic cache automatic cache

invalidationinvalidation Known as Cache SyncKnown as Cache Sync

two lines of codetwo lines of code

Page 32: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

CacheSyncCacheSync

Web Request Query

Results

Subscription

Page 33: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

CacheSyncCacheSync

Web Request

Subscription

Page 34: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

CacheSyncCacheSync

Subscription

UPDATE dbo.Products SET …

Page 35: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

CacheSyncCacheSync

Page 36: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Web Services and SQL Web Services and SQL ServerServer SQL Server 2005 canSQL Server 2005 can

Be used for HTTP-based web services on Be used for HTTP-based web services on any OS that supports HTTP in the kernelany OS that supports HTTP in the kernel

Execute any stored procedure and return Execute any stored procedure and return results using SOAP packetsresults using SOAP packets

Allow custom WSDL to support Allow custom WSDL to support heterogeneous clientsheterogeneous clients Use to wrap internal legacy systemsUse to wrap internal legacy systems

asynchronous accessasynchronous access

Use SQL Express and Web Services as a Use SQL Express and Web Services as a network input to a Service Broker network input to a Service Broker applicationapplication

Page 37: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Summary: ScalabilitySummary: ScalabilityTechnologTechnolog

yy ImprovesImproves WhenWhen

SQLOSSQLOSAdvanced hardware supportAdvanced hardware support

Advanced OS feature supportAdvanced OS feature supportUpgrade Upgrade

ImmediateImmediate

Service Service BrokerBroker

MessagingMessaging

Reliable system, even with partial Reliable system, even with partial outage outage

Scalability - service basedScalability - service based Large scale transaction messagingLarge scale transaction messaging Load balance over machines and Load balance over machines and

timetime

Design and Design and ArchitectureArchitecture

Cache SyncCache Sync Granular synchronizationGranular synchronizationMaster data managementMaster data managementTwo lines of ASP.NET code Two lines of ASP.NET code

Minimal Work Minimal Work To LeverageTo Leverage

Web ServicesWeb ServicesHeterogeneous integrationHeterogeneous integration

Complement to Service BrokerComplement to Service BrokerDesign and Design and ArchitectureArchitecture

SQL Server SQL Server Notification Notification

ServicesServices

Pre-built architecture componentPre-built architecture componentScales to larger number of eventsScales to larger number of eventsUseable over multiple machinesUseable over multiple machines

Upgrade Upgrade ImmediateImmediate

Page 38: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Flexibility Flexibility

Storage OptionsStorage Options Programming OptionsProgramming Options Deployment OptionsDeployment Options

Page 39: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

IT Manager DilemmaIT Manager Dilemma

T-SQLT-SQL

XMLXML

CLRCLR

Relational Relational data accessdata access

Semi-Semi-structuredstructured

data accessdata access

Computation & Computation & Framework Framework

accessaccess

Page 40: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

FlexibilityFlexibility

Storage OptionsStorage Options Programming OptionsProgramming Options Deployment OptionsDeployment Options

Page 41: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Data Type OptionsData Type Options The relational data types serve The relational data types serve

enterprise applications well but...enterprise applications well but... There's always been a tension with large There's always been a tension with large

datadata In database or file systems?In database or file systems?

XML becoming common for all industriesXML becoming common for all industries In B2B, B2C, data exchangeIn B2B, B2C, data exchange XML is a standard for data on the webXML is a standard for data on the web To evolve and integrate your business(es) To evolve and integrate your business(es)

you may need to support XMLyou may need to support XML

Domain-specific types used by some Domain-specific types used by some industriesindustries

Page 42: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Data Type EnhancementsData Type Enhancements

Relational is native for SQL ServerRelational is native for SQL Server Relational "open-schema" helped by PIVOTRelational "open-schema" helped by PIVOT

Assists sparse population & name-value pairsAssists sparse population & name-value pairs

Hierarchical queries with common table Hierarchical queries with common table expressions expressions

Large value type support is betterLarge value type support is better MAX data types subsume TEXT and IMAGEMAX data types subsume TEXT and IMAGE

XML is new built-in alternativeXML is new built-in alternative Through XML data type and queryThrough XML data type and query

Custom types and aggregates availableCustom types and aggregates available Through SQLCLR UDT for custom scalarsThrough SQLCLR UDT for custom scalars Through SQLCLR custom aggregatesThrough SQLCLR custom aggregates

Page 43: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Large Object StorageLarge Object Storage

New LOB support New LOB support VARCHAR(MAX)/NVARCHAR(MAX), VARCHAR(MAX)/NVARCHAR(MAX),

VARBINARY(MAX) VARBINARY(MAX) work like (N)VARCHAR, VARBINARYwork like (N)VARCHAR, VARBINARY support most T-SQL manipulation functionssupport most T-SQL manipulation functions

extended support for large data through extended support for large data through extension methods (WRITE method)extension methods (WRITE method)

up to 2gb in size, extendable in futureup to 2gb in size, extendable in future

Page 44: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

XML SupportXML Support

XML is a first class data type in SQL Server XML is a first class data type in SQL Server 20052005 Native XML storage Native XML storage

no need to store XML as TEXTno need to store XML as TEXT no hassles integrating with XML on file systemno hassles integrating with XML on file system document-centric or data-centric XMLdocument-centric or data-centric XML

XML Schema supportXML Schema support validation on input and updatevalidation on input and update schema collections support schema versioningschema collections support schema versioning

Native XQueryNative XQuery query in place - no need to retrieve over networkquery in place - no need to retrieve over network

XML IndexesXML Indexes XML processing uses same query processor as SQLXML processing uses same query processor as SQL

Page 45: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

XML Data Type & SchemaXML Data Type & Schema

Page 46: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

XML QueryXML Query

XQuery is the standard language for XQuery is the standard language for XML and databasesXML and databases Implemented with XML data type Implemented with XML data type

methodsmethods exist(), value(), query() operate on XMLexist(), value(), query() operate on XML nodes() produces rowsets from XMLnodes() produces rowsets from XML modify() changes XML in placemodify() changes XML in place

Uses XPath for data selectionUses XPath for data selection Can be used with T-SQLCan be used with T-SQL

sql:variable and sql:column available in sql:variable and sql:column available in XQueryXQuery

Can be combined with fulltext searchCan be combined with fulltext search

Page 47: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Scenario for XML Scenario for XML DevelopmentDevelopment

Good Scenario:Good Scenario: Data is semi-structured, Data is semi-structured,

small core of fixed data small core of fixed data with many, sparsely with many, sparsely populated extended populated extended attributes attributes Multi-value Property bagsMulti-value Property bags Complex Property bagsComplex Property bags ““WordXML”WordXML” Fixed data can be stored Fixed data can be stored

as relational columnsas relational columns Documents are large but Documents are large but

rarely updatedrarely updated Indexing will pay off Indexing will pay off

Data is hierarchicalData is hierarchical path expressions are path expressions are well suited for finding well suited for finding datadata

Bad Scenario:Bad Scenario: ““Database in a Cell”Database in a Cell” Documents are large and Documents are large and

updated frequentlyupdated frequently Document update Document update

contention is likelycontention is likely Data is fully structured & Data is fully structured &

populated populated candidate candidate for conversion to for conversion to relational schemarelational schema

Data contains large Data contains large binary objects (2GB binary objects (2GB limitation)limitation)

Page 48: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Improved Support for...Improved Support for...

ModelModel SchemaSchema QueryQuery ExtensioExtensionn

Strict Relational

Tables and

RelationsRelational Schema SQLSQL

T-SQLT-SQLSQLCLRSQLCLR

Hierarchical

Tables or XMLXML

Relational / XML XML

SchemaSchemaSQL Recursive Recursive

CTECTE

Sparse Attribute

Tables or XMLXML

Name/Value XML XML

SchemaSchemaSQL or

XQueryXQuery PIVOTPIVOT

Semi-structuredOr Markup

XMLXML XML XML SchemaSchema

XQuery XQuery XPathXPath

FullTextFullText

T-SQLT-SQLSQLCLRSQLCLR

Unstructured

MAX MAX DatatypeDatatype

ssIFilter FullText

Custom Scalars UDTUDT CustomCustom SQLSQL

CustomCustomMethodsMethods

Page 49: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Summary: Data TypesSummary: Data TypesTechnologyTechnology ImprovesImproves WhenWhen

XML SupportXML Support

Semi-structured data mgmtSemi-structured data mgmtMarkup language document mgmtMarkup language document mgmtValidation/integration of XML and Validation/integration of XML and

SQLSQL XML Indexes can improve XML Indexes can improve

performanceperformance

Design and Design and ArchitectArchitect

SQL SQL EnhancementsEnhancements

Support for hierarchical dataSupport for hierarchical dataOpen schema processing Open schema processing

Sparse attribute data modelsSparse attribute data modelsIn-database aggregationIn-database aggregation

Upgrade Upgrade ImmediateImmediate

New LOBsNew LOBsData just over the VARCHAR limit Data just over the VARCHAR limit

Programming with large dataProgramming with large dataBuffer management for large rowsBuffer management for large rows

Minimal Work Minimal Work to Leverageto Leverage

User-Defined User-Defined Types and Types and

AggregatesAggregates

Domain-specific data management Domain-specific data management Domain-specific formulasDomain-specific formulas

Inter-database interoperabilityInter-database interoperability

Design and Design and ArchitectArchitect

Page 50: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Flexibility Flexibility

Data Type OptionsData Type Options Programming OptionsProgramming Options Deployment OptionsDeployment Options

Page 51: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

T-SQL and SQLCLR T-SQL and SQLCLR

SQL is the language of relational SQL is the language of relational databasedatabase

Procedural code can beProcedural code can be T-SQLT-SQL

Native usage of logic with SQL statementsNative usage of logic with SQL statements Built into SQL Server since its inceptionBuilt into SQL Server since its inception Continuing enhancements with each releaseContinuing enhancements with each release

SQLCLRSQLCLR .NET framework code running in SQL Server.NET framework code running in SQL Server Enhances and compliments T-SQLEnhances and compliments T-SQL Not a replacement for T-SQL Not a replacement for T-SQL

or set based operationsor set based operations

Page 52: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

T-SQL EnhancementsT-SQL Enhancements

T-SQL is the language of 99% of pre-T-SQL is the language of 99% of pre-SQL Server 2005 procedural codeSQL Server 2005 procedural code Procedural enhancementsProcedural enhancements

Robust structured error handling comes to T-Robust structured error handling comes to T-SQLSQL

Output clause in SQLOutput clause in SQL

SQL enhancementsSQL enhancements Standard hierarchical recursive queriesStandard hierarchical recursive queries Better support for sparse attributes (PIVOT)Better support for sparse attributes (PIVOT) Ranking, Row Numbering functionsRanking, Row Numbering functions INTERSECT and EXCEPTINTERSECT and EXCEPT OthersOthers

Page 53: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

T-SQL EnhancementsT-SQL Enhancements

Page 54: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

CLR Runs in SQL Server Process Space:CLR Runs in SQL Server Process Space: SQL Server manages memory access etcSQL Server manages memory access etc Calls to SQL never Cross the Process BoundaryCalls to SQL never Cross the Process Boundary

Assemblies Stored in SQL Server, not the file Assemblies Stored in SQL Server, not the file systemsystem All CLR Objects get included in:All CLR Objects get included in:

BackupsBackups ReplicationReplication MirroringMirroring ClusteringClustering

SecuritySecurity Integration of SQL and CLR securityIntegration of SQL and CLR security Three levels of code access securityThree levels of code access security

Safe, External-Access (verifiable), UnsafeSafe, External-Access (verifiable), Unsafe

ProgrammabilityProgrammability.NET Integration Key Differences.NET Integration Key Differences

Page 55: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

SQLCLR and SQLOSSQLCLR and SQLOS

SQL Server SQL Server 2005 Engine2005 Engine

ApplicationsApplications

SQLOS - System ServicesSQLOS - System Services

Diverse Hardware /Windows Diverse Hardware /Windows Operating SystemsOperating Systems

SQLCLRSQLCLR

CLR HostingCLR Hosting

Integrated ResourceManagement

built-in, not grafted on

Transact-SQLTransact-SQL Assemblies stored inthe database,

not the file system

Page 56: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Summary: Summary: ProgrammabilityProgrammability

TechnologyTechnology ImprovesImproves WhenWhen

T-SQL T-SQL EnhancementEnhancement

ss

More robust error handlingMore robust error handling Row numbering and ranking in DB Row numbering and ranking in DB

Using large rowsets without Using large rowsets without cursorscursors

Minimal Minimal Work to Work to

LeverageLeverage

SQLCLR SQLCLR ProceduresProcedures

Logic intensive proceduresLogic intensive procedures Complex mathematicsComplex mathematics

Functions that are .NET built-insFunctions that are .NET built-ins

Design and Design and ArchitectArchitect

Page 57: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Flexibility Flexibility

Data Type OptionsData Type Options Programming OptionsProgramming Options Deployment OptionsDeployment Options

Page 58: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Deployment OptionsDeployment Options

There's always been a choice There's always been a choice between in-database and middle between in-database and middle tier/client logictier/client logic Sometimes the topology changes over Sometimes the topology changes over

timetime Machine power vs machine numbers Machine power vs machine numbers Network bandwidthNetwork bandwidth Smart clientSmart client

Programming toolkits may facilitate Programming toolkits may facilitate moving processing around (more agile moving processing around (more agile system)system) T-SQL usually best in DBT-SQL usually best in DB .NET code can move from DB <-> middle tier.NET code can move from DB <-> middle tier XML can be processed in either tier tooXML can be processed in either tier too

Page 59: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Flexible DeploymentFlexible Deploymentwith SQLCLR codewith SQLCLR code

Prod_Sched

Prod_Sched

run in middle tierto ease pressure

on database

...or run in databasefor locality of data and logic

Page 60: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

SummarySummary

More secure by defaultMore secure by default Better security integration with policiesBetter security integration with policies Secure code, data, metadataSecure code, data, metadata

More scalableMore scalable Scale up with SQLOSScale up with SQLOS Scale out with Service Orientation in Scale out with Service Orientation in

designdesign More data modelsMore data models

Relational, XML, Large Data, Custom Relational, XML, Large Data, Custom TypesTypes

More robust query modelsMore robust query models Procedural alternativesProcedural alternatives

Page 61: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Architectural Architectural EnhancementsEnhancementsTechnologTechnolog

yy ImprovesImproves WhenWhen

SecuritySecurity Integrated, built-in security policyIntegrated, built-in security policy

Secure data and metadataSecure data and metadataEnables principle of least privilegeEnables principle of least privilege

Upgrade Upgrade ImmediateImmediate

Service Service BrokerBroker

The way to build scalable, The way to build scalable, resilient large-scale systemsresilient large-scale systemsQueues and dialogs with Queues and dialogs with transactional consistencytransactional consistency

Design and Design and ArchitectArchitect

XML SupportXML Support Storage, schema, query, indexingStorage, schema, query, indexing

Business data and documentsBusiness data and documentsNative Web Service supportNative Web Service support

Design and Design and ArchitectArchitect

SQLCLR SQLCLR ProceduresProcedures

Logic intensive service programsLogic intensive service programs Adjunct to Transact-SQLAdjunct to Transact-SQL

Design and Design and ArchitectArchitect

T-SQL T-SQL EnhancemenEnhancemen

tsts

Data access language of SQL Data access language of SQL Server Server

Robust exception handlingRobust exception handling

Minimum Minimum Work to Work to

LeverageLeverage

Page 62: Architecting Scalable, Flexible and Secure Database Systems with SQL Server 2005

Take Advantage When?Take Advantage When?How much work to leverage the How much work to leverage the technology?technology?

User/Schema User/Schema Separation Separation

Query Query NotificationsNotifications

SQLCLR SQLCLR ProceduresProcedures

LOB Data TypesLOB Data Types T-SQL TRY/CATCHT-SQL TRY/CATCH New T-SQL New T-SQL

StatementsStatements Data PagingData Paging XML ProcessingXML Processing

Upgrade Immediate

Minimal Work to Leverage

Design and Architect

Security by Security by PolicyPolicy

Secure Secure MetadataMetadata

Granular Granular PermissionsPermissions

Support for Support for Advanced Advanced OS/Hardware OS/Hardware featuresfeatures

Relational Relational Engine Speed-Engine Speed-upsups

Service BrokerService Broker Web ServicesWeb Services Data Encryption Data Encryption

and Key and Key ManagementManagement

Execute Context Execute Context for Proceduresfor Procedures

XML TypeXML Type UDTs/UDTs/

UDAggregatesUDAggregates

Improving Security and Scalability from Installation to Design

Availability in Layers to allow re-architecting in stages