Mehmet Uner 1

Database Systems “Breaking Out of the Box”

Avi Silberschatz Stan Zdonik Bell Laboratories Brown University

July 7, 1997

Mehmet Uner 2

The Paper’s Theme (Strategic Directions)

1) Database Research should be devoted to the problems of data management no matter where and in what form the data might be found.

2) Database management skills should be applied to new data management environments that potentially require radically new software architectures.

Mehmet Uner 3

Outline

Introduction Background Our Skills Scenarios Barriers Research Conclusions References

Mehmet Uner 4

Introduction

The field of database systems research and development has been very successful over its 30 year history.

It has led to $10 billion industry that touches virtually every major company in the world.

Unthinkable to manage large volume of valuable information that keeps corporations runing without support from commercial database management systems (DBMS).

DBMS is a very complex system incorporating a rich set of technologies.

Suited for solving problems of large-scale data management in the corporate setting.

Mehmet Uner 5

DBMS

DBMS Requirements: Execution Overhead. High level of expertise to install and

maintain. Only manages data in fairly specific file

formats.

Mehmet Uner 6

Solution

At the same time: Data is changing rapidly. Data is stored in different places (e.g. files) Data is obtained in large volumes from external sources

like sensors.Solution: Not full-blown DBMS, a lighter-weight solution Instead of using an existing tool in a new application, it is

better to embed reusable components. Use database system components, techniques and

experience in new ways.

Mehmet Uner 7

Examples

Some examples that could benefit from data management techniques but that typically do not make heavy use of database products:– World Wide Web– Personal Information Systems (e-mail)– News Services – Scientific Applications

Mehmet Uner 8

Background

Database field born with release of IMS in 60’s. – IBM Product

– Managed data as hierarchies

– Data has value, manage independently of application

Codasyl, most well known successor– Based on graph-based structure.

Ted Codd published a paper in 1970– Suggested relational model.

Mehmet Uner 9

Background

Object Oriented Principles in 80’s – Allow users to create their own application-specific

types that can be managed by the DBMS.

Hybrid model in 90’s– Embeds object-oriented features in a relational context.

Mehmet Uner 10

Our Skills

Database Management Systems have been concerned with the following problems:– High Performance– Correctness– Maintainability– Reliability

From point of view of slow-memory devices that must be shared by multiple concurrent users

This approach leads to a set of skills and techniques that can be applied and extended to other problems.

Mehmet Uner 11

Skills and Techniques

Data Modeling– Language for defining structure of database– Language for manipulating those structures.

Query Languages– High-level language to retrieve data from the

database. (SQL) Query Optimization and evaluation State-based views

– Restricted and reorganized view of database.

Mehmet Uner 12

Skills and Techniques

Data Management– Automatic maintenance of data structures– Efficient Movement of data

Transactions– A response to correctness problems introduced by

concurrent access and update Distributed Systems Scalable Systems

– Database systems have been tuned to efficiently and reliably handle data volumes that exceed the size of the the physical memory by several orders of magnitude.

Mehmet Uner 13

Scenarios

The way for future data management systems The technology that would support these scenarios

constitutes a research agenda for the next decade.

1) Instant Virtual Enterprise

2) Personal Information Systems

Mehmet Uner 14

Instant Virtual Enterprise

An “instant virtual enterprise” (IVE) is a group of companies, that do not routinely function as a unit.

Come together to respond to a customer order or request for proposal.

Computer integrated manufacturing (CIM) is an example of an environment requiring IVE cooperation.

Engineering side– Design, Production, Quality Assurance

Administrative side– Planning, Production Control, Resource Management

Mehmet Uner 15

Instant Virtual Enterprise

Companies in IVE needs to exchange and manage large amounts of data

Companies will have many heterogeneous databases

Sharing and exchanging data with coordinating information is critical

Mehmet Uner 16

IVE Scenario

Company ABuilding an oil pipeline

Company QEngineering Firm (IVE)

Company RLicense their design

Company S

Engineering Analysis

Mehmet Uner 17

IVE Scenario

Company TActual Fabrication

Company UCasting

Company VDesign file conversion service

Company W

Documentation and Archiving

Mehmet Uner 18

IVE Scenario

Database Capabilities Needed:

– Executing a query for the design

– Data translation services for engineering analysis

– Coordination and configuration management

– Changes to an object in one subsystem require changes to one or more related objects in other subsystems.

– Security and access control over the information

– Archiving of information, even after the IVE disbands

Mehmet Uner 19

Personal Information Systems Scenario

Provides information to an individual Uses PID (Personal Information Device)

– PDA– Handheld PC– Laptop

Equipped with wireless network connection Access to internet Anywhere, Anytime.

Mehmet Uner 20

Personal Information Systems Scenario

Tightly integrated with individual’s activities. From morning to bed time.

In the morning– Local Weather Report

– List of Reminders

– List of Morning Meetings

– Best Route from home to work

– Personalized Headlines

– Personalized Investment Report

Mehmet Uner 21

Personal Information Systems Scenario

Throughout the day– Tasks for the day– List of customers to contact– Summary of breaking news– Best Driving Routes in the city

At the end of the day– Next day’s activities– Appointments

Mehmet Uner 22

Personal Information Systems Scenario

PID must continuosly query remote databases and monitor broadcast information

PID will magnify today’s client-server performance, scalibility and reliability problems

Where should data reside, PID or Server?

Mehmet Uner 23

Barriers

DBMS provides a tightly controlled and highly uniform environment

For the new applications, database functionality should be provided outside of the limits of a DBMS.

For the vision represented in the scenarios, a number of technical barriers must be removed.

Mehmet Uner 24

Barriers

Overhead– System requirements, expertise, planning, monetary cost

– Builder of personalized newspaper service do not use DBMS because there is no need for many of the advanced features.

– A subset of the traditional database services are needed by many new applications

Scale– Greater volume of data (petabytes)

– Hundreds of servers, client population even larger

Mehmet Uner 25

Barriers

Schema Organization– First create a schema to describe the structure of the database and

populate the database– Many applications currently create data independently of a

database system. (scientific applications, web sites)– Schema is incomplete or inconsistent.– Schema management facilities is needed to adapt the dynamic

nature of foreign data. Data Quality

– Information accessed form a WAN may be of varying quality.– Future information systems must be able to react to the quality of

the data source.

Mehmet Uner 26

Barriers

Heterogeneity– Data exists in many forms

– These dissimilar formats must be integrated to allow applications to access data in a high-level and uniform way

Query Complexity– Different characteristics in future environments

• Conventional, minimize number of disk access

• Future, minimize total “information bill”

Mehmet Uner 27

Barriers

Ease of Use– Highly-trained, full-time staff is assumed to manage a

DBMS– Yet most users have no training in database tech.– Simple set of interfaces needed.

Security– As the amount of shared information grows, the need to

restrict access to specific users of for specific use arises.

Mehmet Uner 28

Barriers

Guaranting Acceptable Outcomes– Transacation managemnet, a barrier to both system

performance and ability to specify acceptable outcomes

– New or enchanced transaction technology is needed

– Making data unavaliable is not acceptable

– Aborting transactions is unacceptable

Technology Transfer– Barrier between research and industry

• Insufficient knowledge of each other

Mehmet Uner 29

Research

In order to achieve the vision and overcome these barriers, a number of central research topics must be addressed:– Extensibility and Componentization– Imprecise Results– Schemaless Databases– Ease-of Use– New transaction Model– Query Optimization– Data Movement– Security– Database Mining

Mehmet Uner 30

Research

– Extensibility and Componentization• DBMS in a modular way

• Lighter-weight applications

– Imprecise Results• In the web search engines do not provide 100%

accuracy

• A general theory of imprecision must be developed

– Schemaless Databases• Able to work with unstructured data

Mehmet Uner 31

Research

– Ease-of-use• Better database interfaces are required.

– New transaction Models• Overcome blocking.• Provides Correctness.

– Query Optimization• New indexing methods, query processing strategies.• Cheaper but slower response time.• Sensitive to bandwidth and power considerations.

Mehmet Uner 32

Research

– Data Movement• In a distributed environment, the cost of moving data can be

extremely high• Asymmetric communication channels, (low bandwidth lines)

– Security• Formulation of an authorization model• Interoperability between differen security policies

– Database Mining• Machine Learning• Statistical Analysis• Database Technologies

Mehmet Uner 33

Conclusions

Database research must be broadly defined. Database community must apply its experience and

expertise to new areas and new solution packet must be found.

The vision is an integration that supports the application of database functionality in small modules that give just the right capability.

These modules should also represent a unified theory of information that allows for the querying information of all types without having to switch languages or paradigms.

Mehmet Uner 34

References

E. F. Codd, “A relational Model for Large Shared Databanks”, Communications of the ACM, 13:6,(June 1970), pp. 377-387.

J. Gray,http://www.cs.washington.edu/homes/lazowska/cra/database.html A. Silberschatz, M. Stonebraker, and J. Ullman, “Database Systems:

Achievements and Opportunities,” SIGMOD Record, 19:4, pp.6-22. A. Silberschatz, M. Stonebraker, and J. Ullman, “Database Systems:

Achievements and Opportunities Into the 21st Century”, http://www.cs.stanford.edu/pub/papers/lagii.ps

J. Toole and P. Young, http://www.hpcc.gov/cic/forum/CIC_Cover.html

Mehmet Uner 35

Thanks!

Any Questions?