Scalable Databases for Mission-Critical Business Applications · Scalable Databases for Mission-Critical Business Applications ... and across whole industry sectors, using IT

White paper May 15, 2000

1

SymfoWARE RDBMS

Scalable Databases for Mission-Critical Business Applications

Today’s databases need to be robust and reliable for 24 by 7 operation and also flexible for application expansion.

Information systems to cope with more users and new services

The business environments have been changing greatly as IT has become a mainstay of business. To lead business successfully, it is no longer enough to follow the old paradigms. The future requires a broader approach to using information. The explosive growth of the network environments has dramatically expanded the arena for the sharing and use of information. At the same time, the way information systems are structured has also changed greatly. The thrust of information systems used to be how efficiently an organization could perform its jobs internally. However, more and more organizations now link their information systems with those of others through networks. New application concepts such as EC/EDI and SCM are recently being implemented by connecting information systems between many different organizations. Internet-based systems, such as global material procurement systems and distribution systems spanning multiple organizations are already in use. Business has begun to move, across individual enterprises and groups, and across whole industry sectors, using IT for the infrastructure. This applies not just to B2B, but also to B2C activities for society and consumers. The spread of the Internet has prompted the advent of new services and methods, such as e-commerce (Internet shopping, Internet banking and Internet trading) and CRM for one-to-one marketing. There has been unprecedented growth in the use of information systems by consumers and society in general, as well as intra- and inter-enterprise. The range the systems cover has also grown on a scale never before seen. All these changes have had an enormous impact on business. The information systems that provide business infrastructure are under particular pressure from the new demands being made on them. The

urgent task is to cope with the increasing numbers of users, and to build frameworks that can facilitate the rapid development of new services. Internet business is characterized by the explosive growth in the number of users. It is not uncommon for the numbers of accesses to jump tenfold or hundredfold overnight. To meet these varied user needs, new applications and services must be implemented in as short a time as possible. Conventional information systems environments can hardly cope with these new conditions. This is why mission-critical applications have been ported to open systems from mainframes. If mission-critical applications are deployed on open systems, then it is easy for other application systems such as DWH and SCM to access and utilize large amounts of the data from the mission critical systems. To make it possible, however, it must be ensured that distributed data can be accessed transparently and also be integrated as required. The enterprise information systems in the Internet era are in urgent need for change both in quality and in quantity.

♦ Expansion of user/service ♦ Expansion of users/services

♦ Explosively growing data ♦ Explosive growth in data

♦ Expansion of mission critical service ♦ Expansion of mission critical services

Inter-organization

Society/Consumer

- Supply chain management (SCM)

- Virtual enterprise Inter-company EC

- Internet shopping

- One-to-one marketing

- Internet banking

- Sales force automation (SFA)

- Intranets

Intra-organization

♦ Exponential change of computerization system, both quality and quantity ♦

Rapid and massive changes to the quantity and quantity of information systems

-

Figure 1. Network computing era

IT=Information Technology EC=Electronic Commerce EDI=Electronic Data Interchange SCM=Supply Chain Management B2B=Business to Business B2C=Business to Consumer CRM=Customer Relationship Management DWH=Data Warehouse RDBMS=Relational Database Management System


2

SymfoWARE RDBMS

Rapid growth of data volumes for both transactional and informational applications

It goes without saying that the RDBMS is the key technology for supporting business. the RDBMS forms the foundation for both mission-critical and informational systems. For this reason, we can safely say that today's business is grounded on the RDBMS. Certainly, as the business environment changes, the uses of the RDBMS and services will expand into new areas. This also means that the amounts of data processed by the RDBMS will grow substantially. Whether they are for mission-critical or informational systems, the volumes of data to be processed will grow explosively in any system environment. Until now, large RDBMS has been used in mission critical applications with the focus on the efficient processing of large volumes of mission-critical data. However, a whole range of new applications has been born surrounding this data. These informational applications have recently attracted not only business enterprises and government bodies, but also society and consumers. Some specific examples are the uses of POS (point of sales) data by convenience stores and large retailers, and the uses of customer data by telephone companies. In these examples, after being used for accounting purposes, each raw data is accumulated on a daily basis and stored over the long term so that the data can be used for the analysis of buying trends and the provision of appropriate services. RDBMS data can be used as a base to determine whether product ranges are appropriate and whether customers are switching to competitors. This means that RDBMS plays an important role for the user company as a tool for developing strategy.

- Ensuring database operation time

- Establishment of recovery method

Risks with large volume databases - To ensure the up time for the database - To establish recovery method

♦ Lower price of hard disks

♦ Easier collection data

♦ Reduction in hard disk prices

♦ Easy collection of data

♦ Intensified competition

♦ Daily increase in online data

♦ Collected data cannot be abandoned

♦ Daily increase in online data ♦ Collected data cannot be discarded

Background

Figure 2. Trends in high capacity databases

The traditional role of RDBMS in processing mission-critical data will not change. But in addition, it will need to support many new business applications that will be developed to exploit the mission-critical data. That means that RDBMS is being called upon to play new roles in processing unimaginably large volumes of data for business.

High-speed processing of large volume transactions as the essential requirement

To support the next generation of business applications, the RDBMS will have to do much more than simply manage large volumes of data. One of those new requirements is higher performance and scalability. As the number of users and the volume of data increase explosively, the processing traffic of the database increases accordingly. Moreover, the speed of growth accelerates every year. In the days of the mainframe, the system scale could be estimated over a period of five or ten years. Now, the traffic can suddenly increase excessively on a particular day. It is impossible to estimate the scale of a system over an extended period of time. Of late, hardware has been evolving rapidly, particularly CPU-related technology. This evolution has not only led to the development of faster processors but also has increased the number of parallel CPUs mounted on a machine. New ways of achieving faster speed are put to practical use one after another. In these situations, the question is how to ensure the growth potential and scalability of a given system. As we noted previously, the volumes of data grow explosively in any system type, be it mission-critical or informational. No matter what application area it is to be used in, RDBMS must equally provide high performance and scalability.


3

SymfoWARE RDBMS

Lurking danger behind applying mission critical

- Guarantee high-speed recovery/restart from down

- Guarantee continuation of operation with DB failure

Risks of mission critical systems • To ensure high speed recovery/restart

from system failure • To ensure continued operation even after DB

failure

♦ Developing intercorporate EC/EDI using networks

♦ Seek organization efficiency by Supply Chain Management♦ Service for consumers using Internet

♦ Introduction of inter-company EC/EDI using networks

♦ Pursuit of efficiency improvement through SCM

♦ Service for consumers using Internet

♦ Direct business activity of mainframe, UNIX, and PC system

♦ Growing sociality of the provided service♦ Mainframe, UNIX and PC systems directly linked to business activities

♦ Growing social impact of the services provided

♦ 1/3 of global business uses 24 hour, 365-day operation system

⇒ Users will be more than 60% in 2000.

♦ 1/3 of world businesses provided 7 by 24 operation in 1999.

⇒ More than 60% in 2000

Background

Figure 3 Trends in the use of databases for

mission critical systems

High reliability and stability as the essential characteristics of a mission critical system

Another important requirement for the RDBMS is reliability and stability that are the essential characteristics of a mission-critical system. The uses of the RDBMS have expanded from the business applications within and between enterprises to Internet-based applications involving consumers and the society as a whole. It was said before the turn of the century that one-third of the business systems around the world operated on a 24 hours a day and 7 days a week (24 by 7) basis. As the systems will be required to handle increased accesses without such traditional constraints as time and place, the RDBMS cannot be allowed to stop at all. High reliability and stability are the absolute requirements for large-scale RDBMS systems. Even in the event of an unexpected failure, the system must be brought back up again as fast as possible. The important point here is how fast the operation can be resumed. It is not enough for the RDBMS to recover. It must do so in as short a time as possible. In the eBusiness world, you cannot afford to wait for days for an application to re-start. It is essential to minimize the effects of a down time all applications. The trouble may not be with the RDBMS itself as with a hard disk failure. At all events, therefore, the RDBMS must provide a facility that can ensure the continuation of the operation at all events.

Lurking danger behind increasing traffic

- Guarantee scalability of a database

- Guarantee system growth

Risks with increased traffic - To guarantee database scalability - To guarantee system growth

♦ Developing intercorporate EC/EDI using networks

♦ Each type of service for consumers using Internet

♦ Progress in inter-company EC/EDI using networks

♦ All types of consumer-oriented services using Internet

♦ Rapid scale up/high-speed of hardware

♦ Exponential growth in number of users

connected by networks

♦ Rapid scaling up/performance improvement of hardware

♦ Exponential growth in number of users through Internet connection

Background

Figure 4 Exponential increase in database

processing traffic

Flexible RDBMS to accomodate new and expanded applications in the terabyte era

As mentioned earlier, to support future business applications, the RDBMS must satisfy the three essential requirements: high performance, high capacity, and high reliability. However, it is not enough. In addition to satisfying these requirements, the RDBMS must also be flexible to allow new applications to be added and maintenance work to be carried out without interrupting operations. There are many other points that we have to consider. For instance, we need to take into account the lifetime of user systems. These days, many new Internet-based applications are developed and deployed one after another. At the same time, we need to consider large mission critical systems that once built cannot be replaced so frequently, and must continue to operate. The RDBMS should be able to support the life cycles of such systems that are used over a long period of time. It is vital that as the major infrastructure component, the RDBMS must be correctly connected with each of the applications in charge of specific tasks. It is clear that the RDBMS plays a critical part in the provision of the 24 by 7 operation of applications. But the definition of a 24 by 7 -operation has changed. This used to simply mean the non-stop operation of a system. Now it means the uninterrupted 24 by 7 operation of the associated applications while allowing the addition of new applications and the expansion of existing applications. The RDBMS cannot be stopped whenever a new service is opened up. In the new era where new business practices are born every three or six months, it is necessary to provide a mechanism by


4

SymfoWARE RDBMS

which new applications are added and existing applications are extended without interrupting the entire system. The RDBMS of the Internet era must be not only highly reliable, stable and powerful, but also it needs to be highly flexible to accommodate rapid changes in applications.


5

SymfoWARE RDBMS

The world’s fastest RDBMS with large capacity, high performance and high stability for enterprise systems

SymfoWARE’s proven record as the highly-reliable RDBMS

Recognizing the above-mentioned requirements for the RDBMS, Fujitsu offers SymfoWARE as the large-capacity, high-performance, and high-reliability RDBMS engine. Some of the main features of SymfoWARE include: • support for terabyte-level data volume

operation • high reliability with recovery in the order of

seconds • scalability • high performance satisfying rapid response

and high throughput • high development productivity and external

connectivity • new mode of maintenance support with

special considerations to the lifetime of user systems

Providing a database for new measurement

in network computing era



Data quantity: assurance of operation in gigabyte units

Reliability : restart of operation in hour order

stop of operation when DB failure occurs Performance : response, throughput Function : development productivity and

connectivity

Maintenance : dependence on the product ’ s generation number

Measurement of operating an existing DB

Data quantity : assurance of operation in terabyte units

Reliability : restart of operation in

second order

Fall back operation + high - speed recovery

Performance : scalability, response, and throughput

Maintenance : proper support for user ’ s request

♦ Back- up solution for

a huge DB system

♦ Operation function of DB

maintenance for 24 hour

operation

♦ Back - up solution for

a huge DB system



operation

♦ Providing the world rankingSymfoWAREperformance

♦ Providing the world rankingSymfoWARE Parallel Server for





Optimum database for adapting to the new measures in the network computing era

Conventional measures Data volume: Support for gigabyte level operation

Reliability: Restart of operation in the order of hour

Operation halt at DB failures

Functionality: Development productivity and

connectivity

Maintenance: Based on the age of the products

New measures for DB operations Data volume: Support for terabyte level operation Reliability : Restart of operation in the order of seconds Fall back operation + high-speed recovery Performance : Scalability, response and throughput Functionality : Development productivity and

connectivity Maintenance : Based on user requirements

♦ Back- up solution for

a huge DB system



operation

♦ Back-up solutions for very large DB systems

♦ Database maintenance operations geared to 24-hour operation



♦ SymfoWARE Server with the world’s top-level performance for OLTP

♦ SymfoWare Parallel Server with theworld’s top-level performance for large warehouse processing

Figure 5 Fujitsu approach

As at the beginning of 2000, Fujitsu shipped more than 20,000 SymfoWARE servers. Some

organizations use a few hundred servers, while others use even several thousands of servers within one organization to support their nation-wide business applications throughout Japan. A wide range of organizations has successfully implemented SymfoWARE in systems that require absolute reliability. For example, SymfoWARE is implemented in the following systems, where absolute reliability needs to be maintained. • Financial systems in banks and securities

companies • Public systems for government and other

public offices at central and local levels • Medical systems • Inter-company payment systems • Stock exchange transaction systems and

stock market report systems Because these systems service many customers and society as a whole, any failure can cause widespread impacts. The fact that SymfoWARE is used on these kinds of systems demonstrates that these organizations have full confidence on SymfoWARE’s performance and reliability.

SymfoWARE ‘s scalability from a small-size system to a large-scale mission critical system

In an open system environment, SymfoWARE can be used on various platforms according to the use and size of the applications. As SymfoWARE runs on such open environments as Windows NT, Windows 2000, and Solaris, the world’s top UNIX operating system, it is flexible to build systems according to the chosen environment. SymfoWARE can be used with a wide range of systems, from regional systems in branch stores or offices to large mission-critical systems for head offices. As the concepts, functions, and interfaces remain the same regardless of the chosen environment, SymfoWARE assets can be scaled up without change when applications grow and expand in future.


6

SymfoWARE RDBMS

Internet business can experience explosive growth on any given day. If the system needs to be reconstructed to cope with the growth at such a critical situation, the business may incur a fatal loss. This is not a worry with a system built on SymfoWARE. SymfoWARE makes it possible to build the best system to fit individual needs, regardless of the platform type and system size.

SymfoWARE’s impressive processing power on enterprise systems

SymfoWARE’s enormous processing power deserves a special mention among the many SymfoWARE features. The core technology of SymfoWARE provides a number of unique functions, including Staging Control, which splits processing in RDBMS into units and provides pipeline processing, and Dynamic Queue Control, which regulates load imbalances between CPUs. SymfoWARE was the first RDBMS that implemented 64-bit functions on Solaris. The high performance provided by the many advanced functions of SymfoWARE has been proven in fair and impartial reviews. The international benchmark for evaluating RDBMS performance, TPC-C, gave SymfoWARE the world’s top transaction performance ratings for both the Solaris 8-CPU and Windows NT 4-CPU environments. SymfoWARE was also found to be world No.1 for price/performance for the Solaris 4CPU environment. (Published June and July 1999). For enterprise systems that need to handle large numbers of transactions at fast speed, SymfoWARE’s massive processing power provides a great advantage. A company has employed SymfoWARE in its mission-critical system that processes more than 1000 simultaneous transactions and millions of orders per day. Employing SymfoWARE provides enormous advantages to organizations that wish to build large databases on open systems. Users are also taking advantage of the enormous processing power of SymfoWARE in their informational systems such as data warehouse systems. SymfoWARE delivers its extensive processing power in informational systems. For instance, SymfoWARE is used by a telecommunications user to analyze tens to hundreds of millions of its daily call data records, and by a retail company to process billions of detail data items with the data accessed by its end-users at fifty offices.

DB server

Client

Size of the system No of clients: 27,720 units No of application servers: 6 Size of database: 3.6 terabytes3-tier architecture

Application server

Solaris 2.6 8 CPU World No.1 (June 24, 1999)

Windows NT 4 CPU Wrold No. 1 (July 20, 1999)

PRIMEPOWER Model 600/8 CPU

SPARC 64GP 300MHz 8GB RAM

Figure 6 SymfoWARE transaction performance

record (Evaluation by International Standard Benchmark (TPC-C))

SymfoWARE’s easy-to-manage maintenance and operation suited for large volumes of data

The RDBMS employed in an enterprise system needs to have the ability to deal with larger volumes of data than ever. However, it is not enough for the RDBMS to simply process the data. It needs to handle the data efficiently. Since its predecessor, RDB II, SymfoWARE has been able to manage a database of 2048 PB (petabytes) data. A database of this massive size has not yet been actually built for practical purposes, but it is comforting to know that SymfoWARE can easily handle any future expansion of the database size. Even if we can say that SymfoWARE can manage such a gigantic-size database, it may not mean much if all we can do is to define such a database. What matters is whether the database can be created, backed up, and restored quickly. Only when we can be certain that the RDBMS can support these critical requirements, we will know that it can bring great advantages to actual applications. In this regard, SymfoWARE stands head and shoulders above other RDBMS products. The results of measurements show that it takes only 47 minutes to create a 100GB database. This is more than three times faster than other RDBMS products measured under the same conditions. Even when compared with other vendors’ database products tailored for data warehousing, SymfoWARE is about 1.5 times faster again. Backup time for SymfoWARE is about half or one-third of the time for other products. SymfoWARE employs its pioneering partitioning technique to provide this superior performance. With

TPC=Transaction Processing Performance Council RDB II=(Name of the SymfoWARE Server’s predecessor)


7

SymfoWARE RDBMS

this technique, SymfoWARE achieves data distribution by subdividing a table into individual axes according to the time or location axis. Through this partitioning technique, units for data operation can be reduced and at the same time be processed in parallel. It produces big improvements in performance. As such a distributed table is presented as single logical table to the application that uses the table, the application does not have to be conscious of the partitioning at all.

Volume 3

Customer table

Branch Person Customer code

0501 500056780510 50033469

Seattle Seattle

N.Y. 0101 10012121N.Y. 0105 10023456

…… ……………

0201 200256700215 20053457

TexasTexas

………………………

……

…… …… ……………

Application

Logical structure

Volume 1Storage structure

Physical structure


N.Y. 0101 10012121N.Y. 0105 10023456Sub-table 1


N.Y. 10012121N.Y. 0105

010310023456

Sub-table 2Branch Person Customer code

Texas 20012121Texas 0205

020320023456

Sub-table 3Branch Person Customer code

Seattle 30012121Seattle 0205

030330023456

Volume 2


Texas 0101 10012121Texas 0105 10023456

Branch Person Customer codeSeattle 0101 10012121Seattle 0105 10023456

No effect on application interface

Figure 7 Approach to management of large

volume data (table partitioning)

The partitioning technique also leads to higher data integrity. If a failure ever occurs, only the defective partition is closed because backup and recovery can be performed in partition units. The remaining partitions can remain open for operation. For example, even if a failure occurs in the data for Branch A, the data for both Branches B and C can continue to be made available, without being affected. Users will find it very helpful and comforting to be able to localize the impact of possible failures like this. Most organizations that have installed SymfoWARE take advantage of this partitioning technique. The user can decide the units for partitions as required. The example case above used each branch as the unit for partitioning. This location based partitioning can be combined with time-based partitioning to expand the dimensions with multiple columns. Another method is to distribute transactions evenly by allocating partitions for the metropolitan area and for the other areas. The system operations can be adjusted in a variety of ways, according to the user’s preferences.

SymfoWARE’s high reliability assured through failure prevention, detection, localization, and rapid recovery

♦♦♦♦ Early detection and separation of failed areas

As the system becomes larger, the number of the hardware components which make up the system increases. Although the reliability of these hardware components improves each year, the probability of failures does not decrease at the same rate when the total number of parts increases. There is also the possibility that the impact of a failure becomes greater as the scale of the system grows larger. When the RDBMS supports an enterprise system, it requires a way of protecting data from such failures. Specifically, this means preventing any failure from happening, and at the same time, minimizing the effects of any failure on applications, by separating the defective components, and providing rapid recovery in the event of a failure. SymfoWARE has a rich set of high reliability measures and techniques to meet these requirements. As the database directory, data, and logs can be distributed on different disks, for example, you can avoid dual failures. As SymfoWARE performs a parity check when reading and writing data, it can detect defective data caused by intermittent disk error or memory corruption.

♦ Parity check on a page to read

- Close the page after 5 failed attempts- Output investigation data to an

external file

♦ Parity check on a page to write

- Output investigation data to an external file

- Overcome memory corruption by restarting operation

SymfoWARE

Prevents the impact of saving defective data from spreading

Page

Database

Database

Prevents the impact of defective data from spreading

Figure 8 Early detection and separation of

errors

♦♦♦♦ Early recovery from failure When the database directory fails, it can be quickly recovered with the Reverse Creation function. This function recovers the database directory from the database. If an RDBMS product does not support this function, it will be necessary to rebuild the entire database to recover the directory. The larger the size


8

SymfoWARE RDBMS

of a database, the longer it takes to rebuild. If an entire database for an enterprise system needs to be reconstructed, it will take such a long time as to greatly affect the applications that use the database. There was an actual case reported in which it took four days to recover after an organization’s RDBMS directory failed. When the same situation was simulated under SymfoWARE, the recovery time was only about two hours.

DB definition data

As it should starts from database recreation, complete recovery is hard to achieve with other DBMS.

DB directory

Unable to continue operation

Dedicated recovery function

Space management data

Tables Tables

DB definition data

Tables Tables

DB directory

Figure 9 Reverse creation

By combining the Reverse Creation function with the Incremental Recovery function, recovery can be accomplished more rapidly. The Incremental Recovery is used to recover only the data required for restoring a specific application. Important applications can be given a high priority and restored earlier than others. The directory data for low priority applications can be recovered at the background in parallel with the restarting of the applications. In this way, applications can be restored in stages with higher priority applications to be restored in a few minutes. SymfoWARE comes with the Adjust Recovery function designed for effective recovery from log failures. If the temporary log files are destroyed under other vendors’ RDBMS, all the databases need to be recovered from the backup files. This will cause serious impacts as explained in an example case above. With SymfoWARE , however, temporary logs and archive logs are captured at the same time, and even if a temporary log fails, its data can be immediately restored up to the point just before the failure.

SymfoWARE’s hot standby option for mission critical systems

To improve the system’s availability and up-time, it is effective to have a redundant configuration with multiple systems. It is an essential measure for any

mission critical system that cannot allow for system downs. SymfoWARE provides cluster system solutions for implementing such high availability environments. One of such solutions is the Hot Standby Option. By adopting this option, an open system environment can take advantage of the same kind of hot standby operation that has long been used on mainframes environments. Of those organizations that have adopted SymfoWARE, nearly 100 are already using or currently implementing the Hot Standby option in their mission-critical systems such as financial and public systems. The Hot Standby option used to be adopted mainly in such high-reliability systems as banking, stock exchange, and medical systems which could not allow for system failures. Recently, hot standby is also adopted in e-Commerce systems, CRM systems, and resident service/accounting systems of local governments. The background to this is that a system failure may cause serious reliability problems for both public and private organizations. Since IT services are now taken for granted, people are very critical when they suddenly find that they cannot get the service they expect. In the United States, there are already cases where users sue over stoppages at an Internet online trading service. It is highly likely that more and more business transactions will be moved to the Internet. To be successful on the Net, each system must be made as robust as possible. Under a hot standby mode operation, a standby system is created with an almost identical configuration to the production system. The standby system takes over the application if a failure occurs to the production system. SymfoWARE has various techniques to bring this about. First of all, the Pre-standby function makes it possible for the standby node to open a database environment that is the same as the one on the production node. This technique comes from mainframe technology. When SymfoWARE is switched to the standby node, it uses the Cache Recovery function to recover the database from memory, and the Parallel Recovery function that partitions the database and recovers it in parallel. These functions are different from pre-standby; they use cutting-edge technology that was not available in the mainframe era. Thanks to these techniques, SymfoWARE can restore applications in 1/2 to 1/5 of the time required under other vendors’ RDBMS products. These are essential features required for non-stop 7 by 24 business applications.


9

SymfoWARE RDBMS

♦ Time to restart: 1/2 to 1/5 that of third-party products(Results of comparison testing in Fujitsu environment)

Production Standby

Pre-standbyPre-standby

Failure

Production

Cash recovery Parallel recoveryCache recoveryParallel recovery

Figure 10 Hot standby operation

SymfoWARE’s flexible standby configurations to choose from depending on requirements and system costs

The users of SymfoWARE can choose one of the three possible configurations of production and standby nodes according to the requirements of the system: 1:1 standby, mutual standby, and n:1 standby. The 1:1 standby configuration is the most basic one in which there is one standby node for each production node. In the event of a failure at the production node, the standby node takes over the application. This mode is already adopted in many system environments. In a mutual standby configuration, both nodes play the standby node role for each other while they run their designated applications. The mutual standby mode operation reserves resources such as CPU and memory, and takes over the running of the applications on the other when a failure occurs. Therefore it resembles the 1:1 standby configuration, but it has the advantage of reducing the cost of providing a dedicated standby system. One SymfoWARE user in the manufacturer industry has adopted this mutual standby configuration for its in-house applications systems that run batch applications on one node and its online ordering applications on the other node. By running batch jobs on the standby node, the user has been able to ensure the reliability and at the same time to reduce the costs of providing the standby system. Another user in the medical industry is considering the adoption of the mutual standby configuration to its accounting system, drug administration system, equipment control system, and electronic patient information system.

The n:1 standby mode was developed to provide cost savings in running a standby system without degrading system performance. One standby node is reserved to cover multiple production nodes. There is no risk of performance being degraded because the number of production nodes remains the same even when a failure occurs. SymfoWARE’s hot standby feature is flexible so that you can select the type of operation according to your needs for the trade-off between cost and performance. SymfoWARE’s hot standby feature brings great advantage to the user by not only making it possible to run systems efficiently, but also guaranteeing the reliability of systems with different loads and load peaks.

Standby 1:11:1 Standby

Production

ProductionFailure

Standby

Mutual standbyMutual standby

Production A

Production A

Standby A

Standby N:1n:1 Standby

ProductionA

ProductionB

Production BStandby B

Production B

StandbyProduction

B

ProductionA

Failure Failure

Figure 11 Types of standby configuration

SymfoWARE’s parallel cluster option for improved availability and scalability

One of SymfoWARE’s cluster system solutions is the Parallel Cluster Option that is adopted to ensure scalability and to offer high levels of availability. With the Parallel Cluster Option, an application system is built on multiple servers. This option makes it possible to distribute the risks and loads by providing a redundant hardware and software configuration. Another advantage of this option is that it provides greater flexibility for any future expansion of the system. There are two ways to achieve parallel transactions: the shared-everything and shared-nothing approaches. SymfoWARE uses the shared-nothing approach. The feature of this approach is to partition the database and to establish a control node for each partition. Each node has direct access to data under the control nodes, and has access to the other data by automatic routing (automatic remote). The biggest advantage of using the shared-nothing approach is that it offers linear improvement in performance as the number of nodes increases. The shared-everything approach suffers from large overheads, and the effects of increasing nodes are limited. Even if a system is expanded under the


10

SymfoWARE RDBMS

shared-everything approach, its result is not fully reflected on the system availability. Users will face difficulties if their RDBMS in their mission-critical systems cannot cope with rapid business growth. System expandability is an important issue, especially in the eBusiness world. SymfoWARE scales well as the number of nodes increases, and so relieves you of this problem.

Shared-nothing

Shared-everything

Informix

Sybase

MS SQL Server

Oracle

Approach DBMS

Number of computers

Shared-everything approach

Shared-nothing approach

SymfoWARE

4 3 2 1 5 6 …

Vector of requirements Requirements vector

Other companies Other vendors

Figure 12 Data sharing in SymfoWARE

SymfoWARE’s assurance for the continuity of applications by switching in seconds

SymfoWARE manages tables and indexes in partitioned units. This means that data may sometimes be updated across multiple nodes. Even in such cases, SymfoWARE ensures the consistency of transactions through its automatic two-phase commit control. SymfoWARE also provides techniques to minimize degradation of operations when a node fails. One such technique is the Flush Treatment Recovery (FTR) function. FTR uses a high-speed interconnection between nodes to transfer the transaction log to the memory of the standby node. When switching occurs, the system does not have to read the log from disk, because the updated log is present in memory. The Delayed Transaction Recovery (DTR) function can be used in the background to handle the temporarily blocked data that was being processed at the time of switching. With a normal switching situation after a system failure, the database must be recovered first before the system proceeds to the next step. However as SymfoWARE first isolates the abnormal data and the database can be recovered in the background, the system can immediately proceed to the resumption of its applications and other tasks.

SymfoWARE can switch over in seconds with these functions. This means that there is very little impact on applications. Many mission-critical systems such as financial, medical, and public administration systems cannot afford to stop running even in the event of a system failure. SymfoWARE provides high-level fault-tolerant and non-stop operations that are essential to such systems.

Active nodeProduction nodeStandby node

Memory

Transaction log

DTR

FTR

Executejob

Switching event

FTR: Flash Treatment RecoveryDTR: Delayed Transaction Recovery

Seconds

Initialize log environment

Close working data only

Reset application connection

Recover working data

Figure 13 HA technology for load sharing

SymfoWARE Parallel Server for centralized enterprise warehouses

We have so far discussed mainly the SymfoWARE products for mission critical systems. SymfoWARE Parallel Server (SPS) is designed for large-scale data warehouses. It is built on SymfoWARE with the parallel processing capability not only for high-speed join, aggregation, and sort operations but also for high-speed append and update operations on large quantities of data that are handled by mission critical systems. This utility lets you add and update large amounts of data. The main areas of applying SymfoWARE Parallel Server include the following: • To build a centralized enterprise data

warehouse and then generate data marts for ad-hoc queries

• To perform detailed analysis on raw data, regardless of whether they are for pre-defined or ad-hoc queries

• To build specialized data marts for power users – for uses such as analyzing large amounts of customer data

In these cases, it will be customary to use OLAP tools, including Fujitsu SymfoWARE Navigator. Recently SymfoWARE Parallel Server is adopted more commonly for the second mode above in which


11

SymfoWARE RDBMS

raw data – transaction details are directly analyzed. This means that instead of building data mart s for analysis, SymfoWARE Parallel Server can be employed to the direct analysis of raw data, taking advantage of SymfoWARE’s parallel database features and advanced hardware features. SymfoWARE Parallel Server can also be used for building data warehousing systems by accumulating data from operational systems. It can be adopted for high-speed batch processing as well. SymfoWARE Parallel Server brings advantages in a variety of ways when you need to utilize operational system data for other business applications. As for large-scale data warehouses, there are two types of RDBMS products available in the market. One is the general-purpose RDBMS products that are enhanced in performance with a parallel processing features, and the other is the specialized RDBMS products that are specifically customized for data warehouse type systems. SymfoWARE Parallel Server belongs to the first type of RDBMS.

Specialized database systems for specific purposes can be disadvantageous in that they are not designed for update operations but usually for particular types of query and reporting operations. On the other hand, the data managed by a centralized warehouse comes in various forms, and is not always analyzed in the same way. It is necessary to provide consistent all kinds of query requests. There can be many cases where a specialized database may be unsuitable for certain purposes. For such reasons, general-purpose RDBMS systems with a parallel processing feature are more advantageous. The use of general-purpose RDBMS systems brings an added advantage. That is, systems can be developed in a shorter since it is possible to make use of rich middleware products and applications.

SymfoWARE’s high-speed search and load enabled by its advanced parallel processing techniques

The basic architectures that SymfoWARE Parallel Server employes for parallel processing are partitioning and pipeline control. SymfoWARE Parallel Server achieves high-speed processing by distributing data and load like an assembly line. SymfoWARE Parallel Server makes use of full partitioning and full pipeline processing.

Partitioning and pipeline control

sortjoin

scan

data

data

data

Full partitioning + Full pipeline

Partitioning + Non-pipeline Partial partitioning + Non-pipeline

join

data

data

data

data

scan

data

data

sort

data

data

data

joindata

sort

data

scan

data

data

data

Figure 14 Parallel processing architectures

SymfoWARE Parallel Server has a number of unique features including the following: • Full-phase parallel processing for high

speed processing with parallel search operations

• Automatic data balancing for storing data evenly on different disks

• Dynamic load balancing for improving the efficiency of using multiple CPUs

• Parallel super loader for faster database creation and update processing

With these features, SymfoWARE Parallel Server can far outperform its competitions. Compared with the performance of other vendors’ products in the same hardware environment, SymfoWARE can load five to twelve times faster and search twice faster. And there is one more important point about the high-speed performance of SymfoWARE Parallel Server: it exhibits best performance with minimum resources. CPU usage is half that of other products, and memory usage is one-third. Even with the same hardware configuration, SymfoWARE can deliver the same processing power as others, and still have capacity to spare. SymfoWARE Parallel Processor also contributes to high-speed search and load processing. The following figures show that SymfoWARE is far superior to other products in a search process with 120 million items of data. It also shows that SymfoWARE displays linear performance improvement as the number of parallel processors rises. How you can make strategic business decisions promptly may depend on how you make use of your accumulated data. SymfoWARE Parallel Server facilitates the fast analytical processing of large volumes of customer data and their transaction data. SymfoWARE Parallel Server can play a significant


12

SymfoWARE RDBMS

role as a tool for implementing your rapid decision support environment that will be needed in future.

♦ ♦ Search performance: 2 times (maximum 3.3 times)

♦ ♦ Load performance: 5 to 12 times

SymfoWARE (12 months: m)

Other DBMS (12 months: m)

Total time for search processing (21 patterns)

SymfoWARE 12 CPUs

Other DBMS 12 CPUs

SymfoWARE 8 CPUs Other DBMS 8 CPUs

SymfoWARE (1 month: 1)

Other DBMS (1 month: 1)

→ Hour

→ Hour

1

3.5

11.8

15.5

1

1.26

1.82

2.78

Figure 15 SymfoWARE’s outstanding performance

♦ ♦ DISK access method

…

IOU IOU IOU IOU …

SymfoWARE No competition

for DISK access!

…

IOU IOU IOU IOU …

Competition for

DISK access!!

♦ ♦ CPU usage : 1/2 of competing products

SymfoWARE 12 CPUs

Other DBMS 12 CPUs SymfoWARE 8 CPUs Other DBMS 8 CPUs

♦ ♦ Memory usage : 1/3 of competing products

SymfoWARE 12 CPUs Other DBMS 12 CPUs SymfoWARE 8 CPUs Other DBMS 8 CPUs

→ Hour

→ Hour

1

1.53

1.84

2.93

1

1

2.98

2.90

Other DBMS

Figure 16 SymfoWARE’s superior technology

Increased adoption of SymfoWARE by many organizations as the base infrastructure for their business

Many organizations have recognized SymfoWARE’s cutting-edge functions. Here are the two latest example success stories of SymfoWARE implementation. ♦♦♦♦ SymfoWARE’s selling points for

financial systems: highly flexible expandibility and high-level availability through switching in seconds

The first example is a case of SymfoWARE adoption in a financial institution. Japan is still undergoing the process of deregulation following the Financial Big Bang. For this reason, financial institutions cannot simply leave their systems exclusively tied to conventional mainframes any more. In the system being discussed, an application

server and a database server were placed on the junction connecting the company’s accounting system with the sales branch system. SymfoWARE was adopted as the RDBMS for this system. This was because the institution recognized SymfoWARE’s ability to switch over in seconds in the event of a system failure, and to handle more than 1000 simultaneous transactions. Currently, the institution is running a 4-node redundant configuration with the parallel cluster option. Another main reason for adopting SymfoWARE is that the system is highly flexible in the sense that it can be easily extended by adding new nodes.

…

Sales branch system

PRIMEPOWER M600 family

App server PRIMEPOWER M600 family

SymfoWARE Enterprise Edition SymfoWARE Server

parallel cluster option

All-node redundant configuration by applying SafeCLUSTER

DB server

- The performance of both the applicationserver and the database server can be enhanced by adding more node

…･

Host

Figure 17 Load sharing example in a large online system

♦♦♦♦ SymfoWARE for Web and other

leading-edge distributed environments

SymfoWARE is also adopted in Web-based computing environments. In one adopter’s case, the accounting applications run on the Web. The system was built in a distributed application environment with such advanced distributed technology as CORBA. In this case, the user chose SymfoWARE because of its high adaptability and readiness for the latest technology and its excellent cost performance.


13

SymfoWARE RDBMS

Enterprise AP server/DB server

(Solaris)

COBOL application

WWW browser

Java applet for jobs

Download

DB

for update

Dept. AP server/ WWW server

(Windows NT)

INTERSTAGE

WWW CORBA

CORBA transmission

PC

(Windows)

COBOL application

CORBA transmission

DB for

reference

WAN

SymfoWARE Server

SymfoWARE Server

INTER

STAGE

CO

RBA

Figure 18 Example Web-based system

SymfoWARE Universal Data Accessor for transparent access to legacy systems and databases on different platforms

As the database engine, SymfoWARE positions itself as the core product in the SymfoWAREfamily of various products. SymfoWARE Universal Data Accessor (UDA) is a member of the family that allows transparent access to databases that are scattered under different systems on different platforms. With the spread of the Internet/Intranet environment, more and more information systems are developed on open systems. On the other hand, there are still many companies that use PRIMEFORCE for large sized mission-critical system applications. It cannot be said to be bad to make creative use of different platforms for different purposes. As long as each system can be operated efficiently, it can be considered quite legitimate to have a heterogeneous environment. However, it may lead to some problems. The biggest problem is whether it is possible to utilize the data scattered on multiple systems in the network by organically linking them together. Although each system functions well independently, the pieces of data that these systems are handling may not be very useful until they are integrated. It should be possible to integrate the data between different systems to allow business users to make good use of their data. UDA provides transparent access to data stored on different platforms. UDA was developed on the basis of Java and object technology, and is used as an important tool for SymfoWARE’s Web solutions. UDA makes it possible for SymfoWARE to be accessed by a Java application under an application

server such as Fujitsu’s INTERSTAGE. Since it has a legacy data connection function, there are no problems in accessing not only SymfoWARE on an open server and PRIMEFORCE but also SAM and VSAM data. UDA can also make it possible for SymfoWARE to access other RDBMS systems with third-party middleware products. Even when many different types of databases coexist in your environment, you can access them from the client in a consistent manner. When this sort of system was developed with Java applications in the past, it was common to use JDBC drivers supplied by the various database vendors to access different databases. However, it used to take a lot of time and effort to develop and test such a system, and thus it was very difficult to develop systems quickly. As the UDA APIs are database independent, you can develop one integrated access mechanism even in an environment where many different systems are operating. Moreover, there is no need to make any adjustments to the destination systems. As the business environment changes, it is also necessary to make information systems more flexible. There is a growing demand to provide data stored on PRIMEFORCE to end-users, and to combine and use data stored on multiple systems. Very few products can offer such open connectivity. This often disrupts an organization’s effort to start up applications for new businesses. UDA can help you to build a high-level information system environment regardless of the differences in platform and database.

Client Application server (INTERSTAGE) Global server SymfoWARE Server SAM/VSAM

JAVA applications

WWW Server

Servlet

SymfoWARE Universal Data Accessor

Class library

JDBC Driver

Gateway

SequeLink

JDBC Driver

Open server SymfoWARE Server

Other vendor DB servers DB2, Informix, Oracle, MS SQL Server etc.

Figure 19 UDA


14

SymfoWARE RDBMS

SymfoWARE Universal Data Interchanger for efficient data interchange between different types of systems

As we noted in the section on UDA, these days it is common to find different types of platforms coexisting in the same enterprise system environment. UDA is used to access data effectively, but there is an increasing need to be able to exchange data itself between different systems. For example, there can be a case in which data is collected from the mission-critical system and from multiple servers, and then the collected data is stored in a data warehouse. In another case, some data in a database for one application may be utilized by other database applications. Of course, these kinds of data interchange requirements do not arise between in-house systems only. They may arise across company boundaries for inter-company applications such as EDI (Electronic Document Exchange) and SCM (Supply Chain Management). In such cases, data are often stored on many different types of storage medium and in different data format depending on the system. This can be a deterrent factor to the efficient use of the data. SymfoWARE Universal Data Interchanger (UDI) solves these problems. UDI extracts data from information systems on a variety of platforms, and then transforms and stores it on another platform. UDI can be used with many enterprise information systems because it supports a wide range of platforms including mainframes, office computers, UNIX, and Windows NT/2000. UDI lets you extract, transform, and store various forms of data - SymfoWARE and other vendor RDBMS, binary files, CSV files and Internet multimedia databases such as Jasmine. UDI also supports XML, which is the standard data format for the next generation. UDI can be used in XML-based EDI systems, which are likely to become very common. UDI lets you collect XML data on Web servers using the Internet.

UDI

Transform

Load

Extract

New application system Use of information

Various data formats

Internet

Windows Mainframe Other company’s public Web server UNIX

XML documents

SymfoWARE VSAM / SAMAIM/DB

SymfoWARE Oracle SQL Server Jasmine XML

SymfoWARE SQL Server Jasmine XML

Various data formats

Figure 20 UDI

♦♦♦♦ Advantages of UDI Let’s discuss the advantages of UDI with an example situation for building a new accounting application system. In recent years many companies in Japan have been implementing new accounting applications to meet international accounting standards. These applications extract and process accounting data from their existing in-house accounting systems and also from their affiliated companies‘ systems. It is common that these source application systems manage their data in different formats. This means that the data must normally be transformed internally before they are aggregated, the codes must be standardized, and the quality and integrity of data must be checked. UDI transforms target data and codes when extracting the data is extracted from a source application system. UDI allows the data to be moved to the new accounting system directly without any additional processing. If you try to build this data interchange capability in your application, it will require considerable time and effort. With UDI, however, necessary data can be made available immediately to a requesting application from the source environment. UDI makes it possible for an application to obtain necessary data in a seamless manner, without regard to individual systems and the company system configuration. UDI is an immensely powerful tool for users who want to rapidly build new applications that use existing data.

JDBC=Java Database Connection CSV=Comma Separated Value XML=eXtensible Markup Language


15

SymfoWARE RDBMS

SymfoWARE’s continuous evolution for the 21st century

Much is expected of IT in the 21st century. The following three points will be of particular importance in responding to the changes in the business environment: • Improved performance

The number of IT users will grow even faster in future. This means that future systems will be required to be more powerful and more scalable to cope with the new situations.

• Larger databases In addition to large amounts of customer data, future database systems will need to deal with new types of data, such as large volumes of multimedia and XML format data. This will call for very large database systems.

• More robust systems for 24 by 7 operations As the ranks of IT users grow and the businesses globalize, future systems will need to be more robust for 24 by 7 operation with higher security measures.

To meet these expectations, Fujitsu is considering the following functions for SymfoWARE.

High performance solutions

Currently, TCP/IP is used to connect networks between servers. Fujitsu is working on a new network framework that we tentatively call SymfoWARE Server AP-Highway. This is for realizing more powerful systems by connecting to virtual interface networks.

Built-in VIA driver on the same layer as TCP/IP driver

SQL processor

RDB_TCP

TCP/IP VIA

Socket Synfinity-VIA

TCP/IP driver VIA driver （ New

LAN,Gigabit

Ethernet AP-Net

SQL runtime

RDB_TCP

TCP/IP VIA

Socket Synfinity-VIA

TCP/IP driver

AP

DB server

AP server

VIA: Virtual Interface Architecture

VIA driver New

Figure 21 SymfoWARE Server AP-Highway (provisional name)

To improve the performance of a large system, it is necessary to improve the efficiency of the logging mechanisms and the operation of the database directory. The solution to this is the adoption of advanced log files. With advanced log files, log data can be partitioned and multiple directory files can be introduced for load sharing, which will improve OLTP performance.

Guaranteeing scalable OLTP performance for the number of CPUsand broad improvement in MTTR

Load concentration Limits on improvementsin MTTR (Mean Time ToRepair)

Single log fileSingle log file

TLF ALFLMFDatabase

LMF: Log Management FileTLF: Temporary Log FileALF: Archive Log File

Advanced log fileAdvanced log file

TLF ALFLMFPartitioning group 3

TLF ALFLMF

Partitioning group 2

TLF ALFLMFPartitioning group 1

Figure 22 Advanced log files

Large DB solutions

Let’s discuss the new approach to the backup/replication and data storage. When it comes to database backup, it is utmost important to be able to back up data as fast as possible without loss. Fujitsu is developing the EC (Equivalent Copy) function to do this. With the EC function, logical volumes will be mirrored to provide redundancy. At a given point, the redundant part will be released and used for backup. By using redundancy technology, it will be possible to perform backup without overloading applications. The EC is a base technology for full backup, and will be used for new backup solutions such as SymfoWARE Server Hyper-Backup (provisional name), and SymfoWARE Server Hyper-Replication (provisional name).

BLOB=Binary Large Object OLTP=OnLine Transaction Processing DB=Database


16

SymfoWARE RDBMS

MTL etc.

Back up by Equivalent Copy Function

Transition

Back-UP Standby restoration

NSP-AFS

Primary

Secondary

Primary

Secondary

Primary

Secondary

Not subject to mirroring

Subject to mirroring

Saving on to secondary medium by standalone NSP without affecting the application server.

- Assurance of data coherence and archiving of diff log

SymfoWARE Log Repair start Point log

Update lLog

Update Log …

Update Log

Figure 23 High-speed backup with NSP-AFS, and

SymfoWARE Server Hyper-Backup (provisional name)

There are two points to be noted on the approach to data storage: advanced BLOB and XML features. Future OLTP systems will have to deal with multimedia data, such as image data. The advanced BLOB feature of SymfoWARE will be able to manage image data. This function will allow flexible application development for image data. The advanced XML feature will be provided for handling XML, which will be used a lot more in future. You will be able to select an interface for storing XML according to the purpose for which it is used. With the advanced XML feature, you will be able to select a storage format for use with each XML document, whether to store it in XML format or in a code format after code conversion.

XML

document

XQL:XML query language

SQL interface

Direct storage Store directly Conversion

storage

Store after conversion

XML

document

Document format

Tag index

RDB RDB

Numeric data

Table format

XQL SQL XQL

Figure 24 Advanced XML

24 by 7 operation solutions

Non-stop 24 by 7 type operation is already realized through cluster technology. In the future, it will be possible to provide continuous operation without stopping the system for database maintenance and operation. In other words, it will be necessary to aim for a maintenance-free database. To achieve this, the following approaches are being considered for SymfoWARE: • Removing the necessity for index

reorganization • Optimization of access plans for a 24-hour

operating environment • Allowing the dynamic addition and

modification of columns and indexes so that application services can be expanded without stopping the DB servers and other applications

Fujitsu is also looking at high security functions to prevent break-ins into systems and the destruction of data. SymfoWARE keeps evolving to meet the demands of the times - it will be the enterprise RDBMS system product for the 21st century.

User definition User definition Audit report Audit report

Audit log

of permissions

Granting and revoking rights

CREATE USER GRANT/REVOKE

User authority

range check

User right range check

User certificate User authentication

CONNECT statement Data manipulation satatements

AP

Data server

Audit log earnings

SymfoWARE Server Enterprise Edition/Secure

Dictionary

Taking audit log

Figure 25 Security structure


17

SymfoWARE RDBMS

Working together to offer the best solutions for new business development

SymfoWARE is an extremely powerful tool for developing customers’ strategic business solutions. However, our partners’ knowledge and experience are also indispensable in enabling us to offer our customers truly excellent solutions.To help our customers to use SymfoWARE in building their successful business solutions in the Internet, we at Fujitsu have established a close collaborative relationship with the partners. We will use all possible endeavors to provide you with the best solutions.

♦♦♦♦ Trademarks Java and other Java related trademarks or logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries. Solaris is a trademark of Sun Microsystems in the United States. SymfoWARE is a trademark of Fujitsu Limited. UNIX is a registered trademark of X/Open Consortium. Microsoft, Windows, Windows NT, and windows 2000 are registered trademarks of Microsoft Corporation in the United States and other countries. Other product names are generally trademarks or registered trademarks of the respective companies. Screen shots used within this manual comply with the copyright guidelines of Microsoft Corporation.

Documents

Scalable Databases for Mission-Critical Business Applications · Scalable Databases for Mission-Critical Business Applications ... and across whole industry sectors, using IT