Specification-Driven Development

An Overview

Purpose

• Development Productivity

• Portability

• Evolvability

Purpose - Productivity

• Development productivity from:– Working at a high level of abstraction;– Generation of all predictable elements; and– Focusing of skill on areas where needed.

Purpose - Portability

• Achieve portability across changes in:– Language;– Platform; and– Deployment Architecture.

Purpose - Evolvability

• Provide evolvability across changes in:– Corporate standards for appearance & behavior;– Specifications for functions & system; and– Fundamental technology.

Sounds good, but how?

First some definitions...

Definitions

Application suite - A group of applications which together provide the computational functionality required to run a company.

Definitions (cont.)

Application - A tightly integrated group of functions which cover one specific area of the overall requirements, e.g., order processing or accounts receivable.

Definitions (cont.)

Function - One indivisible component of an application, i.e., that which one selects from the menu for the application.

Simple enough, but so what?

Now some context...

Context

• Current discussion is focused on enterprise-class transaction processing applications.

• Applications are developed by either:– Corporate development groups responsible for

mission-critical applications; or– Companies developing such applications for

resale.

Common Problems

• Design to master plan which constantly changes.

• Need to preserve investment in software over changes in technology.

• Deployment environment and architecture is constantly changing.

• Demand for features exceeds capacity to fulfill.

Ouch! All too familiar, but is there anything you can do about it?

Let’s talk about layers of information...

Layers of Information

• Unstructured user requirements.

• Formal system specifications.

• Architectural design - data & functions.

• Function design.

• Implementation design.

• Executable form.

Unstructured User Requirements

• Focused on individual elements of problem.• Based on past experience - good and bad.• Stated in operational, not computational terms.• May include mutually exclusive requirements,

especially from different users.• Rarely includes corporate standards.

Formal System Specifications

• Resolves user requirements into integrated whole including procedural flows.

• Corporate standards are applied & expressed.• Defines target functionality of system, but not

how it will be implemented.• Change over time as business needs change.

Architectural Design

• Computational form of specifications.

• Includes:– Data structure;– Organization of operations into functions; and– Description of relational integrity constraints,

algorithms, and other business logic.

So now we start coding, right?

Let’s keep looking at layers for a moment...

Function Design

• Identify and characterize the components of the individual function including:– Data elements involved and their relationships;– Functional operations required; and– Business rules involved.

Implementation Design

• Maps functionality on to current technology and programming philosophy.

• Heavily determined by target platforms and deployment architectures.

• Also strongly influenced by shop standards and practices.

Executable Form

• Now, we have actual code.

• Code embodies standards:– Functionality;– Appearance; and– Implementation.

• Much of the code is specific to a target platform and deployment architecture.

Sure, I suppose that’s true, but how does breaking it into layers help?

Let’s think about tools that might help with each layer...

Design Repository

• A place to organize user requirements leading to a specification.

• No formal methodology implied or required.

• Provides background of why a design is the way it is.

Application

Purpose

ActivityRelationships

Requirements

Activities

ApplicationRelationships

Recording and AnalyzingUser Activities

Application

Purpose

FunctionUsage

Requirements

Functions

ApplicationRelationships

ActivityRelationships

Activities

FunctionRelationships

Analyzing User Activities into a Functional Structure

Activities

User ViewsData Class

Table

DomainsIndex

JoinsColumn

Analyzing User Perceptions of DataInto a Data Structure

User Requirements to Architectural Design

• Start simply by using it as a structured way to take notes

• Organize these notes into elements of architecture

• Documentation of architecture provides specification

• Connection to requirements documents purpose and intent

Design Repository Toolset

• Requirements Repository– User requirements, activities, and views

• Extended Data Dictionary – Data structure, relationships, & properties

• Application Dictionary– Function relationships and specifications

DesignRepository

ExtendedData

Dictionary

ApplicationDictionary

ApplicationBuilder

Design Repository Toolset

Dependencies

• Requirements Repository not required for Application Builder - can add legacy information later

• Extended Data Dictionary and Application Dictionary provide “hints” that facilitate the use of Application Builder and can be used separately

• No formal methodology assumed

OK, I see how that could be useful, but what’sthis Application Builder and where does the productivity come from?

Let’s think about what goes into writing a program for one of these functions...

Layers of Information

• Unstructured user requirements.

• Formal system specifications.

• Architectural design - data & functions.

• Function design.

• Implementation design.

• Executable form.

FunctionDesign

ImplementationDesign

Executable

Form

Sources of Design Information

Extended DataDictionary

ApplicationDictionary

Shop Standards& Practice

Platform &Architecture

Application

Framework

SpecialRoutine

s

But, which of these don’t include any code, merely guidelines of how the code is to be written?

FunctionDesign

ImplementationDesign

Executable

Form

Where is actual code?

Extended DataDictionary

ApplicationDictionary

Shop Standards& Practice

Platform &Architecture

Application

Framework

SpecialRoutine

s

And which contain code that is unique to this function?

FunctionDesign

ImplementationDesign

Executable

Form

Where is actual code?

Extended DataDictionary

ApplicationDictionary

Shop Standards& Practice

Platform &Architecture

Application

Framework

SpecialRoutine

s

Well... sure that is the only part that is really unique,but doesn’t a programmer have to pick and chooseamong the components and glue everything togetherand all that?

Not if we get the tool to do it!

You mean one of those generator things?

We tried one of those, but it wasn’t nearly flexible enoughand so we ended up hand modifying all the code to getwhat we wanted and needed.

Nope!

Some kind of class library then from which you cansupposedly just assemble the application?

We tried that too and by the time we got all the functionalityand variations we needed, no one could find anything andwe kept reinventing things we already had and breaking thingsthat used to work by tinkering with them.

Nope!

Time for a little magic...

Well then, how?

If we have used the Design Repository, or just loadedthe Extended Data Dictionary and Application Dictionarywith the specifications for the function we want to create,there is already a lot of information in the system aboutwhat this function will need to do, including:• Which files are accessed.• Where the function fits in the system.• What type of function it is.• What business rules apply.Some of this information is directly usable, some isjust in narrative form.

So ...

The Application Builder

The developer-user is first presentedwith two windows, one for logicand one for appearance.

Based on the Extended Data Dictionary and Application Dictionary the Application Builder builds a simple default specification for the function based on the function type and the files accessed.

What is a specification?

A specification is a description of the basic functionalityusing a special language. The “words” available for this description are defined by an underlying specificationwhich is not accessed by most developers. All descriptionsare very generic, i.e., instead of indicating that a set of fields should appear together in a tab folder or in a frameaccessed by a strip menu, they are simply indicated as being a group.

While the specification is recorded as text, it is manipulated graphically with reference to a mastertemplate for that specification type.

MasterTemplate

GraphicalDisplay

FunctionSpecifications

Think of these specifications as the kind of instructionswhich a senior programmer might give to a juniorprogrammer, using his or her experience to translatethe operational specifications provided by the user intoa function design, but one which can be communicatedin general terms because the junior programmer is familiar with what the senior one will want.

Then what?

Virtual ObjectLibrary

FunctionSpecifications

Common Specifications

FinishedPrograms

ApplicationBuilder

We create the programs!

Wait a minute!

What about the visualization and the special logicand the deployment platform and the display typeand the corporate standards and what’s a “virtualobject” anyway? Some object that isn’t reallythere?

I was coming to that, one piece at a time...

Virtual Objects

• Virtual Objects are a mixture of code in the deployment language and Virtual Object Description Language (VODL).

• VODL allows that Application Builder to “tailor” the objects to suit the needs of the function instead of maintaining a full class library.

• Virtual objects use conventional objects as components, but are not limited to them.

More Virtual Objects

• Virtual Object code is always accessible to the architect to change the way the specifications are implemented.

• Fresh programs can be created at any time based on current specifications and revised Virtual Objects and/or common specifications.

• Virtual Objects map the general onto the specific, i.e., “group” onto “tab folder”.

Corporate Standards

• In part, are provided by the code in the Virtual Objects which define how function elements are implemented.

• Common specifications are included in each specification by implication and provide readily changable site for common variations and preferences.

Well, I’m not sure how that all works, but how can this specification language covereverything? There are always exceptionsand special conditions and complications.Otherwise, programming would be juststicking things together like they promisedus with OOP.

True! Therefore...

Special Requirements• Virtual Objects are provided with ample opportunities

to supplement or override any of the standard behaviors.

• Special code is written directly in the target language.• All special code is tightly encapsulated with simple

interfaces for ease and speed of development.• Repeated “special” needs can be added to

specification language and become standard.

So, what are we really talking about here?How much of the application comes fromthe Virtual Objects and how much do Istill have to write myself?

Currently about 90-95% of the entire applicationcomes from the Virtual Objects, but we areconstantly improving that.

95%! We must be talking about pretty simpleapplications right? Not the really tough stuffI have to do!

Let’s talk for a minute aboutour design goals....

Design Goals

• Progressive Re-engineering.

• Facilitated Analysis.

• Facilitated Development.

• No Compromise Functionality.

• Plan for the future; work in the present.

Design Goals (cont.)

• Progressive Re-engineering means that the system was designed to be used alongside legacy code and to provide a simple way to move existing applications within the tool as well as creating new applications.

Design Goals (cont.)

• Facilitated Analysis means supporting the developer in arriving at a design without forcing the use of any special methodology.

Design Goals (cont.)

• Facilitated Development means doing everything we can to replace routine work, increase productivity, and focus the programmer’s efforts on those areas where his or her skills are the most needed -- not just in initial development, but over the life of the function.

Design Goals (cont.)

• No-Compromise Functionality means that no tool is any good if you have to compromise your design goals in order to use it. If you can’t get from 99% to 100%, you have nothing.

Design Goals (cont.)

• Plan for the future; work in the present means focusing on what can be done to deliver immediate benefits today, but doing so with a master plan that will insure a graceful transition to a future tool environment in which one can do even more.

OK, I like your goals, but what about the visualization?

Fair enough...

Visualization

• Visualization is also standards-based.

• Like the logic, a master set of rules is manipulated graphically to produce a set of specifications in Visualization Description Language.

• Specifications are in terms of relationships and rules, not details like pixels and rows.

DisplayEnvironments

VisualizationSpecifications

Common Specifications

FinishedDisplay

ApplicationBuilder

We create the display!

More Visualization

• Company defines a set of target display environments - resolution, colors, font size, etc.

• Rules-based specifications for visualization are interpreted relative to the characteristics of each display environment.

• New display environments or global changes in standards are easily incorporated.

This is sounding better and better, but whatabout changes in deployment architecture?We’ve been going crazy in my company! We just get done moving part of our applicationsto client/server and now they’re talking about N-Tier and I know that will change everything.

And this is where the real rewards come in ...

Deployment Architecture

• Because the program architecture is a property of the Virtual Objects, change them and the whole application changes with the next expansion.

• Encapsulated special code is rarely affected, but even if it is, changes are simple and rapid.

But this is all theory, right! Big promisesfor some tool that you think you can make,but we won’t really see for two years andeven then won’t do half what you promise.

On the contrary...

Proven Technology

• Used to create over 1 million lines of Progress 4GL - the approximate equivalent of 10 million lines of COBOL.

• Applications are in daily use in high transaction volume environments on multiple architectures.

• Applications are equal to or better than any available, richer than most exactly because of the tool.

But suppose I don’t like Progress or can’tsell it in my company?

Here’s where the magic getsthe most impressive...

Language Transportability

• Because the specification language is independent of any target language, the same specifications can be used to create applications in any reasonable target language.

• Since 95% of the application is created from the specifications, the only porting required to change from one target language to the next is the special code - a small and decreasing fraction.

• Visualization is also language independent.

So, just how broad is this range offunctionality you are offering?

Initial Commercial Release

• Deployment architectures will include host/terminal, client/server, and N-Tier.

• Deployment user interfaces will include graphical and character.

• Deployment platforms will include Windows, NT, Unix, VMS, and AS/400 - depending only on language.

• Languages will include Progress 4GL and Visual Basic with others to follow.

In Summary

• Development Productivity

• Portability

• Evolvability

Development Productivity

• Measured rates well over 10 times other technologies in initial development and increasing; greater yet over long term.

• High fraction of solid, proven code lowers debugging and reduces maintenance and support costs.

• Downsteam changes at a small fraction of re-development costs.

Portability

• Language independent specifications massively reduces cost of move to a new language.

• Already covers all major target platforms.

Evolvability

• Greatest long term benefit comes from ability to adapt to new technology and techniques without significant change to code.

• Makes re-inventing whole application suites affordable.

Questions?