Neural nets are starting to make sense

Biosensors & Bioelectronics Vol. 7 No. 9 (1992) 621-626

NEURAL NETS ARE STARTING TO MAKE SENSE

by Jo Ann McDonald

America is perceived as being in the forefront of the development and practical application of neural network devices, which may or may not be true since there is exceptional work going on in the field in Asia and Europe as well. But America certainly is engaged in some exciting entrepreneurial work in the field, with one company in particular, Synaptics, based in San Jose, California, making excellent inroads in the commercialization of neural net integrated circuits.

S ynaptics was founded in 1986 by two of the electronic industry’s most famed pioneers,

California Institute of Technology Professor Carver Mead and Dr. Federico Faggin. Among their many collective attributes, Professor Mead was the inventor of the MESFET integrated circuit and silicon compilers, as well as authoring what is now the standard text for VLSI design. Professor Mead is highly regarded in the US and globally acclaimed as a major catalyst in the field of neural networks. Dr. Faggin was responsible for many Silicon Valley microprocessor design “firsts” while first at Fairchild Corporation, then at Intel. In 1974 Dr. Faggin co-founded Zilog where he conceived the 280 microprocessor central processor unit (CPU). Fairchild, Intel and Zilog are three of the most important semiconductor manufacturers in the world.

In addition to their recent launch of Synaptic’s first commercial neural network application of intuitive computation, the I-1000 Neural Eye chip, which has become the heart of VeriFone’s Onyx Check Reader product, the two gentlemen have authored numerous papers on the subject of neural network-based analog computing. Together, through Synaptics, they are making intuitive computation a commercial reality.

INTUITIVE COMPUTATION IN SILICON

The study of natural, human sensation is the basis of Carver Mead’s investigations of adaptive learning, which prior to neural nets, humans could do, but computers could not. By examining the initial layer of neurons behind the photodetector of a human eye, for example, Carver Mead concluded that the type instantaneous processing which nature provided the eye represents the bulk of the brain’s neural activity.

What the company, Synaptics, was formed to provide was a way to embody the brain’s intuitive computation methods in silicon chips. These artificial neural networks provide the architecture for intuitive computation and adaptive analog VLSI (very large scale integration), which is the new hardware technology that is holding the international computation research community spellbound.

An increasing number of integrated circuit pioneers in America are cashing in their digital credentials these days to pursue the intriguing potential of real world analog. Why? After years of artificially digitizing and quantifying, the analog world simply makes more sense. Common sense. Human versus machine type sense. One has only to look at the difference between an analog

0956-5663/92/$05X) 01992 Elsevier Science Publishers Ltd. 621

Biosensors & Bioelectronics Vol. 7 No. 9

versus a digital watch to easily grasp the concept. A digital watch may tell you the precise time, but it does so with no reference to the day as a whole. Once delighted with their novelty of such a handy application of digital wizardry, millions of digital watch owners migrated back to the relative security and warm familiarity of one with a sweep hand showing 12 numbers in relation to one another at a glance.

Thus Synaptics Incorporated began on its quest to develop the architectures, hardware technology, software and design tools for cost-effective applications of intuitive computation. Since 1986 the company has developed proprietary technology for massively parallel computational structures capable of image sensing, real-time sensory pie-processing, adaptive feature extraction, pattern classification, temporal sequence interpretation, machine learning and automatic generalization.

THE NEXT GENERATION

The overall goal of perfecting neural networks is to tap the potential of these exciting devices, for they bring to tomorrow’s computational machine capability what the microprocessor brought to the world in the mid-1970s. When you think about it - and please pardon the pun, because that is the whole point of investigating the potential of neural nets - they will allow machines to think more like humans do.

Heretofore, computers were, and still are, simply large dumb adding machines. 1 and 0 is all they can handle, and all they ever could. Of course they do it incredibly fast, and with clever programming, they can perform miracles that we simply could not achieve without them.

How well we all remember early stage computer wizards sitting in a heavily chilled fish bowl rooms loading stacks of paper cards into a monster machine. One lonely man, one huge computer taking hours and hours of collective labour to achieve what we can do in a moment now on our own individual desktop.

Why do we want more? Because we want our next generation of machines to anticipate, not just

be patient and do as we ask. We want them to be able to tidy up after us and smooth over our shortsightedness. Things of that sort. We don’t want them to think for us so much as to think as we perhaps forget to, or don’t want to. Trivia, tedious tasks, those shorts of things, freeing us for more important or more pleasurable pursuits. The spell-checker or thesaurus function on your word processor, for example is but a hint of things to come.

There are a myriad of reasons for machines that think over those that cannot. The most important of which is that machines powered by neural nets don’t have to be programmed. But more of that later, for we’ve only begun to contemplate what we might do with these new and smarter beasts. The task at hand is understanding the goal and how to collectively begin.

CRAWLING ALONG THE FRONTIERS

As in bringing any new concept or art to fruition, one must learn to walk before running, and at this stage of development, the field of neural network development could be likened to crawling along the frontiers of science with a magnifying glass. Synaptics’ founders have been in this situation before with the microprocessor, and as such are wisely looking to relatively low-level, or simple applications of their initial efforts to attend to what so many others tend to neglect... bringing in the revenue necessary to progress to the walking and running stage. A very practical approach to a not-so-practical field.

Working in partnership with major suppliers of end-user products, Synaptics applies what certainly is an esoteric technology to solve everyday problems in object and character recognition, vision systems, man-machine interface and adaptive systems, with an emphasis on applications that benefit from portability or low power consumption. Synaptics develops and manufactures intuitive computation components and modules that use a combination of neural network architectures, adaptive analog VLSI chips and software, and because of their extraordinary success in the microprocessor

622


industry, the two founders discovered a low cost mass market application to get the ball rolling in the right direction.

FIRST COMMERCIAL APPLICATION OF A NEURAL NET CHIP

The first such partnership produced the company’s first commercial product recently, a neural net chip built with adaptive analog VLSI technology, called the Synaptics I-1000 Neural Eye chip, see photo below.

The chip uses optical rather than magnetic sensing and, like the human eye, “sees” the entire detailed image of a character simultaneously with

one viewing. It doesn’t have to bother “digitizing” it. Consequently, the first application has been for a point-of-sale cheque reader called “Onyx” manufactured and marketed by VeriFone Inc, a company based in Redwood City, California, USA. Verifone is a company with $188M in annual revenues and holds about 64% market share for credit card verification machines used in retail.

Since its introduction, the phones at Synaptics and VeriFone have been ringing nonstop. The product is not only affordable, it is expected to sell for around $250 apiece, but also accurate. That simply was not possible before the existence of Synaptics’ neural net chip.

The world’s first commercial product of its kind, the neural net chip built with adaptive analog VLSI technology, called the Synaptics I-l 000 Neural Eye chip

623


Optical sensing is the critical component that delivers very high accuracy in the expensive cheque readers used in banks and check clearing houses. In contrast, the comparatively inexpensive I-1000 chip harnesses the higher information content of optical sensing at a price affordable for each VeriFone point-of-sale customer. The product has been made so affordable that any store can purchase one. Bad cheques at grocery stores can be a thing of the past. (Currently cheque reading machines have over 45% error rates).

It works almost exactly like simple machines that instaneously read ones magnetic tape bank or other credit cards, handling the account numbers at the bottom of any check with amazing accuracy.

ANALOG LEARNS FROM DIGITAL MARKETING

The secret of this initial product’s mass market success is due to the capability of Carver Mead and Federico Faggin to develop adaptive analog VLSI technology for their neural networks, based on standard CMOS chip fabrication processes that evolved from high volume digital computer chip manufacturing. By inventing new circuits and architectures, but avoiding exotic processing technologies, Synaptics has brought to analog technology the advantages of low cost, wide availability and proven reliability that results from the CMOS digital learning curve. The strategy is quite brilliant, and proving to be quite successful.

HOW SYNAPTICS’ ARTIFICIAL NEURAL NETS WORK

Unlike the human brain, which is obviously the ultimate and most efficient natural neural network known, artificial neural nets perform “intuitive” computation. Rather than being programmed with a set of rules, they learn by example, much the same way our own minds learned as children. So neural nets are adaptive and reach conclusions by using a very large number of individual, highly

interconnected circuits, called neurons, to simultaneously process a huge amount of input data in the light of prior experience.

The revolutionary implication of this is the ability of machines to learn by example, freeing us from having to solve, our many complex problems by existing rule-based techniques. Much of what makes the type neural nets Synaptics produces so special is that they build on what the company likens to the “gut feel” of experts, or accurate intuitive judgments based on experience. Another way of putting it is that these are the type intuitive conclusions that are difficult for those experts to describe precisely.

Heretofore research into neural networks has been performed using models built in software, thereby simulating a neural network with a digital computer. But this approach has limitations in any practical application that needs quick results, because the computer computes the effect of each interconnection, or synapse, one at a time, whereas nature does it all simultaneously. Nature performs many of its useful functions using billions of neurons, each having thousands of synaptic connections to other neurons. To fully exploit the neural net structure, it is necessary to develop electrical circuits that approach nature’s circuits in density, speed and efficiency in order to implement similar complexities.

Digital circuitry simply can’t accomplish the task. Adaptive analog circuits, however, can perform complex computations by exploiting the physics of semiconductors, which often mirror the equations governing the chemistry of biological neurons. In appropriate structures, this can lead to advantages of lOO-to-1 in function density and 10,000-to-1 in power consumption per function, compared with digital circuits. Even with these advantages, analog circuits cannot match the density and efficiency of nature, but the performance of adaptive analog VLSI offers such a dramatic leap beyond digital circuits that it opens up vast new realms of powerful everyday applications beyond simply cheque readers.

624

Biosensors & Bioektronics Vol. 7 No. 9

What lies behind the uniqueness of Synaptics’ technology is an intricate set of architectures, algorithms and adaptive analog devices, circuits and subsystems. The hardware embodiments are, as mentioned above, based on CMOS VLSI silicon, i.e. non-biological, fabrication methods, combining both digital and adaptive analog techniques to achieve up to 100,000 processors, each with its own data storage, on a single chip.

Each elementary processor performs one or more complex operations in one to ten microseconds, resulting in an equivalent computational power on each chip of up to 100 billion operations per second per watt.

To do this, Synaptics developed a variety of elementary processors that perform such computations as multiply-add, min, max, medians, Gaussians, hyperbolic tangents and many other functions. The adaptive elements consist of polysilicon floating gate structures capable of storing, sensing and adjusting analog values. Adaptive elements are used for learning and self-compensation, often performing both functions simultaneously.

By utilizing these elements, Synaptics has implemented subsystems in silicon that perform a variety of useful functions, such as: classifiers and convolvers; silicon retinas with center-surround, motion detection and orientation computation; a variety of neural networks; object imaging and classification on the same chip; and touch sensing.

The company has acquired and developed effective tools to simulate, design and test neural nets architectures and adaptive analog VLSI chips.

GOVERNMENT DIRECTIONS AS LEADING INDICATORS

Synaptics is headed in a highly commercial direction, getting neural nets out in the walks of everyday life. But what is happening behind the scenes in government circles is perhaps a better indication of the powerful potential of these fledgling devices. To follow are brief highlights of the latest and most promising neural network

applications, as paraded before a discerning audience at a recent US Government Microcircuit Applications Conference:

l The utility of neural networks in hyperspectral imagery for pixel-level target detection and segmentation.

. Neural network technology for detection of ocean wakes in noisy synthetic aperture radar images which display the ability to detect wakes in images with signal-to-noise ratios which are beyond human detection capabilities.

l Artificial neural-network methods to improve moving-target detection and tracking by fusion of multiple sensor outputs over time and space where the neural networks use a recurrent detector predictor architecture with a postprocessor for hypotheses verification.

. “Neural manufacturing” for plasma control which uses neural networks to directly interpret signals from the plasma spectrum to improve process monitoring and control.

l An approach to neural networks to transient signal identification, including system design, real-time computer simulation and hardware implementation.

A project which has improved a state-of-the-art computer speech-recognition system by embedding a neural network.

In only a few short years, the list has become long and strong, with potential applications limited only by the human mind’s imagination. In the months ahead and in subsequent issues of Biosensors and Bioelectronics, we’ll thoroughly investigate applications such as those above, and delve more deeply into the subject of neural nets and their obvious association with other efforts in bioelectronics research and development.

WHERE TO GET STARTED

Interested? Many people are. Since Synaptics publicized their first product, and respected

625


popular magazines such as Scientific American covered their story in June of 1992, the little company has been deluged with calls from people with a breadth of potential everyday applications for intuitive computation.

“Many of these are application we had never thought of, a situation reminiscent of the early days of the microprocessor when some thought that its applications would be limited mainly to electronic calculators and data terminals,” states Steve Bisset, Senior Vice President of Synaptics. As the technology for intuitive computation evolves, it too promises to become only “imagination limited” in its applications. “The response strengthens our conviction that, in time, intuitive computation will come to be seen not only as a new wave of technology, but as a new way of thinking about problem solving,” said Mr. Bisset.

The company has moved to larger premises since their product introduction and can be contacted at: Synaptics Incorporated, 2694 Orchard Parkway, San Jose, CA 95134, USA; Tel/Fax: 408/434-0110/9819. But unless you’re a bonafide potential applications partner, or already quite knowledgeable in the field of neural network technology, perhaps it might be better to do a bit of homework before knocking on their busy door.

A highly recommended starting point is a book written by R. Colin Johnson and Chappell Brown titled “COGNIZERS, Neural Networks and Machines that Think” published by Wiley Science Editions of John Wiley & Sons, New York, Chichester, Brisbane, Toronto and Singapore. The book provides a very readable history of the fleeting evolution and origins of neural nets, and provides a baseline for thinking that one doesn’t often find in primers. The authors are both trade press reporters such as myself, and I’ve worked with both gentlemen on the staff of Electronic Engineering Times and on a number of advanced technology pursuits and quests over the years. I highly recommend their book for starters.

Their chapter, “Biotechnology Unleashed” will be of special interest to our B&B readership, as will the Epilogue (in case you can’t find the

time to read the entire book). As they underscore, research in the field is not confined to startups like Synaptics. “Industrial giants like AT&T and TRW are also making big investments in large-scale systems that perform tasks for which conventional computers are proving inadequate.” Johnson and Brown also point to Synaptics’ efforts with admiration, but point out that in Japan, the Asahi Chemical Industry Company has taken a leading role by building a speech recognition system based on Helsinki University of Technology professor Teuvo Kohonen’s circuitry, and that Kohonen modeled his creations on the ear/brain system much like Professor Carver Mead did the eye.

There are *also a number of exciting conferences in neural nets occurring with a growing attendance that seems to amaze even conference organizers. We hope to be reporting to you from some of those gatherings, as I myself will from the Government Microelectronics Applications Conference, mentioned above, and we’ll certainly profile other exciting companies such as Synaptics in the months ahead.

BIOSENSOR INPUT

Input from the biosensor community is sorely needed at this stage, because neural net researchers tend to migrate to the field from microelectronics. As Johnson and Brown point out in the closing pages of their book, “the amount of research still needed to understand the brain is large, and our areas of ignorance are vast. But the questions are well understood, research is ongoing, and the field is growing. It is only a matter of time before cognizers move from being an academic curiosity to being a part of everyday life.” Those words were written in 1988. Synaptics has since entered that mainstream with their spectacular achievement. What’s next? Only one’s gifted mind can foresee that future. Perhaps it will it be yours.

626

Documents

Neural nets are starting to make sense