Can Connectionism Model Developmental Change?

Blackwell Publishers Ltd. 1998, 108 Cowley Road, Oxford, OX4 1JF, UKand 350 Main Street, Malden, MA 02148, USA.Mind & Language, ISSN: 0268-1064Vol. 13. No. 4 December 1998, pp 576–581.

Can Connectionism Model DevelopmentalChange?*

MARGARET HARRIS

There can be no doubt that connectionism has come a long way in the last10 years. Perhaps what is most impressive is that connectionists are nolonger merely content to reproduce features of human performance byjudicious structuring of the training regime (as in the examples describedby Hadley, 1994). Rather, they are now seeking to show commonalities inthe structure of the internal representations of connectionist networks (c-nets) and those assumed to underpin human behaviour. For those who arestill not aware of connectionism, and the many philosophical and psycho-logical issues that it raises, this book provides an excellent starting point.There is an admirably clear account of the basic properties of c-nets and ofthe choices that they offer in terms of architecture and learning algorithms.There are also clear descriptions of a variety of recent c-nets including mod-els of vocabulary development and the acquisition of the English past tense.Equally importantly, the authors have also attempted to address some ofthe many concerns that have been raised about the status of c-nets, such astheir potential for neural instantiation. One issue that comes up repeatedly—and is reflected in the title of the book—is whether c-nets are compatiblewith innateness.

The authors of Rethinking Innateness are anxious to stress that ‘connec-tionist models are not inherently “anti-nativist” and they are certainly notthe natural enemy of those who believe in domain specificity’ (p. 41).Whether this is correct, I leave the reader to judge, but it soon becomes clearthat what makes connectionist models exciting for the authors is that theycan describe interactionist developmental change. Various lines of evidencein favour of interactionist account of development are discussed. Forexample, the authors point to findings from genetics showing that even

*Review of Jeffrey L. Elman, Elizabeth A. Bates, Mark H. Johnson, Annette Karmiloff-Smith,Domenico Parisi and Kim Plunkett, Rethinking Innateness: A Connectionist Perspective onDevelopment. Cambridge, MA: MIT Press, 1996. Pp. xviii 1 447.Address for correspondence: Margaret Harris, Department of Psychology, Royal Hol-loway, University of London, Egham Hill, Egham, Surrey TW20 0EX, UK.Email: uhjt020Kvax.rhbnc.ac.uk.

Can Connectionism Model Developmental Change? 577

apparently simple outcomes (such as eye colour in fruit flies) are determinedby cascades of interactions among genes rather than by a single gene. Thereason for their strong defence of an interactionist view becomes apparentupon discovering that much of the concern of this volume is with the devel-opment of language. Here the nature–nurture debate has continued, andarguments in favour of innately determined processes of development haveoften held sway, as have related views about modularity and domain speci-ficity.

Accounting for developmental change, rather than merely describing it, isthe major task for developmental psychology and the adequacy of anytheory of child development must be judged in terms of the adequacy of itsaccount of change. The most influential theory of child development withinpsychology, that of Jean Piaget, is a theory of changes in the mechanismsunderlying cognition from birth to adolescence. Following on from the earl-ier ideas of James Mark Baldwin, Piaget argued for a self-regulating interac-tion between the child and her physical and social environment. Accordingto Piaget it is this interaction which gives rise to new forms of knowledge.Although Piaget’s view of development has been rightly criticized, both forits lack of attention to the important social-interactional context of learningand for its over-emphasis on motor schemata in early development, fewdevelopmental psychologists would doubt that his original vision of devel-opment as an interaction between genetically determined maturational abili-ties and experience is essentially correct. What I want to consider here iswhether, as the authors claim, c-nets ‘can inspire a truly interactive theoryabout developmental changes’ (p. 47).

It is worth attempting to clarify Piaget’s own view of the nature of devel-opment to see how well c-nets can capture the spirit of his theory. Piagetwas much influenced by the evolutionary biologist C. H. Waddington, fromwhom he adopted the concept of the ‘epigenetic landscape’ as a metaphorfor developmental change. Since this metaphor provides a particularly vividpicture of the complexities of the developmental process, which should becaptured by any satisfactory model, it is worth taking a few moments toappreciate some of the key features of the epigenetic landscape so that wecan ask whether connectionist models can adequately reflect its complexity.It is particularly appropriate to use this model as a point of comparison,since it is mentioned at the beginning of Rethinking Innateness.

The epigenetic landscape contains hills and valleys of differing heightsand depths which represent possible pathways for development. The childis seen as a ball rolling across the landscape. The hills and valleys imposeconstraints on the pathways that can be taken by the ball. For example,should some environmental event knock the ball off course, it is more diffi-cult for it to leave a deeper valley than a shallower one. The presence of thehills and valleys in the epigenetic landscape attempts to capture the ideathat there are natural pathways which development might take which aredifferentially susceptible to influences from the environment. Junctionsbetween the valleys represent critical points in development, where further

Blackwell Publishers Ltd. 1998

578 Mind & Language

development may take one of several forms, depending on environmentalfactors that obtain at the time. One particularly useful feature of the modelis that it captures the idea that many aspects of child development show agreat deal of individual variation. Individuals may differ both in the speedwith which they reach a particular end state and in the route which theytake to arrive there, reflecting the fact that, for many developmental pro-cesses, both rates and characteristics of development vary from child tochild.

As a developmental psychologist, one of my main concerns about c-netsis whether they are able to capture the complexities of developmental changethat are reflected in the epigenetic landscape metaphor. There are three maincriteria against which we can assess their adequacy. First, a c-net shouldaccount for developmental stages; that is, qualitative and often rapid changesin the way that a child solves a particular task such as searching for a hiddenobject, understanding theory of mind scenarios or acquiring new vocabulary.Second, where there are individual differences in rate and course of develop-ment, these should also be captured by the model. For example, many stud-ies of early vocabulary development have highlighted two distinct styles ofdevelopment which are almost certainly two ends of a continuum. Essen-tially these two styles are characterized by the extent to which children’searly comprehension and production vocabulary is dominated by referentialitems, that is, object names. A satisfactory model of early lexical develop-ment must account for the variation in the composition of early vocabularyand also for the more general finding that rates of development are veryvariable, with production vocabularies at 18 months ranging from no wordsto over 300. One final, overarching, concern is the extent to which the modelof learning that emerges from c-nets reflects the processes actually involvedin development since, in theory, a connectionist model could capture bothstage transitions and individual differences in rates and course of develop-ment but still have no psychological reality.

One of the main claims about the strength of connectionist models madeby Elman et al. is that they are able to show how the same learning mech-anism can give rise to different kinds of performance at different times. Thisis a minimal requirement for a model of developmental change. What ismore important is that the model should be able to account for stage tran-sitions in development; that is, to show not just a steady increment in per-formance across time but substantial and rapid changes. There is evidencethat c-nets can exhibit stage transitions. For example, the model of vocabu-lary acquisition described in Rethinking Innateness, in which an auto-associat-ive network associates random dot images with labels, demonstrates thetypical ‘spurt’ in rate of learning new vocabulary that has been found inmany studies of early language development. For the child, production ofwords initially increases slowly but, once 30 or so words have been acquired,there is a marked increase in the rate at which new words are acquired. Inthe c-net, success at matching labels to images remains low for the first 20to 30 epochs of training, but subsequently there is a dramatic increase in the



success rate. The model also exhibits other well established features of earlyvocabulary learning; namely, prototype effects (in which performance withuntrained prototypical images is better than performance with trained pat-terns which are distortions of the prototype) and under- and over-extensionerrors (in which object names are applied either in a restricted way only to asubset of possible referents or over-generalized inappropriately; for example,using the word ‘dog’ to refer to other small animals).

The fact that all these three aspects of vocabulary learning are emergentproperties of the model is certainly impressive. However, I am left with twoworries. My main worry is that the emergent properties arise because thenetwork is being presented with a large number of image-label pairings ineach training epoch. However, it seems unlikely that such a process of simul-taneous learning of many associations occurs in the case of the child. Rather,the evidence suggests that learning of vocabulary is initially very focused,with the child concentrating on a small number of words which are of parti-cular salience in terms of the child’s own activity. My second worry is thatthe model of vocabulary learning does not account for individual differences.What it gives is a picture of what could be seen as ‘averaged’ learning whichreflects the actual learning of very few children.

Returning, now, to the issue of stage transitions, it is worth noting thatnot all developmental psychologists agree that there are developmentalstages. For example, proponents of dynamic systems theory such as vanGeert (1991) and Thelen and Smith (1994) have argued against the view thatthere are fundamental changes in the nature of children’s representation ofthe kind proposed by Piaget. They have demonstrated that what appearto be striking examples of non-linear changes in development (such as theincreasing number of words that a child can produce over the first two yearsof life) can be explained as linear functions. In other words, marked changesin behaviour which may appear to indicate the child’s transition from onestage of development to another can be explained in terms of a single processoperating over the entire course of development. C-nets offer the same possi-bility. They can also show marked changes in performance over time, as inthe example of image-label mapping that we considered earlier. It seems tome that this is one of the main strengths of c-nets. By showing that markedchanges in behaviour over time can be explained as emergent properties ofa system that is functioning in the same way right from the start of learning,developmental psychologists are forced to consider that stage transitionsmay not arise from changes in representational capacity of the kind proposedby Piaget.

A similar challenge to conventional theorizing comes from the capacity ofc-nets to exhibit characteristics that have traditionally been attributed to theexistence of different strategies in human cognition. One well knownexample is the dual-route model of reading, which proposes that, for anirregular orthography like English, there are two distinct routes for pro-nouncing words. One is a direct route, in which a word is identified fromthe internal lexicon and then its pronunciation retrieved. The second route


580 Mind & Language

is indirect and involves the conversion of letters or letter strings to sounds.Many features of word recognition appear to support a dual-route model,most notably the finding that, in the case of irregular words (which areassumed to be read using the direct route), people take longer to read low-frequency words than high-frequency, whereas, for regular words (whichcan be read by the indirect route) the effect of frequency is less marked.There is also evidence from cognitive neuropsychology of a double dis-sociation between these two routes, in that many patients have now beenidentified who can read only using one of the routes but not the other. Sev-eral connectionist models of reading, including that of Plaut, McClelland,Seidenberg and Patterson (1994) which is described in Rethinking Innateness,are successfully able to mimic aspects of reading that have been taken to befirm evidence for the dual-route model and the existence of distinct readingstrategies. As in the case of stage transitions, the possibility that there canbe a more parsimonious explanation for a range of behavioural phenomenaprovides an important theoretical challenge.

Another strength of c-nets is that they can sometimes resolve uncertaintiesin crucial aspects of cognitive psychological models. One area where theinterplay between models derived from empirical research and c-nets hasbeen particularly fruitful is in the area of memory which receives little atten-tion in Rethinking Innateness. Bray, Reilly, Villa and Grupe (1997) describe aresearch programme which has attempted to model the development ofmemory strategies in typically and atypically developing children, using c-nets. They highlight some of the special challenges that arise in connectionistmodelling of developmental phenomena. These include the problems of realversus simulated time and the representation of different levels of learning—the idea that all aspects of learning change with age. Bray et al. discuss waysin which the amount of learning across time can be modified so that, at oneextreme, information learned may not be modified and, at the other, modi-fied within trials.

Hitch, Burgess, Towse and Culpin (1996) have also used c-net modelling tofine-tune their model of working memory. They considered the well known‘temporal grouping effect’, in which introducing pauses into the presentationof a list of seen or heard letters improves immediate recall. In an earlierconnectionist model of working memory, Burgess and Hitch (1992) proposedthat the phonological loop has three sub-components. The first two—thephonological store and subvocal rehearsal—were features of the originalmodel of working memory (Baddeley and Hitch, 1974). The third—a repeat-ing context signal that reflects the rhythm with which items are presented—was introduced as a result of the implementation of the working memorymodel in a c-net. Hitch et al. (1996) propose a further addition; namely, acentral timing mechanism of temporal oscillators that generates the contextsignal which becomes entrained to the inter-group frequency of presenteditems, resetting after every pause. Their work shows how c-net modellingcan be used to extend and enrich models that are derived from experimentaldata. Although this is not a developmental model, it seems entirely possible



that c-nets could also be used to extend and refine existing developmentalmodels in a similar way.

I began by arguing that if connectionism is to account for developmentalchange, then c-nets must be able to exhibit stage-like transitions, to reflectthe range of individual variation in development and, most importantly, tohave psychological reality. Rethinking Innateness left me convinced that c-nets have important things to say about the nature of developmental stageswhich challenge conventional theorizing. However, I remain less convincedabout their ability to reflect individual differences and most concerned aboutthe psychological plausibility of the model of development that they offer.However, this latter issue is beginning to be addressed in connectionist mod-elling and future attempts to constrain c-nets in ways which make theirlearning more childlike will undoubtedly provide developmental psycho-logists with more food for thought.

Department of PsychologyRoyal Holloway University of London

References

Baddeley, A. D. and Hitch, G. J. 1974: Working Memory. In G. Bower (ed.), ThePsychology of Learning and Instruction, vol. 8. New York: Academic Press.

Bray, N. W., Reilly, K. D., Villa, M.F. and Grupe, L. A. 1997: Neural NetworkModels and Mechanisms of Strategy Development. Developmental Review, 17,525–66.

Burgess, N. and Hitch, G. J. 1992: Towards a Network Model of the ArticulatoryLoop. Journal of Memory and Language, 31, 429–60.

Hadley, R.F. 1994: Systematicity in Language Learning. Mind and Language, 9,247–72.

Hitch, G. J., Burgess, N., Towse, J. N. and Culpin, V. 1996: Temporal GroupingEffects in Immediate Recall: A Working Memory Analysis. Quarterly Journalof Experimental Psychology, 49A, 116–39.

Thelen, E. and Smith, L. B. 1994: A Dynamic Systems Approach to the Developmentof Cognition and Action. Cambridge, MA: MIT Press.

van Geert, P. 1991: A Dynamic Systems Model of Cognitive and LanguageGrowth. Psychological Review, 98, 3–53.


Documents

Can Connectionism Model Developmental Change?