Journal of Pre-College Engineering Education Research (J-PEER) Journal of Pre-College Engineering Education Research (J-PEER)

Volume 5 Issue 2 Article 2

2015

Effect of an Engineering Camp on Studentsrsquo Perceptions of Effect of an Engineering Camp on Studentsrsquo Perceptions of

Engineering and Technology Engineering and Technology

Rebekah Hammack Oklahoma State University bhammackstillwaterschoolscom

Toni A Ivey Oklahoma State University toniiveyokstateedu

Juliana Utley Oklahoma State University julianautleyokstateedu

See next page for additional authors

Follow this and additional works at httpsdocslibpurdueedujpeer

Part of the Engineering Education Commons and the Science and Mathematics Education Commons

Recommended Citation Recommended Citation Hammack R Ivey T A Utley J amp High K A (2015) Effect of an Engineering Camp on Studentsrsquo Perceptions of Engineering and Technology Journal of Pre-College Engineering Education Research (J-PEER) 5(2) Article 2 httpsdoiorg1077712157-92881102

This document has been made available through Purdue e-Pubs a service of the Purdue University Libraries Please contact epubspurdueedu for additional information

This is an Open Access journal This means that it uses a funding model that does not charge readers or their institutions for access Readers may freely read download copy distribute print search or link to the full texts of articles This journal is covered under the CC BY-NC-ND license

Effect of an Engineering Camp on Studentsrsquo Perceptions of Engineering and Effect of an Engineering Camp on Studentsrsquo Perceptions of Engineering and Technology Technology

Abstract Abstract Studentsrsquo knowledge about a profession influences their future decisions about careers Research indicates that students tend to hold stereotypical views of engineers which would hinder engineering as a career choice The purpose of this study was to measure how participating in a week long engineering summer camp affected middle school studentsrsquo (N519) attitudes towards engineering and their conceptions of engineering and technology Results indicate that participation in the programs had a positive impact on the studentsrsquo understandings of what technology is and the work engineers do Although the results indicate a positive impact on participants it is not clear which components of the camp contributed to this change The partnership between practicing middle school teachers and engineering faculty was important to the success of the camp revealing the benefits of collaborative efforts between K-12 educators and engineering professionals

Keywords Keywords K-12 engineering education middle school conceptions outreach attitudes

Document Type Document Type Article

Authors Authors Rebekah Hammack Toni A Ivey Juliana Utley and Karen A High

This article is available in Journal of Pre-College Engineering Education Research (J-PEER) httpsdocslibpurdueedujpeervol5iss22

Available online at httpdocslibpurdueedujpeer

Journal of Pre-College Engineering Education Research 52 (2015) 10ndash21

Effect of an Engineering Camp on Studentsrsquo Perceptions of Engineeringand Technology

Rebekah Hammack1 Toni A Ivey1 Juliana Utley1 and Karen A High2

1Oklahoma State University2Clemson University

Abstract

Studentsrsquo knowledge about a profession influences their future decisions about careers Research indicates that students tend to holdstereotypical views of engineers which would hinder engineering as a career choice The purpose of this study was to measure howparticipating in a week long engineering summer camp affected middle school studentsrsquo (N519) attitudes towards engineering and theirconceptions of engineering and technology Results indicate that participation in the programs had a positive impact on the studentsrsquounderstandings of what technology is and the work engineers do Although the results indicate a positive impact on participants it is notclear which components of the camp contributed to this change The partnership between practicing middle school teachers andengineering faculty was important to the success of the camp revealing the benefits of collaborative efforts between K-12 educators andengineering professionals

Keywords K-12 engineering education middle school conceptions outreach attitudes

Introduction

According to the US Bureau of Labor Statistics (2014) occupations in Science Technology Engineering andMathematics (STEM) are projected to grow 13 between 2012 and 2022 whereas non-STEM occupations are onlyprojected to grow 11 STEM degree holders typically have higher salaries and lower unemployment rates than non-STEMdegree holders Furthermore STEM workers play a vital role in promoting economic growth and stability of the USeconomy (US Department of Commerce 2011) Owing to its key role in product development and innovation engineeringis important for the continued improvement of humanity This coupled with a large wave of future engineering retireesmakes the recruitment of diverse highly qualified high school graduates extremely important (National Academy ofEngineering amp National Research Council 2009)

It has been over 30 years since the National Science Foundation released the report Educating Americans for the Twenty-First Century Report of the National Science Board Commission on Precollege Education in Mathematics Science andTechnology (National Science Foundation 1983) that sparked a movement calling for all Americans to become scienceliterate This report clearly documented that the lsquolsquoErsquorsquo was the only one of the four STEM disciplines not represented in the

Correspondence concerning this article should be sent to Rebekah Hammack at bhammackstillwaterschoolscom

1httpdxdoiorg1077712157-92881102

assessment of the quality of precollege STEM educa-tion In fact engineering is the only STEM discipline thatdoes not have its own set of national content standardsEfforts have been made however to integrate engineeringwithin technology and science standards (Achieve Inc2013 International Technology Education Association2000)

Research has suggested that one of the benefits of PK-12engineering education is the improvement of studentsrsquoability to apply mathematics and science skills (Committeeon K-12 Engineering Education 2009) Until recentlyteachers in the United States have focused on mathematicsand science coursework as the way to encourage studentsto enter the engineering pipeline (National Academy ofEngineering amp National Research Council 2009) Howeverthis focus on mathematics and science has not resulted inenough students who enter post-secondary engineeringprograms to meet projected job demands According toSocial Cognitive Career Theory individuals choose careerpaths based in part on their interests attitudes and values(Lent Brown amp Hackett 1994) Further interest in STEMoccupations has been linked to studentsrsquo knowledge ofSTEM careers (Robinson amp Kenny 2003) The lack ofstudents entering engineering programs may be attributableto a lack of understanding of what type of work engineersperform

There is a plethora of research related to studentsrsquo attitudestowards science (ie Jones Howe amp Rua 2000 OsborneSimon amp Collins 2003) mathematics (for example WeinbergBasile amp Albright 2011 Zambo amp Zambo 2006) and theimpact of outreach programs on those attitudes (ie Oliver ampVenville 2011 Stake amp Mares 2001) However the literaturepertaining to studentsrsquo attitudes towards engineering is some-what limited Much of the work pertaining to perceptions ofengineers has focused on elementary studentsrsquo participationin the Engineering is Elementary curriculum developed bythe Boston Museum of Science (Cunningham Lachapelle ampLindgren-Streicher 2005) The existing dearth of researchfocused on middle school studentsrsquo attitudes towardsengineering and technology makes the current study animportant addition to the field This study attempts toaddress a void in the literature by examining the impactsof a summer engineering outreach program on middleschools studentsrsquo attitudes toward conceptions of engi-neering and technology

Purpose and Rationale

There is an emerging yet limited body of work related tothe impact of summer outreach programs on studentsrsquoperceptions of engineering Studies focused on attitudestowards engineering can not only identify student perceptionsof engineering but also lead to a better understanding of thereasons why those perceptions exist Identifying reasonsbehind student perceptions could help lead to interventions to

improve student attitudes towards engineering and eventuallyresult in more high school graduates pursuing engineeringdegrees

The goal of the summer camp was to engage middle schoolstudents in hands-on age-appropriate activities that would

N allow students to differentiate between the activities ofscientists and engineers

N introduce students to engineering as a possible careerpath with a focus on the field of chemical engineering

N engage students in collaborative problem-based learn-ing assignments that integrate mathematics scienceand engineering and

N provide examples of how engineers engage inengineering design to solve problems

The purpose of this study was to determine howparticipating in an engineering summer camp affectedmiddle school studentsrsquo attitudes towards engineering andtheir conceptions of engineering and technology Ourprimary research question addressed in this study was Howdoes participating in an engineering summer camp impactmiddle school studentsrsquo attitudes towards and conceptionsof engineering and technology

Background Literature

K-12 Student Perceptions of Scientists and Engineers

Draw-a-Scientist (DAS) (Mason Kahle amp Gardner1991) and more recently Draw-an-Engineer (DAE)(Knight amp Cunningham 2004) instruments are commonlyused to identify studentsrsquo perceptions of scientists andengineers respectively DAS and DAE require students todraw a scientist or engineer at work and often ask thestudents to describe what they have drawn Becausestudents often confuse engineers with other occupationssuch as scientists (Karatas Micklos amp Bodner 2011) it isexpected that parallels can be drawn between the two

Analysis of drawings from DAS highlight variousstereotypes associated with science For example in astudy of 1600 middle school students Fralick KearnThompson and Lyons (2009) reported that 33 of thestudents drew female scientists Likewise when asked todraw a picture of a scientist Cavallo (2007) reported onlyfive out of 150 fourth through sixth grade students depictedfemale scientists in their drawings Additionally researchfindings indicate that many individuals hold stereotypicalviews of scientists as individuals who lead solitary asunappealing lifestyles Cavallo (2007) administered theDAS to 150 middle school students whose most commonimages were of lonely white men who often had crazy hairor monster-like features In a survey of high schoolstudents Miller Blessing and Schwartz (2006) found thatparticipants viewed scientists as uncaring and passionlessloners Furthermore the girls in their study tended to

R Hammack et al Journal of Pre-College Engineering Education Research 11

2httpdxdoiorg1077712157-92881102

perceive scientists as unsociable individuals who do notbenefit society (Miller et al 2006)

Studies implementing DAE have reported similar findingsas those using DAS For example Fralick et al (2009)reported that only 13 of the 1600 participating studentsdrew female engineers Similarly in a study of 20 sixth-grade students conducted by Karatas et al (2011) only onestudent drew a female when asked to draw an engineer atwork Knight and Cunningham (2004) reported higherpercentages of DAE images that portrayed females in theirdrawings (39) however most of these drawings came froma classroom where two female engineering college studentsworked with the children Likewise students participatingin an after school engineering program for girls weremore likely to depict female engineers in their drawings(Hammack and High 2014) however it is important to notethat all of the engineers involved in the after school programwere female and may have served as the basis for theparticipantsrsquo drawings Moreover DAE images suggest thatstudents tend to view the job duties of engineers as fixingmachines building things and driving trains (Fralick et al2009 Cunningham et al 2005 Knight amp Cunningham2004) It is reasonable to believe that these stereotypes stemfrom a lack of knowledge about scientists engineers and thenature of engineering

Influences on Career Choice

Studentsrsquo prior knowledge about a profession influencestheir decision to pursue that profession For exampleWyss Heulskamp and Siebert (2012) reported that view-ing recorded video interviews with STEM professionalshad a positive impact on middle school studentsrsquo interestsin STEM careers Additionally students begin makingdecisions about careers as early as middle school soproviding information about STEM careers prior to andduring the middle school years is important (Wyss et al2012) Thus elementary- and middle-level students needmore exposure to STEM initiatives which can only besuccessful if their teachers know how to implement STEMmaterials into the curriculum (DeJarnette 2012) In a recentstudy high school and college students described that themost important factor influencing their career choice was apersonal interest in the area which was primarily due to theSTEM knowledge of teachers and school counselors andparental influence about the career (Hall Dickerson BattsKauffmann amp Bosse 2011) Additionally Maltese and Tai(2010) reported that nearly 40 of science professionalsand graduate students reported school-related factorssuch as teachers as the reason for their science interestsBecause teachers play an important role in shapingadolescentsrsquo views of STEM it is important that teachersare also knowledgeable of STEM careers

Although adolescents report the influence of parentsteachers and counselors on their career choices these

individuals often lack knowledge of STEM careers that isneeded to properly advise students In a study conducted byHall et al (2011) parents rated their knowledge of STEMareas lower than other career fields whereas 613 of highschool STEM teachers and counselors did not feel knowl-edgeable of engineering careers This points to the limitedunderstanding some high school counselors and teachersmay hold about science

Programs to Improve Attitudes Toward STEM

There are numerous studies pointing to the benefits of scienceoutreach programs on studentsrsquo attitudes towards science (forexample Cavanaugh 2007 Metin amp Leblebicioglu 2011Rahm Moore amp Martel-Reny 2005 Robbins amp Schoenfisch2005) and mathematics (for example Weinberg Basile ampAlbright 2011) Engineering andor technology focusedcamps have been found to have positive impacts on studentsrsquoviews of engineering (Elam Donham amp Soloman 2012Nadelson and Callahan 2011) and technology (NugentBarker Grandgenett amp Adamchuk 2010) Additionallyoutreach programs that incorporate engineering into the schoolcurriculum have had positive impacts on studentsrsquo views ofengineering (Hirsch Carpinelli Kimmel Rockland amp Bloom2007 Plant Baylor Doerr amp Rosenberg-Kima 2009) Plantet al (2009) investigated the impact of using computeranimated interface agents on 106 middle schools studentsrsquomath performance and attitudes towards the utility of math andhard sciences and found that interacting with a female agentresulted in fewer gender-related perceptions of engineering inmale students but female studentsrsquo perceptions remainedunchanged Hirsch et al (2007) reported that middle schoolstudents who were exposed to engineering within their schoolmath and science curriculum had more positive attitudestowards engineering and a greater knowledge of engineeringcareers than students who were not exposed to engineeringconcepts within school curriculum Unlike Hirsch et al(2007) Mooney and Laubach (2002) reported that partici-pating in a one to three-week-long inquiry-based engi-neering curriculum units within science and mathcurriculum had limited impacts on studentsrsquo attitudestowards engineering with attitudes declining for somefemale subgroups However Mooney and Laubach (2002)reported improved attitudes towards mathematics andscience after participating in the engineering curriculumwith some male students showing improved attitudestowards engineering

Although the previously mentioned studies point to thebenefits that some engineering outreach programs have formiddle school students the diverse results and limitednumber of studies point to a need to further research on theimpacts of middle school engineering outreach programsSimilarly a deficit still exists in the research literature onstudies devoted solely to the impact of middle schoolengineering camps particularly camps devoted to the field

12 R Hammack et al Journal of Pre-College Engineering Education Research

3httpdxdoiorg1077712157-92881102

of chemical engineering As such our understanding of theimpact of chemical engineering outreach programs onstudentsrsquo attitudes toward engineering is limited

Pedagogical Content Knowledge and Co-Teaching

Effective instruction requires both subject matter knowl-edge and pedagogical content knowledge (Shulman 1986)Pedagogical content knowledge can be defined as lsquolsquothe waysof representing and formulating the subject that make itcomprehensible to othersrsquorsquo (Shulman 1986 p 9) Teacherswith increased subject matter knowledge and pedagogicalcontent knowledge are better able to determine their studentsrsquounderstanding of a topic and make changes to instruction thatimprove student learning (Bischoff 2006)

Co-teaching can be used to allow individuals with expertisein different areas to work together to teach a course This iscommonly seen in K-12 classrooms between a generaleducation and special education teacher There are manyproposed benefits of co-teaching (for example smaller student-teacher ratio more feedback to students more individual orsmall group instruction) however little research exists on theeffectiveness of co-teaching at the K-12 level (Sweigart ampLandrum 2015) and there is an absence of literature related touniversity and K-12 teachers engaging in co-teaching

Methodology

This study utilized a repeated measures design in whichsurvey and content tests were given before and afterparticipating in a summer engineering camp The quantitativesurvey data were used to test the hypothesis that participationin the camp positively impacted studentsrsquo attitudes towardsmathematics science and engineering Additionally open-ended questions were used to further explore studentsrsquo under-standings of engineering and technology Both quantitativeand qualitative data were used to compare results and bringgreater insight into the problem than by using a single method

Setting and Participants

Participants included 19 students (female54 male515white518 multi-racial51 special needs51) who had justcompleted sixth grade at the middle school where the campwas held The middle school was located in a Midwesterntown of approximately 45000 people and served approxi-mately 750 sixth- and seventh-grade students at the time ofthis study The camp was open to all students enrolled at themiddle school There was a small fee to attend the camp butscholarships were available to those in need to ensure thatsocioeconomic status did not prevent participation

Intervention

The summer engineering camp met for 35 hours eachday for four days Two middle school science teachers and

a chemical engineering professor from a local universityfacilitated the camp The middle school science teachersboth held at least a bachelorrsquos degree in a science contentarea The chemical engineering professor selected theactivities and assisted the teachers in the implementationEach of the activities required students to implementengineering design and introduced students to the types ofactivities engineers perform There was a large emphasisplaced on chemical engineering owing to the experiences ofthe professor in that area The students completed thefollowing activities during the camp

N Day 1 Students completed preassessments and thenparticipated in a brief measurement practice activity toensure that they could use graduated cylinders formeasuring volume and digital scales for measuringmass Next the chemical engineer lead a discussionon the similarities and differences between scientistsand engineers Students then completed a candyairplane activity in which they built a model of anairplane using only the provided supplies (forexample candy toothpicks paperclips rubber bands)within a limited amount of time After building themodel the students mass-produced the design Thechemical engineer led a discussion over product andprocess design using the information in Figure 1(Cheville amp Bunting 2011) The importance of processdesign in chemical engineering was discussed One ofthe major features of process design that was discussedwas its use with the mass production of products

N Day 2 Students completed a popcorn challenge inwhich they built a filter for a movie theater that couldseparate popped and unpopped kernels and allow theunpopped kernels to be recycled The chemicalengineer led a discussion about chemical processesand recycling using the information in Figure 2 Therewas a discussion of how the popcorn flowchartcorresponds with a chemical flowchart that would beused by a chemical engineer After completing thepopcorn challenge students were introduced to a slimeactivity During this activity students made slimeobserved its characteristics and then created a newproduct that utilized the slime Students also created anadvertisement for their product This activity was usedto demonstrate the differences between scientists(chemists) and engineers (chemical engineers) On thisday students made a single batch of slime much as achemist would do when identifying the formulation ofa new chemical substance

N Day 3 Students finished the slime activity by workingas chemical engineers to design package and marketa product that made use of the slime We alsodiscussed the role that chemical engineers play in themass production of chemicals that are required to beused in certain products such as pharmaceuticals andpetrochemical

R Hammack et al Journal of Pre-College Engineering Education Research 13

4httpdxdoiorg1077712157-92881102

N Day 4 Students experimented with materials todetermine which reactants would make the bestpropellants for film canister rockets Students designeda rocket body to attach to their film canisters and testedtheir chosen propellants A chemical engineerrsquos role inthe development and testing of rocket propellants wasdiscussed as well as the importance of mass productionof propellants

Prior to each activity students were presented with aproblem scenario Then they had to complete a designchallenge to address the problem For example prior tothe candy airplane activity students were told that anaircraft manufacturer was planning to expand its opera-tions and was looking for a new airplane model and manu-facturing facility Students then designed a model aircraftto present to the manufacturer and then mass-producedtheir design Each scenario provided students with a realworld context in which engineers would work Uponcompletion of each activity facilitators discussed howstudents engaged in engineering design throughout theactivities they created and reiterated real world applica-tions of the concepts

Instruments

Researchers used a variety of instruments in order tomeasure the impact of the program on the participantsrsquoattitudes toward mathematics science and engineeringconceptions of technology and conceptions of engineeringThese measures are described in detail below

Attitudes to Mathematics Science and Engineering(AMSE) The AMSE is a 5-point Likert-scaled instrumentconsisting of 36 items that are categorized into 8 subscalesTable 1 provides a list and description of each subscaleincluding the range of scores Scores on the overall AMSEcan range from 36 to 180 with higher scores being indicativeof a more positive attitude toward mathematics science andengineering (Hirsch et al 2007) Illustrative items includelsquolsquoEngineers are just people who do a lot of math and sciencersquorsquoand lsquolsquoEngineers help make peoplersquos lives betterrsquorsquo

What is Technology Cunningham and colleagues(2005) developed the What is Technology instrument toassess elementary studentsrsquo conceptions of technology Theinstrument consists of images and names of 16 items andone open-ended item How do you know if something istechnology Participants circle the images they associatewith technology and complete the open-ended responseExample items are shoes cellular phone and bird Scoresprovide data related to participantsrsquo ability to correctlyidentify examples of technology which can help research-ers identify misconceptions held by participants

What is Engineering Cunningham and colleagues(2005) also developed the What is Engineering Instrumentwhich consists of 16 images and descriptions of people atwork and one open ended response What is an engineer Forthis instrument participants circle the images with which theyassociate the work engineers perform as a part of their jobsand complete the open-ended response question Illustrativeitems include improve bandages drive machines and work as

Figure 1 Engineering design process used throughout the summer camp

Figure 2 Flowchart used to explain the process in the popcorn activity

14 R Hammack et al Journal of Pre-College Engineering Education Research

5httpdxdoiorg1077712157-92881102

a team Scores provide data related to participantsrsquo ability tocorrectly identify tasks completed by engineers as they workwhich can help researchers identify misconceptions heldabout engineers

Engineering Content Test On the first day of the campparticipants were given a precamp content test developedby the chemical engineering faculty member The surveycontained questions about content specific to chemicalengineering confidences in participating in engineeringdesign and the following journal prompts lsquolsquoWhat does achemical engineer dorsquorsquo and lsquolsquoHave you thought aboutbecoming an engineerrsquorsquo On the last day of camp participantscompleted a post-camp content test and responded to thefollowing prompts lsquolsquoWhat does a chemical engineer dorsquorsquoand lsquolsquoDid camp encourage you to consider engineeringrsquorsquo

Data Collection and Analysis

Camp facilitators pretested all participants at thebeginning of the first day of camp and post-tested on thelast day of camp All students completed the AMSE Whatis an Engineer and What is Technology instruments andEngineering Content test

Quantitative Data Analysis Owing to the smallsample size and lack of a normal distribution of dataresearchers used nonparametric statistics for data analy-sis A Wilcoxon Signed Rank test was performed inSPSS version 210 for the content test score overallAMSE score and all AMSE subscales except thelsquolsquootherrsquorsquo subscale Individualsrsquo subscale scores weredetermined by summing all of the item responses for aparticular subscale Table 1 presents the possible rangesfor an individualrsquos subscale scores On the presurveytwo participants left one item blank from the negativeopinions of mathematics and science subscale On thepost-survey two participants left one item blank onefrom the negative opinions of mathematics and sciencesubscale and one from the interest in engineeringstereotypic aspects subscale To account for eachparticipantrsquos missing value the mean for each indivi-dualrsquos subscales were imputed for the missing item

For example items 4 12 13 21 and 29 comprise thenegative opinions of mathematics and science subscaleParticipant A left item 21 blank on the pretest Hisscores from items 4 12 13 and 29 were averaged andthe resulting value was assigned to item 21

Each participantrsquos What is Technology and What isEngineering responses were scored based on the percen-tage of correct images that were circled Pre and post-testpercentages were entered into SPSS and a WilcoxonSigned Rank analysis was performed Researchers thenexamined the frequencies to identify trends

Each participantrsquos Engineering Content Test was given ascore based on the percentage of items that were correctlyanswered Pre- and post-test percentages were entered intoSPSS and a Wilcoxon Signed Rank analysis was performed

Qualitative Data Analysis The open-ended responseitems on the What is Technology and What is Engineeringinstruments as well as the lsquolsquoWhat does a chemicalengineer dorsquorsquo question were printed onto cards andexamined independently by three researchers to developa coding scheme for the responses The researchersdiscussed the three coding schemes and consolidatedthem into a single coding scheme The cards wereindependently recoded using the consolidated codingscheme There was 95 agreement between the resultingcodes the three researchers discussed the differencesuntil 100 consensus was reached (Emerson Fretz ampShaw 1995)

Results

Attitudes to Mathematics Science and Engineering

Researchers calculated a Cronbachrsquos alpha (086) todetermine the internal consistency for the AMSE instru-ment this value was consistent with the value reported(085) by Hirsch et al (2007) Table 2 presents thedescriptive statistics and Wilcoxon Signed Rank testresults for the seven AMSE subscales Only the Engineersubscale showed a significant difference pre to post(Z 5 0003 p 001)

Table 1Attitudes Toward Mathematics Science and Engineering (AMSE) subscales descriptions and score ranges

Subscale

Subscale ranges

Description Min Max

Overall AMSE 36 180Interest in engineering stereotypic aspects (ISA) Studentsrsquo interest in engineering based on common stereotypes 7 21Interest in engineering non-stereotypic aspects (INSA) Studentsrsquo interest in engineering not based on common stereotypes 4 12Positive opinions of mathematics and science (PO) Studentsrsquo positive opinions towards math and science 5 15Negative opinions of mathematics and science (NO) Studentsrsquo negative opinions towards math and science 5 15Problem solving (PS) Studentsrsquo attitudes towards their problem solving abilities 4 12Technical skills (TS) Studentsrsquo attitudes towards their technical skills 4 12Engineer (E) Studentsrsquo understanding of the type of work engineers perform 5 15

R Hammack et al Journal of Pre-College Engineering Education Research 15

6httpdxdoiorg1077712157-92881102

What is Technology What is Engineering andEngineering Content

Table 3 presents the descriptive statistics and WilcoxonSigned Rank results for the What is Engineering What isTechnology and engineering content measures Pre- to post-test differences were significant for three measures (p 001)

Figure 3 illustrates the percentage of items selected ascorrect on the What is Technology pre and post measuresAn examination of these percentages indicated that manyparticipants associated technology with items that wereelectronic (100 selected TV and phones) or involved theuse of electricity (100 selected subways 95 selectedpower lines) and none of the participants associated techno-logy with living organisms (0 selected bird tree ordandelions) Fewer participants associated technology withnon-electrical manmade items however a greater percentageof participants selected items that may be associated withconstruction (58 selected bridge 63 selected house 84selected factory) than items not generally associated withconstruction (47 selected shoes cups and bandages 52selected books and bikes) On the What is Technology Post-test however participants clearly associated technology withitems that do not exist in nature (95 selected bandages100 selected shoes subway phone bridge TV cupfactory house power lines bicycle books)

Participantsrsquo responses to the What is Engineeringinstrument are displayed in Figure 4 An examination ofstudent responses indicates that seven of the nine itemsassociated with activities of an engineer (marked with anasterisk in Figure 4) increased at the conclusion of theengineering camp These items include verbs such as

improve develop design and create The other two items (ieDesign clean water figure how to track luggage) showed nochange in the number of responses however more than half ofthe students correctly selected these items Of the seven itemsnot associated with activities of an engineer fewer studentsselected that engineers construct buildings sell food and cleanteeth at the conclusion of camp There was an increase in thenumber of students who incorrectly selected driving machinesand repairing cars as acts of an engineer and over half stillchose that an engineer performs the act of installing wiringSurprisingly two participants held on to the misconceptionthat the work of an engineer includes arranging flowers

Open-Ended ResponsesThis section present results from the qualitative analysis

of an open-ended response from three data sources(1) What is an Engineer (2) What is Technology and(3) Engineering Content Test The percent occurrence ofemergent themes from each data source is presentedin Figures 5 6 and 7 On the What is Technology instrumentstudentsrsquo initial responses to lsquolsquoHow do you know something istechnologyrsquorsquo were categorized as electrical manmade orhelps people However at the end of the engineering camp allstudentsrsquo responses indicated an understanding that technologywas something that is manmade Additionally a large percen-tage of studentsrsquo initial responses to the prompt lsquolsquoWhat is anengineerrsquorsquo were categorized as someone who builds designsor improves things to help people On the post-test howeverthe percentage of responses falling within these categoriesdeclined greatly and a new theme (mass production) wasidentified in 68 of responses Studentsrsquo Engineering ContentTest responses indicated growth in their understanding of

Table 2Summary statistics for participantsrsquo (n519) scores on Attitudes Towards Mathematics Science and Engineering Scale (AMSE)

Subscale

Pretest Post-test

Z pMin Max Mdn Min Max Mdn

AMSE-Overall 81 139 111 78 140 114 20829 0407ISA 13 32 22 9 31 24 20156 0876INSA 5 18 10 4 19 10 20604 0546PO 5 23 17 8 23 16 21225 0220NO 5 13 9 5 21 8 20192 0848PS 6 20 12 6 16 13 20287 0774TS 10 20 14 9 20 13 0528 0528E 14 22 18 16 23 19 0003 0003

p 001

Table 3Summary statistics for participantrsquos scores on What is Engineering and What is Technology and Engineering Content instruments

Instrument n

Pretest Post-test

Z PMin Max Mdn Min Max Mdn

Engineering 19 375 9375 625 50 100 75 23285 0001Technology 19 4375 100 75 100 100 100 22818 0005Content 19 0 57 14 29 86 57 23456 0001

p 001

16 R Hammack et al Journal of Pre-College Engineering Education Research

7httpdxdoiorg1077712157-92881102

chemical engineers Examples of studentsrsquo open-endedresponses on the Engineering Content Test are presentedin Table 4 When asked what chemical engineers doresponses of lsquolsquoI donrsquot knowrsquorsquo decreased from 26 to 5by the end of the camp Additionally analysis ofstudentsrsquo open-ended responses indicated a shift in theirunderstanding from thinking that chemical engineersonly mix and use chemicals to engaging in more complexwork (ie experimenting) with chemicals

Discussion

Overall the results indicate that participants improved their(1) understanding of technology (2) chemical engineeringcontent knowledge and (3) attitudes towards engineering Ofthe seven AMSE subscales tested only the responses on theEngineer subscale changed significantly This indicates thatover the course of the camp participantsrsquo attitudes towardsengineering became more positive with more participants

reporting that they know what an engineer does These resultsare consistent with the open-ended response questionsResponses to the open-ended lsquolsquoWhat is an Engineerrsquorsquo prompton the What is Engineering post-test indicate that partici-pants gained an understanding of engineers as people whowork in teams and mass produce products

The scores on the What is Technology instrument weresignificantly higher on the post-test indicating that the campintervention had a positive impact on the participantsrsquo under-standings of technology As evident in the frequency counts andopen-ended responses many participants began camp with themisconception that technology only included items that requiredelectricity These results are consistent with the findings ofCunningham et al (2005) who found that students in grades1ndash5 tend to associate technology with objects requiringelectricity By the end of camp all participants reported thattechnology referred to any object that was manmade

Scores on the Engineering Content Test were significantlyhigher on the post-test indicating that the intervention had

0

20

40

60

80

100

S

elec

ted

Item

Pre

Post

Figure 3 Frequency of participantsrsquo responses for items on the What is Technology instrument Items that are considered technology

0

20

40

60

80

100

S

elec

ted

Item

Pre

Post

Figure 4 Frequency of participantsrsquo responses for items on the What is Engineering instrument Items associated with engineering

R Hammack et al Journal of Pre-College Engineering Education Research 17

8httpdxdoiorg1077712157-92881102

a positive impact on participantsrsquo content knowledge relatedto chemical engineering Although the median post-testscore was over 40 points higher than the pretest medianit was still only 57 which is below the 70 proficiencyscore used by the school where the camp was held Thissuggests that while participants increased their understandingof chemical engineering they still did not master all of thecontent covered in camp The short duration of camp maynot have provided enough time for students to fully masterthe new concepts

The range in frequencies on the What is Engineeringpretest point to participantsrsquo uncertainties about the activitiesof an engineer There was still a large range in the post-testresults indicating that participants still had some degree ofuncertainty about the work of engineers However there weresome positive changes worth noting Prior to participating incamp 68 of participants viewed engineers as people whowork as a team whereas all campers circled lsquolsquowork as ateamrsquorsquo on the post instrument All of the camp activitiesrequired that participants work as a team to solve problemswhich may have accounted for this change Recognizing that

engineers are not solitary loners but in fact work in teams totackle important real world problems could help combat thecommon negative stereotypes previously reported (Cavallo2007 Miller et al 2006) and may prevent more adole-scents form abandoning the STEM pipeline Additionallyparticipantsrsquo Engineering Content Test responses also indi-cated a positive shift in their attitudes towards engineering as acareer For instance only 26 of camp participants reportedthat they had considered becoming a chemical engineer priorto camp whereas 63 of participants reported that campencouraged them to consider chemical engineering as a career

Although the findings indicate an increase in the number ofstudent considering a career in engineering they still heldmany misconceptions about the work of engineers It isimportant to note that the engineering camp did not explicitlyaddress all misconceptions presented on the What isEngineering instrument However the camp did discuss thatengineers focus their work on the design and creative aspectsof construction but are not the actual builders This highlights atrend in our findings that the participantsrsquo views of engineeringshifted away from engineers as mere builders and a movetowards an understanding that engineers design solutions toproblems This trend is mirrored in the open-ended responsesas nearly all participants recognized that lsquolsquobuildingrsquorsquo was notan act of engineering by the end of the camp The only twolsquolsquobuildingrsquorsquo responses on the post-test were

lsquolsquoAn engineer is a person who builds a lot of one thingrsquorsquolsquolsquoSomeone who builds lots of stuffrsquorsquoIt is important to note that one major focus of the candy

airplane activity was the role engineers play in industry andmass production and the mass production of chemicals wasdiscussed during both the slime and film canister rocketactivities It is therefore not surprising that over one-half of theparticipants referred to mass production in their post-testresponses The two participants who wrote lsquolsquobuildrsquorsquo in their post

0102030405060708090

100

r

espo

ndin

g

Theme

Pre

Post

Figure 5 Frequency of themes for What is Engineering instrument

0

20

40

60

80

100

120

Electrical Man made Helps people

r

espo

ndin

g

Theme

Pre

Post

Figure 6 Frequency of themes for What is Technology instrument

18 R Hammack et al Journal of Pre-College Engineering Education Research

9httpdxdoiorg1077712157-92881102

responses may also have been referring to mass productionrather than simply building or constructing an object

Interestingly there are conflicting findings between thequantitative and qualitative responses on the What is

Engineering instrument Although the number of imagesstudents selected of engineers designing and improvingproducts increased students qualitative responses related todesigning and improving products decreased Because the

0

10

20

30

40

50

60

r

espo

ndin

g

Theme

Pre

Post

Figure 7 Frequency of themes for Engineering Content Test responses

Table 4Examples of participantsrsquo open-ended responses by theme

Theme Example Pre-program Responses Example Post-program Responses

How do you know something is technologyElectrical It creates energy and has circuit boards and stuff NAMan made Man invents technology eg bicycle subway Something that is technology is a man-made objectHelps people If it was invented by people to help other people NA

What is an engineerMass production NA An engineer creates multiple copies of a productBuilds Someone who builds something Someone who builds lots of stuffDesigns Somebody who designs and builds stuff An engineer designs some product and fins a way

to mass produce itImproves thingshelps others Someone who researches and develops innovations

to make this world a better and easier placeSomeone who improves inventions

Fixes things An engineer is someone who fixes or creates things An engineer is a person that fixes things andworks as a team

Team work NA An engineer is a person that fixes things andworks as a team

Uses tools I think it is someone who builds or designsthings and works with tools and differentkinds of machinery

NA

Uses math and science People who do math and science NA

What does a chemical engineer doI donrsquot know I donrsquot really know I canrsquot describe itMixesUses chemicals Mixes chemicals Someone who works with mixing and learning

about chemicalsManufacturingdesigns new

products or chemicalsWorks with chemistry to make new materials Helps manufacture chemicals

Experiments with chemicals A chemical engineer experiments with different chemicalsto see if they do something useful or harmful

Experiments with different chemicals to seewhat they do

Works with food NA Works with food

R Hammack et al Journal of Pre-College Engineering Education Research 19

10httpdxdoiorg1077712157-92881102

candy airplane slime and rocket activities all dealt with massproduction it is possible that participants concentrated on theaspect of mass production during their open-ended responsesHowever half of participantsrsquo journal prompts about the workof a chemical engineer referred to some aspect of designing orcreating a product on the post-test Additionally there wasa decrease in the qualitative responses related to lsquolsquohelpingpeoplersquorsquo on the What is Technology and What isEngineering instruments The decrease in balanced bythe increase in lsquolsquomanmadersquorsquo responses on the What isTechnology instrument and lsquolsquomass productionrsquorsquo responseson the What is Engineering instrument

Overall the findings indicate that the participants gainedan understanding that engineers are involved with the massproduction of products as well as their design andimprovement As such the decrease in studentsrsquo responsesin these categories is balanced by the increase in thenumber of responses that indicated engineers as massproducers of items Follow-up interviews with participantswould have been valuable as a means of providingadditional information as to how participantsrsquo perceptionsof engineers changed as a result of camp

Limitations

Although the results indicate a positive impact onparticipants it is not clear which components of the camp(hands-on activities interaction with a professional engi-neer combination of factors) contributed to this changeFollow-up interviews could have helped identify thesefactors Campers self-selected into the program and mayhave already had an above average interest in STEM Thiscoupled with the small sample size limits the general-izability of the findings Results were also limited by theinstruments used The What is Engineering instrumentuses a specific definition of engineering that is notuniversally accepted Additionally the validity of thecontent test has not been established and the test has onlybeen used with the 19 study participants Because we onlycollected and analyzed data immediately following thesummer camp our study only addresses short-termimpacts Future research should be conducted to determinelong-term impacts of engineering outreach programs

Conclusion and Implications

In answering our research questions we found thatparticipation in an engineering summer camp had a positiveimpact on middle school studentsrsquo understandings of whattechnology is and what engineers do evidenced by both thequantitative and qualitative data The open-ended questionssupported and added detail to the quantitative findingsproviding more evidence than if either data set had beenused independently Participantsrsquo attitudes towards mathe-matics and science were not significantly impacted by the

summer camp However their attitudes towards engineer-ing were significantly impacted The short duration of thecamp limited the quantity of information that could bepresented to participants with the current intervention focusingprimarily on engineering Although measurable impacts wereaccomplished during the week long summer camp long terminterventions such as the infusion of engineering activitieswithin the school curriculum could allow for attitudinalchanges across multiple STEM disciplines

The partnership between practicing teachers and engineer-ing faculty was important to the success of the camp Althoughthe extent to which co-teaching impacted outcomes was notmeasured the diverse backgrounds of the camp facilitatorsallowed for the combination of pedagogical knowledge andengineering content expertise to create rich engineeringexperiences that were developmentally appropriate for middleschool participants These experiences positively impactedparticipantsrsquo attitudes towards engineering revealing thebenefits of collaborative efforts between K-12 educators andengineering professionals Additional research in this area isneeded

References

Achieve Inc (2013) Next generation science standards WashingtonDC Achieve Inc

Bischoff P J (2006) The role of knowledge structures in the ability ofpreservice elementary teachers to diagnose a childrsquos understanding ofmolecular kinetics Science Education 90(5) 936ndash951 doi101002sce20155

Cavallo A (2007) Draw-a-ScientistMystery box Science and Children45(3) 37ndash41 Retrieved from httpwwwnstaorgpublicationsbrowse_journalsaspxaction5issueampid51025053sc07_045_03

Cavanaugh S (2007) Science camp Just for the girls Academic camps are onthe rise across the country including ones to get adolescent girls excitedabout the exploration of science Education Week 26(45) 26ndash28

Cheville A amp Bunting C (2011) Engineering students for the 21stcentury Student development through the curriculum Advances inEngineering Education 2(4) 1ndash36 Retrieved from httpadvancesaseeorgwp-contentuploadsvol02issue04papersaee-vol02-issue04-p10pdf

Committee on K-12 Engineering Education (2009) In L Katehi GPearson amp M Feder (Eds) Engineering in K-12 educationunderstanding the status and improving the prospects WashingtonDC The National Academies Press

Cunningham C Lachapelle C amp Lindgren-Streicher A (2005) Assessingelementary schools studentsrsquo conceptions of engineering and technologyProceedings of the 2005 American Society for Engineering EducationAnnual Conference amp Exposition Portland OR

DeJarnette N K (2012) Americarsquos children Providing early exposure toSTEM (science technology engineering and math) initiativesEducation 133(1) 77ndash84 Retrieved from httpwwwprojectinnovationbizeducation_2006html

Elam M E Donham B L amp Soloman S R (2012) An engineeringsummer program for underrepresented students from rural schooldistricts Journal of STEM Education 13(2) 35ndash44 Retrieved fromhttpojsjstemorgindexphpjournal5JSTEMamppage5articleampop5viewamppath5B5D51619amppath5B5D51437

Emerson R M Fretz R I amp Shaw L L (1995) Writing EthnographicFieldnotes Chicago IL University of Chicago Press

20 R Hammack et al Journal of Pre-College Engineering Education Research

11httpdxdoiorg1077712157-92881102

assessment of the quality of precollege STEM educa-tion In fact engineering is the only STEM discipline thatdoes not have its own set of national content standardsEfforts have been made however to integrate engineeringwithin technology and science standards (Achieve Inc2013 International Technology Education Association2000)

Research has suggested that one of the benefits of PK-12engineering education is the improvement of studentsrsquoability to apply mathematics and science skills (Committeeon K-12 Engineering Education 2009) Until recentlyteachers in the United States have focused on mathematicsand science coursework as the way to encourage studentsto enter the engineering pipeline (National Academy ofEngineering amp National Research Council 2009) Howeverthis focus on mathematics and science has not resulted inenough students who enter post-secondary engineeringprograms to meet projected job demands According toSocial Cognitive Career Theory individuals choose careerpaths based in part on their interests attitudes and values(Lent Brown amp Hackett 1994) Further interest in STEMoccupations has been linked to studentsrsquo knowledge ofSTEM careers (Robinson amp Kenny 2003) The lack ofstudents entering engineering programs may be attributableto a lack of understanding of what type of work engineersperform

There is a plethora of research related to studentsrsquo attitudestowards science (ie Jones Howe amp Rua 2000 OsborneSimon amp Collins 2003) mathematics (for example WeinbergBasile amp Albright 2011 Zambo amp Zambo 2006) and theimpact of outreach programs on those attitudes (ie Oliver ampVenville 2011 Stake amp Mares 2001) However the literaturepertaining to studentsrsquo attitudes towards engineering is some-what limited Much of the work pertaining to perceptions ofengineers has focused on elementary studentsrsquo participationin the Engineering is Elementary curriculum developed bythe Boston Museum of Science (Cunningham Lachapelle ampLindgren-Streicher 2005) The existing dearth of researchfocused on middle school studentsrsquo attitudes towardsengineering and technology makes the current study animportant addition to the field This study attempts toaddress a void in the literature by examining the impactsof a summer engineering outreach program on middleschools studentsrsquo attitudes toward conceptions of engi-neering and technology

Purpose and Rationale

There is an emerging yet limited body of work related tothe impact of summer outreach programs on studentsrsquoperceptions of engineering Studies focused on attitudestowards engineering can not only identify student perceptionsof engineering but also lead to a better understanding of thereasons why those perceptions exist Identifying reasonsbehind student perceptions could help lead to interventions to

improve student attitudes towards engineering and eventuallyresult in more high school graduates pursuing engineeringdegrees

The goal of the summer camp was to engage middle schoolstudents in hands-on age-appropriate activities that would

N allow students to differentiate between the activities ofscientists and engineers

N introduce students to engineering as a possible careerpath with a focus on the field of chemical engineering

N engage students in collaborative problem-based learn-ing assignments that integrate mathematics scienceand engineering and

N provide examples of how engineers engage inengineering design to solve problems

The purpose of this study was to determine howparticipating in an engineering summer camp affectedmiddle school studentsrsquo attitudes towards engineering andtheir conceptions of engineering and technology Ourprimary research question addressed in this study was Howdoes participating in an engineering summer camp impactmiddle school studentsrsquo attitudes towards and conceptionsof engineering and technology

Background Literature

K-12 Student Perceptions of Scientists and Engineers

Draw-a-Scientist (DAS) (Mason Kahle amp Gardner1991) and more recently Draw-an-Engineer (DAE)(Knight amp Cunningham 2004) instruments are commonlyused to identify studentsrsquo perceptions of scientists andengineers respectively DAS and DAE require students todraw a scientist or engineer at work and often ask thestudents to describe what they have drawn Becausestudents often confuse engineers with other occupationssuch as scientists (Karatas Micklos amp Bodner 2011) it isexpected that parallels can be drawn between the two

Analysis of drawings from DAS highlight variousstereotypes associated with science For example in astudy of 1600 middle school students Fralick KearnThompson and Lyons (2009) reported that 33 of thestudents drew female scientists Likewise when asked todraw a picture of a scientist Cavallo (2007) reported onlyfive out of 150 fourth through sixth grade students depictedfemale scientists in their drawings Additionally researchfindings indicate that many individuals hold stereotypicalviews of scientists as individuals who lead solitary asunappealing lifestyles Cavallo (2007) administered theDAS to 150 middle school students whose most commonimages were of lonely white men who often had crazy hairor monster-like features In a survey of high schoolstudents Miller Blessing and Schwartz (2006) found thatparticipants viewed scientists as uncaring and passionlessloners Furthermore the girls in their study tended to

R Hammack et al Journal of Pre-College Engineering Education Research 11

2httpdxdoiorg1077712157-92881102

perceive scientists as unsociable individuals who do notbenefit society (Miller et al 2006)

Studies implementing DAE have reported similar findingsas those using DAS For example Fralick et al (2009)reported that only 13 of the 1600 participating studentsdrew female engineers Similarly in a study of 20 sixth-grade students conducted by Karatas et al (2011) only onestudent drew a female when asked to draw an engineer atwork Knight and Cunningham (2004) reported higherpercentages of DAE images that portrayed females in theirdrawings (39) however most of these drawings came froma classroom where two female engineering college studentsworked with the children Likewise students participatingin an after school engineering program for girls weremore likely to depict female engineers in their drawings(Hammack and High 2014) however it is important to notethat all of the engineers involved in the after school programwere female and may have served as the basis for theparticipantsrsquo drawings Moreover DAE images suggest thatstudents tend to view the job duties of engineers as fixingmachines building things and driving trains (Fralick et al2009 Cunningham et al 2005 Knight amp Cunningham2004) It is reasonable to believe that these stereotypes stemfrom a lack of knowledge about scientists engineers and thenature of engineering

Influences on Career Choice

Studentsrsquo prior knowledge about a profession influencestheir decision to pursue that profession For exampleWyss Heulskamp and Siebert (2012) reported that view-ing recorded video interviews with STEM professionalshad a positive impact on middle school studentsrsquo interestsin STEM careers Additionally students begin makingdecisions about careers as early as middle school soproviding information about STEM careers prior to andduring the middle school years is important (Wyss et al2012) Thus elementary- and middle-level students needmore exposure to STEM initiatives which can only besuccessful if their teachers know how to implement STEMmaterials into the curriculum (DeJarnette 2012) In a recentstudy high school and college students described that themost important factor influencing their career choice was apersonal interest in the area which was primarily due to theSTEM knowledge of teachers and school counselors andparental influence about the career (Hall Dickerson BattsKauffmann amp Bosse 2011) Additionally Maltese and Tai(2010) reported that nearly 40 of science professionalsand graduate students reported school-related factorssuch as teachers as the reason for their science interestsBecause teachers play an important role in shapingadolescentsrsquo views of STEM it is important that teachersare also knowledgeable of STEM careers

Although adolescents report the influence of parentsteachers and counselors on their career choices these

individuals often lack knowledge of STEM careers that isneeded to properly advise students In a study conducted byHall et al (2011) parents rated their knowledge of STEMareas lower than other career fields whereas 613 of highschool STEM teachers and counselors did not feel knowl-edgeable of engineering careers This points to the limitedunderstanding some high school counselors and teachersmay hold about science

Programs to Improve Attitudes Toward STEM

There are numerous studies pointing to the benefits of scienceoutreach programs on studentsrsquo attitudes towards science (forexample Cavanaugh 2007 Metin amp Leblebicioglu 2011Rahm Moore amp Martel-Reny 2005 Robbins amp Schoenfisch2005) and mathematics (for example Weinberg Basile ampAlbright 2011) Engineering andor technology focusedcamps have been found to have positive impacts on studentsrsquoviews of engineering (Elam Donham amp Soloman 2012Nadelson and Callahan 2011) and technology (NugentBarker Grandgenett amp Adamchuk 2010) Additionallyoutreach programs that incorporate engineering into the schoolcurriculum have had positive impacts on studentsrsquo views ofengineering (Hirsch Carpinelli Kimmel Rockland amp Bloom2007 Plant Baylor Doerr amp Rosenberg-Kima 2009) Plantet al (2009) investigated the impact of using computeranimated interface agents on 106 middle schools studentsrsquomath performance and attitudes towards the utility of math andhard sciences and found that interacting with a female agentresulted in fewer gender-related perceptions of engineering inmale students but female studentsrsquo perceptions remainedunchanged Hirsch et al (2007) reported that middle schoolstudents who were exposed to engineering within their schoolmath and science curriculum had more positive attitudestowards engineering and a greater knowledge of engineeringcareers than students who were not exposed to engineeringconcepts within school curriculum Unlike Hirsch et al(2007) Mooney and Laubach (2002) reported that partici-pating in a one to three-week-long inquiry-based engi-neering curriculum units within science and mathcurriculum had limited impacts on studentsrsquo attitudestowards engineering with attitudes declining for somefemale subgroups However Mooney and Laubach (2002)reported improved attitudes towards mathematics andscience after participating in the engineering curriculumwith some male students showing improved attitudestowards engineering

Although the previously mentioned studies point to thebenefits that some engineering outreach programs have formiddle school students the diverse results and limitednumber of studies point to a need to further research on theimpacts of middle school engineering outreach programsSimilarly a deficit still exists in the research literature onstudies devoted solely to the impact of middle schoolengineering camps particularly camps devoted to the field

12 R Hammack et al Journal of Pre-College Engineering Education Research

3httpdxdoiorg1077712157-92881102

of chemical engineering As such our understanding of theimpact of chemical engineering outreach programs onstudentsrsquo attitudes toward engineering is limited

Pedagogical Content Knowledge and Co-Teaching

Effective instruction requires both subject matter knowl-edge and pedagogical content knowledge (Shulman 1986)Pedagogical content knowledge can be defined as lsquolsquothe waysof representing and formulating the subject that make itcomprehensible to othersrsquorsquo (Shulman 1986 p 9) Teacherswith increased subject matter knowledge and pedagogicalcontent knowledge are better able to determine their studentsrsquounderstanding of a topic and make changes to instruction thatimprove student learning (Bischoff 2006)

Co-teaching can be used to allow individuals with expertisein different areas to work together to teach a course This iscommonly seen in K-12 classrooms between a generaleducation and special education teacher There are manyproposed benefits of co-teaching (for example smaller student-teacher ratio more feedback to students more individual orsmall group instruction) however little research exists on theeffectiveness of co-teaching at the K-12 level (Sweigart ampLandrum 2015) and there is an absence of literature related touniversity and K-12 teachers engaging in co-teaching

Methodology

This study utilized a repeated measures design in whichsurvey and content tests were given before and afterparticipating in a summer engineering camp The quantitativesurvey data were used to test the hypothesis that participationin the camp positively impacted studentsrsquo attitudes towardsmathematics science and engineering Additionally open-ended questions were used to further explore studentsrsquo under-standings of engineering and technology Both quantitativeand qualitative data were used to compare results and bringgreater insight into the problem than by using a single method

Setting and Participants

Participants included 19 students (female54 male515white518 multi-racial51 special needs51) who had justcompleted sixth grade at the middle school where the campwas held The middle school was located in a Midwesterntown of approximately 45000 people and served approxi-mately 750 sixth- and seventh-grade students at the time ofthis study The camp was open to all students enrolled at themiddle school There was a small fee to attend the camp butscholarships were available to those in need to ensure thatsocioeconomic status did not prevent participation

Intervention

The summer engineering camp met for 35 hours eachday for four days Two middle school science teachers and

a chemical engineering professor from a local universityfacilitated the camp The middle school science teachersboth held at least a bachelorrsquos degree in a science contentarea The chemical engineering professor selected theactivities and assisted the teachers in the implementationEach of the activities required students to implementengineering design and introduced students to the types ofactivities engineers perform There was a large emphasisplaced on chemical engineering owing to the experiences ofthe professor in that area The students completed thefollowing activities during the camp

N Day 1 Students completed preassessments and thenparticipated in a brief measurement practice activity toensure that they could use graduated cylinders formeasuring volume and digital scales for measuringmass Next the chemical engineer lead a discussionon the similarities and differences between scientistsand engineers Students then completed a candyairplane activity in which they built a model of anairplane using only the provided supplies (forexample candy toothpicks paperclips rubber bands)within a limited amount of time After building themodel the students mass-produced the design Thechemical engineer led a discussion over product andprocess design using the information in Figure 1(Cheville amp Bunting 2011) The importance of processdesign in chemical engineering was discussed One ofthe major features of process design that was discussedwas its use with the mass production of products

N Day 2 Students completed a popcorn challenge inwhich they built a filter for a movie theater that couldseparate popped and unpopped kernels and allow theunpopped kernels to be recycled The chemicalengineer led a discussion about chemical processesand recycling using the information in Figure 2 Therewas a discussion of how the popcorn flowchartcorresponds with a chemical flowchart that would beused by a chemical engineer After completing thepopcorn challenge students were introduced to a slimeactivity During this activity students made slimeobserved its characteristics and then created a newproduct that utilized the slime Students also created anadvertisement for their product This activity was usedto demonstrate the differences between scientists(chemists) and engineers (chemical engineers) On thisday students made a single batch of slime much as achemist would do when identifying the formulation ofa new chemical substance

N Day 3 Students finished the slime activity by workingas chemical engineers to design package and marketa product that made use of the slime We alsodiscussed the role that chemical engineers play in themass production of chemicals that are required to beused in certain products such as pharmaceuticals andpetrochemical

R Hammack et al Journal of Pre-College Engineering Education Research 13

4httpdxdoiorg1077712157-92881102

N Day 4 Students experimented with materials todetermine which reactants would make the bestpropellants for film canister rockets Students designeda rocket body to attach to their film canisters and testedtheir chosen propellants A chemical engineerrsquos role inthe development and testing of rocket propellants wasdiscussed as well as the importance of mass productionof propellants

Prior to each activity students were presented with aproblem scenario Then they had to complete a designchallenge to address the problem For example prior tothe candy airplane activity students were told that anaircraft manufacturer was planning to expand its opera-tions and was looking for a new airplane model and manu-facturing facility Students then designed a model aircraftto present to the manufacturer and then mass-producedtheir design Each scenario provided students with a realworld context in which engineers would work Uponcompletion of each activity facilitators discussed howstudents engaged in engineering design throughout theactivities they created and reiterated real world applica-tions of the concepts

Instruments

Researchers used a variety of instruments in order tomeasure the impact of the program on the participantsrsquoattitudes toward mathematics science and engineeringconceptions of technology and conceptions of engineeringThese measures are described in detail below

Attitudes to Mathematics Science and Engineering(AMSE) The AMSE is a 5-point Likert-scaled instrumentconsisting of 36 items that are categorized into 8 subscalesTable 1 provides a list and description of each subscaleincluding the range of scores Scores on the overall AMSEcan range from 36 to 180 with higher scores being indicativeof a more positive attitude toward mathematics science andengineering (Hirsch et al 2007) Illustrative items includelsquolsquoEngineers are just people who do a lot of math and sciencersquorsquoand lsquolsquoEngineers help make peoplersquos lives betterrsquorsquo

What is Technology Cunningham and colleagues(2005) developed the What is Technology instrument toassess elementary studentsrsquo conceptions of technology Theinstrument consists of images and names of 16 items andone open-ended item How do you know if something istechnology Participants circle the images they associatewith technology and complete the open-ended responseExample items are shoes cellular phone and bird Scoresprovide data related to participantsrsquo ability to correctlyidentify examples of technology which can help research-ers identify misconceptions held by participants

What is Engineering Cunningham and colleagues(2005) also developed the What is Engineering Instrumentwhich consists of 16 images and descriptions of people atwork and one open ended response What is an engineer Forthis instrument participants circle the images with which theyassociate the work engineers perform as a part of their jobsand complete the open-ended response question Illustrativeitems include improve bandages drive machines and work as

Figure 1 Engineering design process used throughout the summer camp

Figure 2 Flowchart used to explain the process in the popcorn activity

14 R Hammack et al Journal of Pre-College Engineering Education Research

5httpdxdoiorg1077712157-92881102

a team Scores provide data related to participantsrsquo ability tocorrectly identify tasks completed by engineers as they workwhich can help researchers identify misconceptions heldabout engineers

Engineering Content Test On the first day of the campparticipants were given a precamp content test developedby the chemical engineering faculty member The surveycontained questions about content specific to chemicalengineering confidences in participating in engineeringdesign and the following journal prompts lsquolsquoWhat does achemical engineer dorsquorsquo and lsquolsquoHave you thought aboutbecoming an engineerrsquorsquo On the last day of camp participantscompleted a post-camp content test and responded to thefollowing prompts lsquolsquoWhat does a chemical engineer dorsquorsquoand lsquolsquoDid camp encourage you to consider engineeringrsquorsquo

Data Collection and Analysis

Camp facilitators pretested all participants at thebeginning of the first day of camp and post-tested on thelast day of camp All students completed the AMSE Whatis an Engineer and What is Technology instruments andEngineering Content test

Quantitative Data Analysis Owing to the smallsample size and lack of a normal distribution of dataresearchers used nonparametric statistics for data analy-sis A Wilcoxon Signed Rank test was performed inSPSS version 210 for the content test score overallAMSE score and all AMSE subscales except thelsquolsquootherrsquorsquo subscale Individualsrsquo subscale scores weredetermined by summing all of the item responses for aparticular subscale Table 1 presents the possible rangesfor an individualrsquos subscale scores On the presurveytwo participants left one item blank from the negativeopinions of mathematics and science subscale On thepost-survey two participants left one item blank onefrom the negative opinions of mathematics and sciencesubscale and one from the interest in engineeringstereotypic aspects subscale To account for eachparticipantrsquos missing value the mean for each indivi-dualrsquos subscales were imputed for the missing item

For example items 4 12 13 21 and 29 comprise thenegative opinions of mathematics and science subscaleParticipant A left item 21 blank on the pretest Hisscores from items 4 12 13 and 29 were averaged andthe resulting value was assigned to item 21

Each participantrsquos What is Technology and What isEngineering responses were scored based on the percen-tage of correct images that were circled Pre and post-testpercentages were entered into SPSS and a WilcoxonSigned Rank analysis was performed Researchers thenexamined the frequencies to identify trends

Each participantrsquos Engineering Content Test was given ascore based on the percentage of items that were correctlyanswered Pre- and post-test percentages were entered intoSPSS and a Wilcoxon Signed Rank analysis was performed

Qualitative Data Analysis The open-ended responseitems on the What is Technology and What is Engineeringinstruments as well as the lsquolsquoWhat does a chemicalengineer dorsquorsquo question were printed onto cards andexamined independently by three researchers to developa coding scheme for the responses The researchersdiscussed the three coding schemes and consolidatedthem into a single coding scheme The cards wereindependently recoded using the consolidated codingscheme There was 95 agreement between the resultingcodes the three researchers discussed the differencesuntil 100 consensus was reached (Emerson Fretz ampShaw 1995)

Results

Attitudes to Mathematics Science and Engineering

Researchers calculated a Cronbachrsquos alpha (086) todetermine the internal consistency for the AMSE instru-ment this value was consistent with the value reported(085) by Hirsch et al (2007) Table 2 presents thedescriptive statistics and Wilcoxon Signed Rank testresults for the seven AMSE subscales Only the Engineersubscale showed a significant difference pre to post(Z 5 0003 p 001)

Table 1Attitudes Toward Mathematics Science and Engineering (AMSE) subscales descriptions and score ranges

Subscale

Subscale ranges

Description Min Max

Overall AMSE 36 180Interest in engineering stereotypic aspects (ISA) Studentsrsquo interest in engineering based on common stereotypes 7 21Interest in engineering non-stereotypic aspects (INSA) Studentsrsquo interest in engineering not based on common stereotypes 4 12Positive opinions of mathematics and science (PO) Studentsrsquo positive opinions towards math and science 5 15Negative opinions of mathematics and science (NO) Studentsrsquo negative opinions towards math and science 5 15Problem solving (PS) Studentsrsquo attitudes towards their problem solving abilities 4 12Technical skills (TS) Studentsrsquo attitudes towards their technical skills 4 12Engineer (E) Studentsrsquo understanding of the type of work engineers perform 5 15

R Hammack et al Journal of Pre-College Engineering Education Research 15

6httpdxdoiorg1077712157-92881102

What is Technology What is Engineering andEngineering Content

Table 3 presents the descriptive statistics and WilcoxonSigned Rank results for the What is Engineering What isTechnology and engineering content measures Pre- to post-test differences were significant for three measures (p 001)

Figure 3 illustrates the percentage of items selected ascorrect on the What is Technology pre and post measuresAn examination of these percentages indicated that manyparticipants associated technology with items that wereelectronic (100 selected TV and phones) or involved theuse of electricity (100 selected subways 95 selectedpower lines) and none of the participants associated techno-logy with living organisms (0 selected bird tree ordandelions) Fewer participants associated technology withnon-electrical manmade items however a greater percentageof participants selected items that may be associated withconstruction (58 selected bridge 63 selected house 84selected factory) than items not generally associated withconstruction (47 selected shoes cups and bandages 52selected books and bikes) On the What is Technology Post-test however participants clearly associated technology withitems that do not exist in nature (95 selected bandages100 selected shoes subway phone bridge TV cupfactory house power lines bicycle books)

Participantsrsquo responses to the What is Engineeringinstrument are displayed in Figure 4 An examination ofstudent responses indicates that seven of the nine itemsassociated with activities of an engineer (marked with anasterisk in Figure 4) increased at the conclusion of theengineering camp These items include verbs such as

improve develop design and create The other two items (ieDesign clean water figure how to track luggage) showed nochange in the number of responses however more than half ofthe students correctly selected these items Of the seven itemsnot associated with activities of an engineer fewer studentsselected that engineers construct buildings sell food and cleanteeth at the conclusion of camp There was an increase in thenumber of students who incorrectly selected driving machinesand repairing cars as acts of an engineer and over half stillchose that an engineer performs the act of installing wiringSurprisingly two participants held on to the misconceptionthat the work of an engineer includes arranging flowers

Open-Ended ResponsesThis section present results from the qualitative analysis

of an open-ended response from three data sources(1) What is an Engineer (2) What is Technology and(3) Engineering Content Test The percent occurrence ofemergent themes from each data source is presentedin Figures 5 6 and 7 On the What is Technology instrumentstudentsrsquo initial responses to lsquolsquoHow do you know something istechnologyrsquorsquo were categorized as electrical manmade orhelps people However at the end of the engineering camp allstudentsrsquo responses indicated an understanding that technologywas something that is manmade Additionally a large percen-tage of studentsrsquo initial responses to the prompt lsquolsquoWhat is anengineerrsquorsquo were categorized as someone who builds designsor improves things to help people On the post-test howeverthe percentage of responses falling within these categoriesdeclined greatly and a new theme (mass production) wasidentified in 68 of responses Studentsrsquo Engineering ContentTest responses indicated growth in their understanding of

Table 2Summary statistics for participantsrsquo (n519) scores on Attitudes Towards Mathematics Science and Engineering Scale (AMSE)

Subscale

Pretest Post-test

Z pMin Max Mdn Min Max Mdn

AMSE-Overall 81 139 111 78 140 114 20829 0407ISA 13 32 22 9 31 24 20156 0876INSA 5 18 10 4 19 10 20604 0546PO 5 23 17 8 23 16 21225 0220NO 5 13 9 5 21 8 20192 0848PS 6 20 12 6 16 13 20287 0774TS 10 20 14 9 20 13 0528 0528E 14 22 18 16 23 19 0003 0003

p 001

Table 3Summary statistics for participantrsquos scores on What is Engineering and What is Technology and Engineering Content instruments

Instrument n

Pretest Post-test

Z PMin Max Mdn Min Max Mdn

Engineering 19 375 9375 625 50 100 75 23285 0001Technology 19 4375 100 75 100 100 100 22818 0005Content 19 0 57 14 29 86 57 23456 0001

p 001

16 R Hammack et al Journal of Pre-College Engineering Education Research

7httpdxdoiorg1077712157-92881102

chemical engineers Examples of studentsrsquo open-endedresponses on the Engineering Content Test are presentedin Table 4 When asked what chemical engineers doresponses of lsquolsquoI donrsquot knowrsquorsquo decreased from 26 to 5by the end of the camp Additionally analysis ofstudentsrsquo open-ended responses indicated a shift in theirunderstanding from thinking that chemical engineersonly mix and use chemicals to engaging in more complexwork (ie experimenting) with chemicals

Discussion

Overall the results indicate that participants improved their(1) understanding of technology (2) chemical engineeringcontent knowledge and (3) attitudes towards engineering Ofthe seven AMSE subscales tested only the responses on theEngineer subscale changed significantly This indicates thatover the course of the camp participantsrsquo attitudes towardsengineering became more positive with more participants

reporting that they know what an engineer does These resultsare consistent with the open-ended response questionsResponses to the open-ended lsquolsquoWhat is an Engineerrsquorsquo prompton the What is Engineering post-test indicate that partici-pants gained an understanding of engineers as people whowork in teams and mass produce products

The scores on the What is Technology instrument weresignificantly higher on the post-test indicating that the campintervention had a positive impact on the participantsrsquo under-standings of technology As evident in the frequency counts andopen-ended responses many participants began camp with themisconception that technology only included items that requiredelectricity These results are consistent with the findings ofCunningham et al (2005) who found that students in grades1ndash5 tend to associate technology with objects requiringelectricity By the end of camp all participants reported thattechnology referred to any object that was manmade

Scores on the Engineering Content Test were significantlyhigher on the post-test indicating that the intervention had

0

20

40

60

80

100

S

elec

ted

Item

Pre

Post

Figure 3 Frequency of participantsrsquo responses for items on the What is Technology instrument Items that are considered technology

0

20

40

60

80

100

S

elec

ted

Item

Pre

Post

Figure 4 Frequency of participantsrsquo responses for items on the What is Engineering instrument Items associated with engineering

R Hammack et al Journal of Pre-College Engineering Education Research 17

8httpdxdoiorg1077712157-92881102

a positive impact on participantsrsquo content knowledge relatedto chemical engineering Although the median post-testscore was over 40 points higher than the pretest medianit was still only 57 which is below the 70 proficiencyscore used by the school where the camp was held Thissuggests that while participants increased their understandingof chemical engineering they still did not master all of thecontent covered in camp The short duration of camp maynot have provided enough time for students to fully masterthe new concepts

The range in frequencies on the What is Engineeringpretest point to participantsrsquo uncertainties about the activitiesof an engineer There was still a large range in the post-testresults indicating that participants still had some degree ofuncertainty about the work of engineers However there weresome positive changes worth noting Prior to participating incamp 68 of participants viewed engineers as people whowork as a team whereas all campers circled lsquolsquowork as ateamrsquorsquo on the post instrument All of the camp activitiesrequired that participants work as a team to solve problemswhich may have accounted for this change Recognizing that

engineers are not solitary loners but in fact work in teams totackle important real world problems could help combat thecommon negative stereotypes previously reported (Cavallo2007 Miller et al 2006) and may prevent more adole-scents form abandoning the STEM pipeline Additionallyparticipantsrsquo Engineering Content Test responses also indi-cated a positive shift in their attitudes towards engineering as acareer For instance only 26 of camp participants reportedthat they had considered becoming a chemical engineer priorto camp whereas 63 of participants reported that campencouraged them to consider chemical engineering as a career

Although the findings indicate an increase in the number ofstudent considering a career in engineering they still heldmany misconceptions about the work of engineers It isimportant to note that the engineering camp did not explicitlyaddress all misconceptions presented on the What isEngineering instrument However the camp did discuss thatengineers focus their work on the design and creative aspectsof construction but are not the actual builders This highlights atrend in our findings that the participantsrsquo views of engineeringshifted away from engineers as mere builders and a movetowards an understanding that engineers design solutions toproblems This trend is mirrored in the open-ended responsesas nearly all participants recognized that lsquolsquobuildingrsquorsquo was notan act of engineering by the end of the camp The only twolsquolsquobuildingrsquorsquo responses on the post-test were

lsquolsquoAn engineer is a person who builds a lot of one thingrsquorsquolsquolsquoSomeone who builds lots of stuffrsquorsquoIt is important to note that one major focus of the candy

airplane activity was the role engineers play in industry andmass production and the mass production of chemicals wasdiscussed during both the slime and film canister rocketactivities It is therefore not surprising that over one-half of theparticipants referred to mass production in their post-testresponses The two participants who wrote lsquolsquobuildrsquorsquo in their post

0102030405060708090

100

r

espo

ndin

g

Theme

Pre

Post

Figure 5 Frequency of themes for What is Engineering instrument

0

20

40

60

80

100

120

Electrical Man made Helps people

r

espo

ndin

g

Theme

Pre

Post

Figure 6 Frequency of themes for What is Technology instrument

18 R Hammack et al Journal of Pre-College Engineering Education Research

9httpdxdoiorg1077712157-92881102

responses may also have been referring to mass productionrather than simply building or constructing an object

Interestingly there are conflicting findings between thequantitative and qualitative responses on the What is

Engineering instrument Although the number of imagesstudents selected of engineers designing and improvingproducts increased students qualitative responses related todesigning and improving products decreased Because the

0

10

20

30

40

50

60

r

espo

ndin

g

Theme

Pre

Post

Figure 7 Frequency of themes for Engineering Content Test responses

Table 4Examples of participantsrsquo open-ended responses by theme

Theme Example Pre-program Responses Example Post-program Responses

How do you know something is technologyElectrical It creates energy and has circuit boards and stuff NAMan made Man invents technology eg bicycle subway Something that is technology is a man-made objectHelps people If it was invented by people to help other people NA

What is an engineerMass production NA An engineer creates multiple copies of a productBuilds Someone who builds something Someone who builds lots of stuffDesigns Somebody who designs and builds stuff An engineer designs some product and fins a way

to mass produce itImproves thingshelps others Someone who researches and develops innovations

to make this world a better and easier placeSomeone who improves inventions

Fixes things An engineer is someone who fixes or creates things An engineer is a person that fixes things andworks as a team

Team work NA An engineer is a person that fixes things andworks as a team

Uses tools I think it is someone who builds or designsthings and works with tools and differentkinds of machinery

NA

Uses math and science People who do math and science NA

What does a chemical engineer doI donrsquot know I donrsquot really know I canrsquot describe itMixesUses chemicals Mixes chemicals Someone who works with mixing and learning

about chemicalsManufacturingdesigns new

products or chemicalsWorks with chemistry to make new materials Helps manufacture chemicals

Experiments with chemicals A chemical engineer experiments with different chemicalsto see if they do something useful or harmful

Experiments with different chemicals to seewhat they do

Works with food NA Works with food

R Hammack et al Journal of Pre-College Engineering Education Research 19

10httpdxdoiorg1077712157-92881102

candy airplane slime and rocket activities all dealt with massproduction it is possible that participants concentrated on theaspect of mass production during their open-ended responsesHowever half of participantsrsquo journal prompts about the workof a chemical engineer referred to some aspect of designing orcreating a product on the post-test Additionally there wasa decrease in the qualitative responses related to lsquolsquohelpingpeoplersquorsquo on the What is Technology and What isEngineering instruments The decrease in balanced bythe increase in lsquolsquomanmadersquorsquo responses on the What isTechnology instrument and lsquolsquomass productionrsquorsquo responseson the What is Engineering instrument

Overall the findings indicate that the participants gainedan understanding that engineers are involved with the massproduction of products as well as their design andimprovement As such the decrease in studentsrsquo responsesin these categories is balanced by the increase in thenumber of responses that indicated engineers as massproducers of items Follow-up interviews with participantswould have been valuable as a means of providingadditional information as to how participantsrsquo perceptionsof engineers changed as a result of camp

Limitations

Although the results indicate a positive impact onparticipants it is not clear which components of the camp(hands-on activities interaction with a professional engi-neer combination of factors) contributed to this changeFollow-up interviews could have helped identify thesefactors Campers self-selected into the program and mayhave already had an above average interest in STEM Thiscoupled with the small sample size limits the general-izability of the findings Results were also limited by theinstruments used The What is Engineering instrumentuses a specific definition of engineering that is notuniversally accepted Additionally the validity of thecontent test has not been established and the test has onlybeen used with the 19 study participants Because we onlycollected and analyzed data immediately following thesummer camp our study only addresses short-termimpacts Future research should be conducted to determinelong-term impacts of engineering outreach programs

Conclusion and Implications

In answering our research questions we found thatparticipation in an engineering summer camp had a positiveimpact on middle school studentsrsquo understandings of whattechnology is and what engineers do evidenced by both thequantitative and qualitative data The open-ended questionssupported and added detail to the quantitative findingsproviding more evidence than if either data set had beenused independently Participantsrsquo attitudes towards mathe-matics and science were not significantly impacted by the

summer camp However their attitudes towards engineer-ing were significantly impacted The short duration of thecamp limited the quantity of information that could bepresented to participants with the current intervention focusingprimarily on engineering Although measurable impacts wereaccomplished during the week long summer camp long terminterventions such as the infusion of engineering activitieswithin the school curriculum could allow for attitudinalchanges across multiple STEM disciplines

The partnership between practicing teachers and engineer-ing faculty was important to the success of the camp Althoughthe extent to which co-teaching impacted outcomes was notmeasured the diverse backgrounds of the camp facilitatorsallowed for the combination of pedagogical knowledge andengineering content expertise to create rich engineeringexperiences that were developmentally appropriate for middleschool participants These experiences positively impactedparticipantsrsquo attitudes towards engineering revealing thebenefits of collaborative efforts between K-12 educators andengineering professionals Additional research in this area isneeded

References

Achieve Inc (2013) Next generation science standards WashingtonDC Achieve Inc

Bischoff P J (2006) The role of knowledge structures in the ability ofpreservice elementary teachers to diagnose a childrsquos understanding ofmolecular kinetics Science Education 90(5) 936ndash951 doi101002sce20155

Cavallo A (2007) Draw-a-ScientistMystery box Science and Children45(3) 37ndash41 Retrieved from httpwwwnstaorgpublicationsbrowse_journalsaspxaction5issueampid51025053sc07_045_03

Cavanaugh S (2007) Science camp Just for the girls Academic camps are onthe rise across the country including ones to get adolescent girls excitedabout the exploration of science Education Week 26(45) 26ndash28

Cheville A amp Bunting C (2011) Engineering students for the 21stcentury Student development through the curriculum Advances inEngineering Education 2(4) 1ndash36 Retrieved from httpadvancesaseeorgwp-contentuploadsvol02issue04papersaee-vol02-issue04-p10pdf

Committee on K-12 Engineering Education (2009) In L Katehi GPearson amp M Feder (Eds) Engineering in K-12 educationunderstanding the status and improving the prospects WashingtonDC The National Academies Press

Cunningham C Lachapelle C amp Lindgren-Streicher A (2005) Assessingelementary schools studentsrsquo conceptions of engineering and technologyProceedings of the 2005 American Society for Engineering EducationAnnual Conference amp Exposition Portland OR

DeJarnette N K (2012) Americarsquos children Providing early exposure toSTEM (science technology engineering and math) initiativesEducation 133(1) 77ndash84 Retrieved from httpwwwprojectinnovationbizeducation_2006html

Elam M E Donham B L amp Soloman S R (2012) An engineeringsummer program for underrepresented students from rural schooldistricts Journal of STEM Education 13(2) 35ndash44 Retrieved fromhttpojsjstemorgindexphpjournal5JSTEMamppage5articleampop5viewamppath5B5D51619amppath5B5D51437

Emerson R M Fretz R I amp Shaw L L (1995) Writing EthnographicFieldnotes Chicago IL University of Chicago Press

20 R Hammack et al Journal of Pre-College Engineering Education Research

11httpdxdoiorg1077712157-92881102

perceive scientists as unsociable individuals who do notbenefit society (Miller et al 2006)

Studies implementing DAE have reported similar findingsas those using DAS For example Fralick et al (2009)reported that only 13 of the 1600 participating studentsdrew female engineers Similarly in a study of 20 sixth-grade students conducted by Karatas et al (2011) only onestudent drew a female when asked to draw an engineer atwork Knight and Cunningham (2004) reported higherpercentages of DAE images that portrayed females in theirdrawings (39) however most of these drawings came froma classroom where two female engineering college studentsworked with the children Likewise students participatingin an after school engineering program for girls weremore likely to depict female engineers in their drawings(Hammack and High 2014) however it is important to notethat all of the engineers involved in the after school programwere female and may have served as the basis for theparticipantsrsquo drawings Moreover DAE images suggest thatstudents tend to view the job duties of engineers as fixingmachines building things and driving trains (Fralick et al2009 Cunningham et al 2005 Knight amp Cunningham2004) It is reasonable to believe that these stereotypes stemfrom a lack of knowledge about scientists engineers and thenature of engineering

Influences on Career Choice

Studentsrsquo prior knowledge about a profession influencestheir decision to pursue that profession For exampleWyss Heulskamp and Siebert (2012) reported that view-ing recorded video interviews with STEM professionalshad a positive impact on middle school studentsrsquo interestsin STEM careers Additionally students begin makingdecisions about careers as early as middle school soproviding information about STEM careers prior to andduring the middle school years is important (Wyss et al2012) Thus elementary- and middle-level students needmore exposure to STEM initiatives which can only besuccessful if their teachers know how to implement STEMmaterials into the curriculum (DeJarnette 2012) In a recentstudy high school and college students described that themost important factor influencing their career choice was apersonal interest in the area which was primarily due to theSTEM knowledge of teachers and school counselors andparental influence about the career (Hall Dickerson BattsKauffmann amp Bosse 2011) Additionally Maltese and Tai(2010) reported that nearly 40 of science professionalsand graduate students reported school-related factorssuch as teachers as the reason for their science interestsBecause teachers play an important role in shapingadolescentsrsquo views of STEM it is important that teachersare also knowledgeable of STEM careers

Although adolescents report the influence of parentsteachers and counselors on their career choices these

individuals often lack knowledge of STEM careers that isneeded to properly advise students In a study conducted byHall et al (2011) parents rated their knowledge of STEMareas lower than other career fields whereas 613 of highschool STEM teachers and counselors did not feel knowl-edgeable of engineering careers This points to the limitedunderstanding some high school counselors and teachersmay hold about science

Programs to Improve Attitudes Toward STEM

There are numerous studies pointing to the benefits of scienceoutreach programs on studentsrsquo attitudes towards science (forexample Cavanaugh 2007 Metin amp Leblebicioglu 2011Rahm Moore amp Martel-Reny 2005 Robbins amp Schoenfisch2005) and mathematics (for example Weinberg Basile ampAlbright 2011) Engineering andor technology focusedcamps have been found to have positive impacts on studentsrsquoviews of engineering (Elam Donham amp Soloman 2012Nadelson and Callahan 2011) and technology (NugentBarker Grandgenett amp Adamchuk 2010) Additionallyoutreach programs that incorporate engineering into the schoolcurriculum have had positive impacts on studentsrsquo views ofengineering (Hirsch Carpinelli Kimmel Rockland amp Bloom2007 Plant Baylor Doerr amp Rosenberg-Kima 2009) Plantet al (2009) investigated the impact of using computeranimated interface agents on 106 middle schools studentsrsquomath performance and attitudes towards the utility of math andhard sciences and found that interacting with a female agentresulted in fewer gender-related perceptions of engineering inmale students but female studentsrsquo perceptions remainedunchanged Hirsch et al (2007) reported that middle schoolstudents who were exposed to engineering within their schoolmath and science curriculum had more positive attitudestowards engineering and a greater knowledge of engineeringcareers than students who were not exposed to engineeringconcepts within school curriculum Unlike Hirsch et al(2007) Mooney and Laubach (2002) reported that partici-pating in a one to three-week-long inquiry-based engi-neering curriculum units within science and mathcurriculum had limited impacts on studentsrsquo attitudestowards engineering with attitudes declining for somefemale subgroups However Mooney and Laubach (2002)reported improved attitudes towards mathematics andscience after participating in the engineering curriculumwith some male students showing improved attitudestowards engineering

Although the previously mentioned studies point to thebenefits that some engineering outreach programs have formiddle school students the diverse results and limitednumber of studies point to a need to further research on theimpacts of middle school engineering outreach programsSimilarly a deficit still exists in the research literature onstudies devoted solely to the impact of middle schoolengineering camps particularly camps devoted to the field

12 R Hammack et al Journal of Pre-College Engineering Education Research

3httpdxdoiorg1077712157-92881102

of chemical engineering As such our understanding of theimpact of chemical engineering outreach programs onstudentsrsquo attitudes toward engineering is limited

Pedagogical Content Knowledge and Co-Teaching

Effective instruction requires both subject matter knowl-edge and pedagogical content knowledge (Shulman 1986)Pedagogical content knowledge can be defined as lsquolsquothe waysof representing and formulating the subject that make itcomprehensible to othersrsquorsquo (Shulman 1986 p 9) Teacherswith increased subject matter knowledge and pedagogicalcontent knowledge are better able to determine their studentsrsquounderstanding of a topic and make changes to instruction thatimprove student learning (Bischoff 2006)

Co-teaching can be used to allow individuals with expertisein different areas to work together to teach a course This iscommonly seen in K-12 classrooms between a generaleducation and special education teacher There are manyproposed benefits of co-teaching (for example smaller student-teacher ratio more feedback to students more individual orsmall group instruction) however little research exists on theeffectiveness of co-teaching at the K-12 level (Sweigart ampLandrum 2015) and there is an absence of literature related touniversity and K-12 teachers engaging in co-teaching

Methodology

This study utilized a repeated measures design in whichsurvey and content tests were given before and afterparticipating in a summer engineering camp The quantitativesurvey data were used to test the hypothesis that participationin the camp positively impacted studentsrsquo attitudes towardsmathematics science and engineering Additionally open-ended questions were used to further explore studentsrsquo under-standings of engineering and technology Both quantitativeand qualitative data were used to compare results and bringgreater insight into the problem than by using a single method

Setting and Participants

Participants included 19 students (female54 male515white518 multi-racial51 special needs51) who had justcompleted sixth grade at the middle school where the campwas held The middle school was located in a Midwesterntown of approximately 45000 people and served approxi-mately 750 sixth- and seventh-grade students at the time ofthis study The camp was open to all students enrolled at themiddle school There was a small fee to attend the camp butscholarships were available to those in need to ensure thatsocioeconomic status did not prevent participation

Intervention

The summer engineering camp met for 35 hours eachday for four days Two middle school science teachers and

a chemical engineering professor from a local universityfacilitated the camp The middle school science teachersboth held at least a bachelorrsquos degree in a science contentarea The chemical engineering professor selected theactivities and assisted the teachers in the implementationEach of the activities required students to implementengineering design and introduced students to the types ofactivities engineers perform There was a large emphasisplaced on chemical engineering owing to the experiences ofthe professor in that area The students completed thefollowing activities during the camp

N Day 1 Students completed preassessments and thenparticipated in a brief measurement practice activity toensure that they could use graduated cylinders formeasuring volume and digital scales for measuringmass Next the chemical engineer lead a discussionon the similarities and differences between scientistsand engineers Students then completed a candyairplane activity in which they built a model of anairplane using only the provided supplies (forexample candy toothpicks paperclips rubber bands)within a limited amount of time After building themodel the students mass-produced the design Thechemical engineer led a discussion over product andprocess design using the information in Figure 1(Cheville amp Bunting 2011) The importance of processdesign in chemical engineering was discussed One ofthe major features of process design that was discussedwas its use with the mass production of products

N Day 2 Students completed a popcorn challenge inwhich they built a filter for a movie theater that couldseparate popped and unpopped kernels and allow theunpopped kernels to be recycled The chemicalengineer led a discussion about chemical processesand recycling using the information in Figure 2 Therewas a discussion of how the popcorn flowchartcorresponds with a chemical flowchart that would beused by a chemical engineer After completing thepopcorn challenge students were introduced to a slimeactivity During this activity students made slimeobserved its characteristics and then created a newproduct that utilized the slime Students also created anadvertisement for their product This activity was usedto demonstrate the differences between scientists(chemists) and engineers (chemical engineers) On thisday students made a single batch of slime much as achemist would do when identifying the formulation ofa new chemical substance

N Day 3 Students finished the slime activity by workingas chemical engineers to design package and marketa product that made use of the slime We alsodiscussed the role that chemical engineers play in themass production of chemicals that are required to beused in certain products such as pharmaceuticals andpetrochemical

R Hammack et al Journal of Pre-College Engineering Education Research 13

4httpdxdoiorg1077712157-92881102

N Day 4 Students experimented with materials todetermine which reactants would make the bestpropellants for film canister rockets Students designeda rocket body to attach to their film canisters and testedtheir chosen propellants A chemical engineerrsquos role inthe development and testing of rocket propellants wasdiscussed as well as the importance of mass productionof propellants

Prior to each activity students were presented with aproblem scenario Then they had to complete a designchallenge to address the problem For example prior tothe candy airplane activity students were told that anaircraft manufacturer was planning to expand its opera-tions and was looking for a new airplane model and manu-facturing facility Students then designed a model aircraftto present to the manufacturer and then mass-producedtheir design Each scenario provided students with a realworld context in which engineers would work Uponcompletion of each activity facilitators discussed howstudents engaged in engineering design throughout theactivities they created and reiterated real world applica-tions of the concepts

Instruments

Researchers used a variety of instruments in order tomeasure the impact of the program on the participantsrsquoattitudes toward mathematics science and engineeringconceptions of technology and conceptions of engineeringThese measures are described in detail below

Attitudes to Mathematics Science and Engineering(AMSE) The AMSE is a 5-point Likert-scaled instrumentconsisting of 36 items that are categorized into 8 subscalesTable 1 provides a list and description of each subscaleincluding the range of scores Scores on the overall AMSEcan range from 36 to 180 with higher scores being indicativeof a more positive attitude toward mathematics science andengineering (Hirsch et al 2007) Illustrative items includelsquolsquoEngineers are just people who do a lot of math and sciencersquorsquoand lsquolsquoEngineers help make peoplersquos lives betterrsquorsquo

What is Technology Cunningham and colleagues(2005) developed the What is Technology instrument toassess elementary studentsrsquo conceptions of technology Theinstrument consists of images and names of 16 items andone open-ended item How do you know if something istechnology Participants circle the images they associatewith technology and complete the open-ended responseExample items are shoes cellular phone and bird Scoresprovide data related to participantsrsquo ability to correctlyidentify examples of technology which can help research-ers identify misconceptions held by participants

What is Engineering Cunningham and colleagues(2005) also developed the What is Engineering Instrumentwhich consists of 16 images and descriptions of people atwork and one open ended response What is an engineer Forthis instrument participants circle the images with which theyassociate the work engineers perform as a part of their jobsand complete the open-ended response question Illustrativeitems include improve bandages drive machines and work as

Figure 1 Engineering design process used throughout the summer camp

Figure 2 Flowchart used to explain the process in the popcorn activity

14 R Hammack et al Journal of Pre-College Engineering Education Research

5httpdxdoiorg1077712157-92881102

a team Scores provide data related to participantsrsquo ability tocorrectly identify tasks completed by engineers as they workwhich can help researchers identify misconceptions heldabout engineers

Engineering Content Test On the first day of the campparticipants were given a precamp content test developedby the chemical engineering faculty member The surveycontained questions about content specific to chemicalengineering confidences in participating in engineeringdesign and the following journal prompts lsquolsquoWhat does achemical engineer dorsquorsquo and lsquolsquoHave you thought aboutbecoming an engineerrsquorsquo On the last day of camp participantscompleted a post-camp content test and responded to thefollowing prompts lsquolsquoWhat does a chemical engineer dorsquorsquoand lsquolsquoDid camp encourage you to consider engineeringrsquorsquo

Data Collection and Analysis

Camp facilitators pretested all participants at thebeginning of the first day of camp and post-tested on thelast day of camp All students completed the AMSE Whatis an Engineer and What is Technology instruments andEngineering Content test

Quantitative Data Analysis Owing to the smallsample size and lack of a normal distribution of dataresearchers used nonparametric statistics for data analy-sis A Wilcoxon Signed Rank test was performed inSPSS version 210 for the content test score overallAMSE score and all AMSE subscales except thelsquolsquootherrsquorsquo subscale Individualsrsquo subscale scores weredetermined by summing all of the item responses for aparticular subscale Table 1 presents the possible rangesfor an individualrsquos subscale scores On the presurveytwo participants left one item blank from the negativeopinions of mathematics and science subscale On thepost-survey two participants left one item blank onefrom the negative opinions of mathematics and sciencesubscale and one from the interest in engineeringstereotypic aspects subscale To account for eachparticipantrsquos missing value the mean for each indivi-dualrsquos subscales were imputed for the missing item

For example items 4 12 13 21 and 29 comprise thenegative opinions of mathematics and science subscaleParticipant A left item 21 blank on the pretest Hisscores from items 4 12 13 and 29 were averaged andthe resulting value was assigned to item 21

Each participantrsquos What is Technology and What isEngineering responses were scored based on the percen-tage of correct images that were circled Pre and post-testpercentages were entered into SPSS and a WilcoxonSigned Rank analysis was performed Researchers thenexamined the frequencies to identify trends

Each participantrsquos Engineering Content Test was given ascore based on the percentage of items that were correctlyanswered Pre- and post-test percentages were entered intoSPSS and a Wilcoxon Signed Rank analysis was performed

Qualitative Data Analysis The open-ended responseitems on the What is Technology and What is Engineeringinstruments as well as the lsquolsquoWhat does a chemicalengineer dorsquorsquo question were printed onto cards andexamined independently by three researchers to developa coding scheme for the responses The researchersdiscussed the three coding schemes and consolidatedthem into a single coding scheme The cards wereindependently recoded using the consolidated codingscheme There was 95 agreement between the resultingcodes the three researchers discussed the differencesuntil 100 consensus was reached (Emerson Fretz ampShaw 1995)

Results

Attitudes to Mathematics Science and Engineering

Researchers calculated a Cronbachrsquos alpha (086) todetermine the internal consistency for the AMSE instru-ment this value was consistent with the value reported(085) by Hirsch et al (2007) Table 2 presents thedescriptive statistics and Wilcoxon Signed Rank testresults for the seven AMSE subscales Only the Engineersubscale showed a significant difference pre to post(Z 5 0003 p 001)

Table 1Attitudes Toward Mathematics Science and Engineering (AMSE) subscales descriptions and score ranges

Subscale

Subscale ranges

Description Min Max

Overall AMSE 36 180Interest in engineering stereotypic aspects (ISA) Studentsrsquo interest in engineering based on common stereotypes 7 21Interest in engineering non-stereotypic aspects (INSA) Studentsrsquo interest in engineering not based on common stereotypes 4 12Positive opinions of mathematics and science (PO) Studentsrsquo positive opinions towards math and science 5 15Negative opinions of mathematics and science (NO) Studentsrsquo negative opinions towards math and science 5 15Problem solving (PS) Studentsrsquo attitudes towards their problem solving abilities 4 12Technical skills (TS) Studentsrsquo attitudes towards their technical skills 4 12Engineer (E) Studentsrsquo understanding of the type of work engineers perform 5 15

R Hammack et al Journal of Pre-College Engineering Education Research 15

6httpdxdoiorg1077712157-92881102

What is Technology What is Engineering andEngineering Content

Table 3 presents the descriptive statistics and WilcoxonSigned Rank results for the What is Engineering What isTechnology and engineering content measures Pre- to post-test differences were significant for three measures (p 001)

Figure 3 illustrates the percentage of items selected ascorrect on the What is Technology pre and post measuresAn examination of these percentages indicated that manyparticipants associated technology with items that wereelectronic (100 selected TV and phones) or involved theuse of electricity (100 selected subways 95 selectedpower lines) and none of the participants associated techno-logy with living organisms (0 selected bird tree ordandelions) Fewer participants associated technology withnon-electrical manmade items however a greater percentageof participants selected items that may be associated withconstruction (58 selected bridge 63 selected house 84selected factory) than items not generally associated withconstruction (47 selected shoes cups and bandages 52selected books and bikes) On the What is Technology Post-test however participants clearly associated technology withitems that do not exist in nature (95 selected bandages100 selected shoes subway phone bridge TV cupfactory house power lines bicycle books)

Participantsrsquo responses to the What is Engineeringinstrument are displayed in Figure 4 An examination ofstudent responses indicates that seven of the nine itemsassociated with activities of an engineer (marked with anasterisk in Figure 4) increased at the conclusion of theengineering camp These items include verbs such as

improve develop design and create The other two items (ieDesign clean water figure how to track luggage) showed nochange in the number of responses however more than half ofthe students correctly selected these items Of the seven itemsnot associated with activities of an engineer fewer studentsselected that engineers construct buildings sell food and cleanteeth at the conclusion of camp There was an increase in thenumber of students who incorrectly selected driving machinesand repairing cars as acts of an engineer and over half stillchose that an engineer performs the act of installing wiringSurprisingly two participants held on to the misconceptionthat the work of an engineer includes arranging flowers

Open-Ended ResponsesThis section present results from the qualitative analysis

of an open-ended response from three data sources(1) What is an Engineer (2) What is Technology and(3) Engineering Content Test The percent occurrence ofemergent themes from each data source is presentedin Figures 5 6 and 7 On the What is Technology instrumentstudentsrsquo initial responses to lsquolsquoHow do you know something istechnologyrsquorsquo were categorized as electrical manmade orhelps people However at the end of the engineering camp allstudentsrsquo responses indicated an understanding that technologywas something that is manmade Additionally a large percen-tage of studentsrsquo initial responses to the prompt lsquolsquoWhat is anengineerrsquorsquo were categorized as someone who builds designsor improves things to help people On the post-test howeverthe percentage of responses falling within these categoriesdeclined greatly and a new theme (mass production) wasidentified in 68 of responses Studentsrsquo Engineering ContentTest responses indicated growth in their understanding of

Table 2Summary statistics for participantsrsquo (n519) scores on Attitudes Towards Mathematics Science and Engineering Scale (AMSE)

Subscale

Pretest Post-test

Z pMin Max Mdn Min Max Mdn

AMSE-Overall 81 139 111 78 140 114 20829 0407ISA 13 32 22 9 31 24 20156 0876INSA 5 18 10 4 19 10 20604 0546PO 5 23 17 8 23 16 21225 0220NO 5 13 9 5 21 8 20192 0848PS 6 20 12 6 16 13 20287 0774TS 10 20 14 9 20 13 0528 0528E 14 22 18 16 23 19 0003 0003

p 001

Table 3Summary statistics for participantrsquos scores on What is Engineering and What is Technology and Engineering Content instruments

Instrument n

Pretest Post-test

Z PMin Max Mdn Min Max Mdn

Engineering 19 375 9375 625 50 100 75 23285 0001Technology 19 4375 100 75 100 100 100 22818 0005Content 19 0 57 14 29 86 57 23456 0001

p 001

16 R Hammack et al Journal of Pre-College Engineering Education Research

7httpdxdoiorg1077712157-92881102

chemical engineers Examples of studentsrsquo open-endedresponses on the Engineering Content Test are presentedin Table 4 When asked what chemical engineers doresponses of lsquolsquoI donrsquot knowrsquorsquo decreased from 26 to 5by the end of the camp Additionally analysis ofstudentsrsquo open-ended responses indicated a shift in theirunderstanding from thinking that chemical engineersonly mix and use chemicals to engaging in more complexwork (ie experimenting) with chemicals

Discussion

Overall the results indicate that participants improved their(1) understanding of technology (2) chemical engineeringcontent knowledge and (3) attitudes towards engineering Ofthe seven AMSE subscales tested only the responses on theEngineer subscale changed significantly This indicates thatover the course of the camp participantsrsquo attitudes towardsengineering became more positive with more participants

reporting that they know what an engineer does These resultsare consistent with the open-ended response questionsResponses to the open-ended lsquolsquoWhat is an Engineerrsquorsquo prompton the What is Engineering post-test indicate that partici-pants gained an understanding of engineers as people whowork in teams and mass produce products

The scores on the What is Technology instrument weresignificantly higher on the post-test indicating that the campintervention had a positive impact on the participantsrsquo under-standings of technology As evident in the frequency counts andopen-ended responses many participants began camp with themisconception that technology only included items that requiredelectricity These results are consistent with the findings ofCunningham et al (2005) who found that students in grades1ndash5 tend to associate technology with objects requiringelectricity By the end of camp all participants reported thattechnology referred to any object that was manmade

Scores on the Engineering Content Test were significantlyhigher on the post-test indicating that the intervention had

0

20

40

60

80

100

S

elec

ted

Item

Pre

Post

Figure 3 Frequency of participantsrsquo responses for items on the What is Technology instrument Items that are considered technology

0

20

40

60

80

100

S

elec

ted

Item

Pre

Post

Figure 4 Frequency of participantsrsquo responses for items on the What is Engineering instrument Items associated with engineering

R Hammack et al Journal of Pre-College Engineering Education Research 17

8httpdxdoiorg1077712157-92881102

a positive impact on participantsrsquo content knowledge relatedto chemical engineering Although the median post-testscore was over 40 points higher than the pretest medianit was still only 57 which is below the 70 proficiencyscore used by the school where the camp was held Thissuggests that while participants increased their understandingof chemical engineering they still did not master all of thecontent covered in camp The short duration of camp maynot have provided enough time for students to fully masterthe new concepts

The range in frequencies on the What is Engineeringpretest point to participantsrsquo uncertainties about the activitiesof an engineer There was still a large range in the post-testresults indicating that participants still had some degree ofuncertainty about the work of engineers However there weresome positive changes worth noting Prior to participating incamp 68 of participants viewed engineers as people whowork as a team whereas all campers circled lsquolsquowork as ateamrsquorsquo on the post instrument All of the camp activitiesrequired that participants work as a team to solve problemswhich may have accounted for this change Recognizing that

engineers are not solitary loners but in fact work in teams totackle important real world problems could help combat thecommon negative stereotypes previously reported (Cavallo2007 Miller et al 2006) and may prevent more adole-scents form abandoning the STEM pipeline Additionallyparticipantsrsquo Engineering Content Test responses also indi-cated a positive shift in their attitudes towards engineering as acareer For instance only 26 of camp participants reportedthat they had considered becoming a chemical engineer priorto camp whereas 63 of participants reported that campencouraged them to consider chemical engineering as a career

Although the findings indicate an increase in the number ofstudent considering a career in engineering they still heldmany misconceptions about the work of engineers It isimportant to note that the engineering camp did not explicitlyaddress all misconceptions presented on the What isEngineering instrument However the camp did discuss thatengineers focus their work on the design and creative aspectsof construction but are not the actual builders This highlights atrend in our findings that the participantsrsquo views of engineeringshifted away from engineers as mere builders and a movetowards an understanding that engineers design solutions toproblems This trend is mirrored in the open-ended responsesas nearly all participants recognized that lsquolsquobuildingrsquorsquo was notan act of engineering by the end of the camp The only twolsquolsquobuildingrsquorsquo responses on the post-test were

lsquolsquoAn engineer is a person who builds a lot of one thingrsquorsquolsquolsquoSomeone who builds lots of stuffrsquorsquoIt is important to note that one major focus of the candy

airplane activity was the role engineers play in industry andmass production and the mass production of chemicals wasdiscussed during both the slime and film canister rocketactivities It is therefore not surprising that over one-half of theparticipants referred to mass production in their post-testresponses The two participants who wrote lsquolsquobuildrsquorsquo in their post

0102030405060708090

100

r

espo

ndin

g

Theme

Pre

Post

Figure 5 Frequency of themes for What is Engineering instrument

0

20

40

60

80

100

120

Electrical Man made Helps people

r

espo

ndin

g

Theme

Pre

Post

Figure 6 Frequency of themes for What is Technology instrument

18 R Hammack et al Journal of Pre-College Engineering Education Research

9httpdxdoiorg1077712157-92881102

responses may also have been referring to mass productionrather than simply building or constructing an object

Interestingly there are conflicting findings between thequantitative and qualitative responses on the What is

Engineering instrument Although the number of imagesstudents selected of engineers designing and improvingproducts increased students qualitative responses related todesigning and improving products decreased Because the

0

10

20

30

40

50

60

r

espo

ndin

g

Theme

Pre

Post

Figure 7 Frequency of themes for Engineering Content Test responses

Table 4Examples of participantsrsquo open-ended responses by theme

Theme Example Pre-program Responses Example Post-program Responses

How do you know something is technologyElectrical It creates energy and has circuit boards and stuff NAMan made Man invents technology eg bicycle subway Something that is technology is a man-made objectHelps people If it was invented by people to help other people NA

What is an engineerMass production NA An engineer creates multiple copies of a productBuilds Someone who builds something Someone who builds lots of stuffDesigns Somebody who designs and builds stuff An engineer designs some product and fins a way

to mass produce itImproves thingshelps others Someone who researches and develops innovations

to make this world a better and easier placeSomeone who improves inventions

Fixes things An engineer is someone who fixes or creates things An engineer is a person that fixes things andworks as a team

Team work NA An engineer is a person that fixes things andworks as a team

Uses tools I think it is someone who builds or designsthings and works with tools and differentkinds of machinery

NA

Uses math and science People who do math and science NA

What does a chemical engineer doI donrsquot know I donrsquot really know I canrsquot describe itMixesUses chemicals Mixes chemicals Someone who works with mixing and learning

about chemicalsManufacturingdesigns new

products or chemicalsWorks with chemistry to make new materials Helps manufacture chemicals

Experiments with chemicals A chemical engineer experiments with different chemicalsto see if they do something useful or harmful

Experiments with different chemicals to seewhat they do

Works with food NA Works with food

R Hammack et al Journal of Pre-College Engineering Education Research 19

10httpdxdoiorg1077712157-92881102

candy airplane slime and rocket activities all dealt with massproduction it is possible that participants concentrated on theaspect of mass production during their open-ended responsesHowever half of participantsrsquo journal prompts about the workof a chemical engineer referred to some aspect of designing orcreating a product on the post-test Additionally there wasa decrease in the qualitative responses related to lsquolsquohelpingpeoplersquorsquo on the What is Technology and What isEngineering instruments The decrease in balanced bythe increase in lsquolsquomanmadersquorsquo responses on the What isTechnology instrument and lsquolsquomass productionrsquorsquo responseson the What is Engineering instrument

Overall the findings indicate that the participants gainedan understanding that engineers are involved with the massproduction of products as well as their design andimprovement As such the decrease in studentsrsquo responsesin these categories is balanced by the increase in thenumber of responses that indicated engineers as massproducers of items Follow-up interviews with participantswould have been valuable as a means of providingadditional information as to how participantsrsquo perceptionsof engineers changed as a result of camp

Limitations

Although the results indicate a positive impact onparticipants it is not clear which components of the camp(hands-on activities interaction with a professional engi-neer combination of factors) contributed to this changeFollow-up interviews could have helped identify thesefactors Campers self-selected into the program and mayhave already had an above average interest in STEM Thiscoupled with the small sample size limits the general-izability of the findings Results were also limited by theinstruments used The What is Engineering instrumentuses a specific definition of engineering that is notuniversally accepted Additionally the validity of thecontent test has not been established and the test has onlybeen used with the 19 study participants Because we onlycollected and analyzed data immediately following thesummer camp our study only addresses short-termimpacts Future research should be conducted to determinelong-term impacts of engineering outreach programs

Conclusion and Implications

In answering our research questions we found thatparticipation in an engineering summer camp had a positiveimpact on middle school studentsrsquo understandings of whattechnology is and what engineers do evidenced by both thequantitative and qualitative data The open-ended questionssupported and added detail to the quantitative findingsproviding more evidence than if either data set had beenused independently Participantsrsquo attitudes towards mathe-matics and science were not significantly impacted by the

summer camp However their attitudes towards engineer-ing were significantly impacted The short duration of thecamp limited the quantity of information that could bepresented to participants with the current intervention focusingprimarily on engineering Although measurable impacts wereaccomplished during the week long summer camp long terminterventions such as the infusion of engineering activitieswithin the school curriculum could allow for attitudinalchanges across multiple STEM disciplines

The partnership between practicing teachers and engineer-ing faculty was important to the success of the camp Althoughthe extent to which co-teaching impacted outcomes was notmeasured the diverse backgrounds of the camp facilitatorsallowed for the combination of pedagogical knowledge andengineering content expertise to create rich engineeringexperiences that were developmentally appropriate for middleschool participants These experiences positively impactedparticipantsrsquo attitudes towards engineering revealing thebenefits of collaborative efforts between K-12 educators andengineering professionals Additional research in this area isneeded

References

Achieve Inc (2013) Next generation science standards WashingtonDC Achieve Inc

Bischoff P J (2006) The role of knowledge structures in the ability ofpreservice elementary teachers to diagnose a childrsquos understanding ofmolecular kinetics Science Education 90(5) 936ndash951 doi101002sce20155

Cavallo A (2007) Draw-a-ScientistMystery box Science and Children45(3) 37ndash41 Retrieved from httpwwwnstaorgpublicationsbrowse_journalsaspxaction5issueampid51025053sc07_045_03

Cavanaugh S (2007) Science camp Just for the girls Academic camps are onthe rise across the country including ones to get adolescent girls excitedabout the exploration of science Education Week 26(45) 26ndash28

Cheville A amp Bunting C (2011) Engineering students for the 21stcentury Student development through the curriculum Advances inEngineering Education 2(4) 1ndash36 Retrieved from httpadvancesaseeorgwp-contentuploadsvol02issue04papersaee-vol02-issue04-p10pdf

Committee on K-12 Engineering Education (2009) In L Katehi GPearson amp M Feder (Eds) Engineering in K-12 educationunderstanding the status and improving the prospects WashingtonDC The National Academies Press

Cunningham C Lachapelle C amp Lindgren-Streicher A (2005) Assessingelementary schools studentsrsquo conceptions of engineering and technologyProceedings of the 2005 American Society for Engineering EducationAnnual Conference amp Exposition Portland OR

DeJarnette N K (2012) Americarsquos children Providing early exposure toSTEM (science technology engineering and math) initiativesEducation 133(1) 77ndash84 Retrieved from httpwwwprojectinnovationbizeducation_2006html

Elam M E Donham B L amp Soloman S R (2012) An engineeringsummer program for underrepresented students from rural schooldistricts Journal of STEM Education 13(2) 35ndash44 Retrieved fromhttpojsjstemorgindexphpjournal5JSTEMamppage5articleampop5viewamppath5B5D51619amppath5B5D51437

Emerson R M Fretz R I amp Shaw L L (1995) Writing EthnographicFieldnotes Chicago IL University of Chicago Press

20 R Hammack et al Journal of Pre-College Engineering Education Research

11httpdxdoiorg1077712157-92881102

of chemical engineering As such our understanding of theimpact of chemical engineering outreach programs onstudentsrsquo attitudes toward engineering is limited

Pedagogical Content Knowledge and Co-Teaching

Effective instruction requires both subject matter knowl-edge and pedagogical content knowledge (Shulman 1986)Pedagogical content knowledge can be defined as lsquolsquothe waysof representing and formulating the subject that make itcomprehensible to othersrsquorsquo (Shulman 1986 p 9) Teacherswith increased subject matter knowledge and pedagogicalcontent knowledge are better able to determine their studentsrsquounderstanding of a topic and make changes to instruction thatimprove student learning (Bischoff 2006)

Co-teaching can be used to allow individuals with expertisein different areas to work together to teach a course This iscommonly seen in K-12 classrooms between a generaleducation and special education teacher There are manyproposed benefits of co-teaching (for example smaller student-teacher ratio more feedback to students more individual orsmall group instruction) however little research exists on theeffectiveness of co-teaching at the K-12 level (Sweigart ampLandrum 2015) and there is an absence of literature related touniversity and K-12 teachers engaging in co-teaching

Methodology

This study utilized a repeated measures design in whichsurvey and content tests were given before and afterparticipating in a summer engineering camp The quantitativesurvey data were used to test the hypothesis that participationin the camp positively impacted studentsrsquo attitudes towardsmathematics science and engineering Additionally open-ended questions were used to further explore studentsrsquo under-standings of engineering and technology Both quantitativeand qualitative data were used to compare results and bringgreater insight into the problem than by using a single method

Setting and Participants

Participants included 19 students (female54 male515white518 multi-racial51 special needs51) who had justcompleted sixth grade at the middle school where the campwas held The middle school was located in a Midwesterntown of approximately 45000 people and served approxi-mately 750 sixth- and seventh-grade students at the time ofthis study The camp was open to all students enrolled at themiddle school There was a small fee to attend the camp butscholarships were available to those in need to ensure thatsocioeconomic status did not prevent participation

Intervention

The summer engineering camp met for 35 hours eachday for four days Two middle school science teachers and

a chemical engineering professor from a local universityfacilitated the camp The middle school science teachersboth held at least a bachelorrsquos degree in a science contentarea The chemical engineering professor selected theactivities and assisted the teachers in the implementationEach of the activities required students to implementengineering design and introduced students to the types ofactivities engineers perform There was a large emphasisplaced on chemical engineering owing to the experiences ofthe professor in that area The students completed thefollowing activities during the camp

N Day 1 Students completed preassessments and thenparticipated in a brief measurement practice activity toensure that they could use graduated cylinders formeasuring volume and digital scales for measuringmass Next the chemical engineer lead a discussionon the similarities and differences between scientistsand engineers Students then completed a candyairplane activity in which they built a model of anairplane using only the provided supplies (forexample candy toothpicks paperclips rubber bands)within a limited amount of time After building themodel the students mass-produced the design Thechemical engineer led a discussion over product andprocess design using the information in Figure 1(Cheville amp Bunting 2011) The importance of processdesign in chemical engineering was discussed One ofthe major features of process design that was discussedwas its use with the mass production of products

N Day 2 Students completed a popcorn challenge inwhich they built a filter for a movie theater that couldseparate popped and unpopped kernels and allow theunpopped kernels to be recycled The chemicalengineer led a discussion about chemical processesand recycling using the information in Figure 2 Therewas a discussion of how the popcorn flowchartcorresponds with a chemical flowchart that would beused by a chemical engineer After completing thepopcorn challenge students were introduced to a slimeactivity During this activity students made slimeobserved its characteristics and then created a newproduct that utilized the slime Students also created anadvertisement for their product This activity was usedto demonstrate the differences between scientists(chemists) and engineers (chemical engineers) On thisday students made a single batch of slime much as achemist would do when identifying the formulation ofa new chemical substance

N Day 3 Students finished the slime activity by workingas chemical engineers to design package and marketa product that made use of the slime We alsodiscussed the role that chemical engineers play in themass production of chemicals that are required to beused in certain products such as pharmaceuticals andpetrochemical

R Hammack et al Journal of Pre-College Engineering Education Research 13

4httpdxdoiorg1077712157-92881102

N Day 4 Students experimented with materials todetermine which reactants would make the bestpropellants for film canister rockets Students designeda rocket body to attach to their film canisters and testedtheir chosen propellants A chemical engineerrsquos role inthe development and testing of rocket propellants wasdiscussed as well as the importance of mass productionof propellants

Prior to each activity students were presented with aproblem scenario Then they had to complete a designchallenge to address the problem For example prior tothe candy airplane activity students were told that anaircraft manufacturer was planning to expand its opera-tions and was looking for a new airplane model and manu-facturing facility Students then designed a model aircraftto present to the manufacturer and then mass-producedtheir design Each scenario provided students with a realworld context in which engineers would work Uponcompletion of each activity facilitators discussed howstudents engaged in engineering design throughout theactivities they created and reiterated real world applica-tions of the concepts

Instruments

Researchers used a variety of instruments in order tomeasure the impact of the program on the participantsrsquoattitudes toward mathematics science and engineeringconceptions of technology and conceptions of engineeringThese measures are described in detail below

Attitudes to Mathematics Science and Engineering(AMSE) The AMSE is a 5-point Likert-scaled instrumentconsisting of 36 items that are categorized into 8 subscalesTable 1 provides a list and description of each subscaleincluding the range of scores Scores on the overall AMSEcan range from 36 to 180 with higher scores being indicativeof a more positive attitude toward mathematics science andengineering (Hirsch et al 2007) Illustrative items includelsquolsquoEngineers are just people who do a lot of math and sciencersquorsquoand lsquolsquoEngineers help make peoplersquos lives betterrsquorsquo

What is Technology Cunningham and colleagues(2005) developed the What is Technology instrument toassess elementary studentsrsquo conceptions of technology Theinstrument consists of images and names of 16 items andone open-ended item How do you know if something istechnology Participants circle the images they associatewith technology and complete the open-ended responseExample items are shoes cellular phone and bird Scoresprovide data related to participantsrsquo ability to correctlyidentify examples of technology which can help research-ers identify misconceptions held by participants

What is Engineering Cunningham and colleagues(2005) also developed the What is Engineering Instrumentwhich consists of 16 images and descriptions of people atwork and one open ended response What is an engineer Forthis instrument participants circle the images with which theyassociate the work engineers perform as a part of their jobsand complete the open-ended response question Illustrativeitems include improve bandages drive machines and work as

Figure 1 Engineering design process used throughout the summer camp

Figure 2 Flowchart used to explain the process in the popcorn activity

14 R Hammack et al Journal of Pre-College Engineering Education Research

5httpdxdoiorg1077712157-92881102

a team Scores provide data related to participantsrsquo ability tocorrectly identify tasks completed by engineers as they workwhich can help researchers identify misconceptions heldabout engineers

Engineering Content Test On the first day of the campparticipants were given a precamp content test developedby the chemical engineering faculty member The surveycontained questions about content specific to chemicalengineering confidences in participating in engineeringdesign and the following journal prompts lsquolsquoWhat does achemical engineer dorsquorsquo and lsquolsquoHave you thought aboutbecoming an engineerrsquorsquo On the last day of camp participantscompleted a post-camp content test and responded to thefollowing prompts lsquolsquoWhat does a chemical engineer dorsquorsquoand lsquolsquoDid camp encourage you to consider engineeringrsquorsquo

Data Collection and Analysis

Camp facilitators pretested all participants at thebeginning of the first day of camp and post-tested on thelast day of camp All students completed the AMSE Whatis an Engineer and What is Technology instruments andEngineering Content test

Quantitative Data Analysis Owing to the smallsample size and lack of a normal distribution of dataresearchers used nonparametric statistics for data analy-sis A Wilcoxon Signed Rank test was performed inSPSS version 210 for the content test score overallAMSE score and all AMSE subscales except thelsquolsquootherrsquorsquo subscale Individualsrsquo subscale scores weredetermined by summing all of the item responses for aparticular subscale Table 1 presents the possible rangesfor an individualrsquos subscale scores On the presurveytwo participants left one item blank from the negativeopinions of mathematics and science subscale On thepost-survey two participants left one item blank onefrom the negative opinions of mathematics and sciencesubscale and one from the interest in engineeringstereotypic aspects subscale To account for eachparticipantrsquos missing value the mean for each indivi-dualrsquos subscales were imputed for the missing item

For example items 4 12 13 21 and 29 comprise thenegative opinions of mathematics and science subscaleParticipant A left item 21 blank on the pretest Hisscores from items 4 12 13 and 29 were averaged andthe resulting value was assigned to item 21

Each participantrsquos What is Technology and What isEngineering responses were scored based on the percen-tage of correct images that were circled Pre and post-testpercentages were entered into SPSS and a WilcoxonSigned Rank analysis was performed Researchers thenexamined the frequencies to identify trends

Each participantrsquos Engineering Content Test was given ascore based on the percentage of items that were correctlyanswered Pre- and post-test percentages were entered intoSPSS and a Wilcoxon Signed Rank analysis was performed

Qualitative Data Analysis The open-ended responseitems on the What is Technology and What is Engineeringinstruments as well as the lsquolsquoWhat does a chemicalengineer dorsquorsquo question were printed onto cards andexamined independently by three researchers to developa coding scheme for the responses The researchersdiscussed the three coding schemes and consolidatedthem into a single coding scheme The cards wereindependently recoded using the consolidated codingscheme There was 95 agreement between the resultingcodes the three researchers discussed the differencesuntil 100 consensus was reached (Emerson Fretz ampShaw 1995)

Results

Attitudes to Mathematics Science and Engineering

Researchers calculated a Cronbachrsquos alpha (086) todetermine the internal consistency for the AMSE instru-ment this value was consistent with the value reported(085) by Hirsch et al (2007) Table 2 presents thedescriptive statistics and Wilcoxon Signed Rank testresults for the seven AMSE subscales Only the Engineersubscale showed a significant difference pre to post(Z 5 0003 p 001)

Table 1Attitudes Toward Mathematics Science and Engineering (AMSE) subscales descriptions and score ranges

Subscale

Subscale ranges

Description Min Max

Overall AMSE 36 180Interest in engineering stereotypic aspects (ISA) Studentsrsquo interest in engineering based on common stereotypes 7 21Interest in engineering non-stereotypic aspects (INSA) Studentsrsquo interest in engineering not based on common stereotypes 4 12Positive opinions of mathematics and science (PO) Studentsrsquo positive opinions towards math and science 5 15Negative opinions of mathematics and science (NO) Studentsrsquo negative opinions towards math and science 5 15Problem solving (PS) Studentsrsquo attitudes towards their problem solving abilities 4 12Technical skills (TS) Studentsrsquo attitudes towards their technical skills 4 12Engineer (E) Studentsrsquo understanding of the type of work engineers perform 5 15

R Hammack et al Journal of Pre-College Engineering Education Research 15

6httpdxdoiorg1077712157-92881102

What is Technology What is Engineering andEngineering Content

Table 3 presents the descriptive statistics and WilcoxonSigned Rank results for the What is Engineering What isTechnology and engineering content measures Pre- to post-test differences were significant for three measures (p 001)

Figure 3 illustrates the percentage of items selected ascorrect on the What is Technology pre and post measuresAn examination of these percentages indicated that manyparticipants associated technology with items that wereelectronic (100 selected TV and phones) or involved theuse of electricity (100 selected subways 95 selectedpower lines) and none of the participants associated techno-logy with living organisms (0 selected bird tree ordandelions) Fewer participants associated technology withnon-electrical manmade items however a greater percentageof participants selected items that may be associated withconstruction (58 selected bridge 63 selected house 84selected factory) than items not generally associated withconstruction (47 selected shoes cups and bandages 52selected books and bikes) On the What is Technology Post-test however participants clearly associated technology withitems that do not exist in nature (95 selected bandages100 selected shoes subway phone bridge TV cupfactory house power lines bicycle books)

Participantsrsquo responses to the What is Engineeringinstrument are displayed in Figure 4 An examination ofstudent responses indicates that seven of the nine itemsassociated with activities of an engineer (marked with anasterisk in Figure 4) increased at the conclusion of theengineering camp These items include verbs such as

improve develop design and create The other two items (ieDesign clean water figure how to track luggage) showed nochange in the number of responses however more than half ofthe students correctly selected these items Of the seven itemsnot associated with activities of an engineer fewer studentsselected that engineers construct buildings sell food and cleanteeth at the conclusion of camp There was an increase in thenumber of students who incorrectly selected driving machinesand repairing cars as acts of an engineer and over half stillchose that an engineer performs the act of installing wiringSurprisingly two participants held on to the misconceptionthat the work of an engineer includes arranging flowers

Open-Ended ResponsesThis section present results from the qualitative analysis

of an open-ended response from three data sources(1) What is an Engineer (2) What is Technology and(3) Engineering Content Test The percent occurrence ofemergent themes from each data source is presentedin Figures 5 6 and 7 On the What is Technology instrumentstudentsrsquo initial responses to lsquolsquoHow do you know something istechnologyrsquorsquo were categorized as electrical manmade orhelps people However at the end of the engineering camp allstudentsrsquo responses indicated an understanding that technologywas something that is manmade Additionally a large percen-tage of studentsrsquo initial responses to the prompt lsquolsquoWhat is anengineerrsquorsquo were categorized as someone who builds designsor improves things to help people On the post-test howeverthe percentage of responses falling within these categoriesdeclined greatly and a new theme (mass production) wasidentified in 68 of responses Studentsrsquo Engineering ContentTest responses indicated growth in their understanding of

Table 2Summary statistics for participantsrsquo (n519) scores on Attitudes Towards Mathematics Science and Engineering Scale (AMSE)

Subscale

Pretest Post-test

Z pMin Max Mdn Min Max Mdn

AMSE-Overall 81 139 111 78 140 114 20829 0407ISA 13 32 22 9 31 24 20156 0876INSA 5 18 10 4 19 10 20604 0546PO 5 23 17 8 23 16 21225 0220NO 5 13 9 5 21 8 20192 0848PS 6 20 12 6 16 13 20287 0774TS 10 20 14 9 20 13 0528 0528E 14 22 18 16 23 19 0003 0003

p 001

Table 3Summary statistics for participantrsquos scores on What is Engineering and What is Technology and Engineering Content instruments

Instrument n

Pretest Post-test

Z PMin Max Mdn Min Max Mdn

Engineering 19 375 9375 625 50 100 75 23285 0001Technology 19 4375 100 75 100 100 100 22818 0005Content 19 0 57 14 29 86 57 23456 0001

p 001

16 R Hammack et al Journal of Pre-College Engineering Education Research

7httpdxdoiorg1077712157-92881102

chemical engineers Examples of studentsrsquo open-endedresponses on the Engineering Content Test are presentedin Table 4 When asked what chemical engineers doresponses of lsquolsquoI donrsquot knowrsquorsquo decreased from 26 to 5by the end of the camp Additionally analysis ofstudentsrsquo open-ended responses indicated a shift in theirunderstanding from thinking that chemical engineersonly mix and use chemicals to engaging in more complexwork (ie experimenting) with chemicals

Discussion

Overall the results indicate that participants improved their(1) understanding of technology (2) chemical engineeringcontent knowledge and (3) attitudes towards engineering Ofthe seven AMSE subscales tested only the responses on theEngineer subscale changed significantly This indicates thatover the course of the camp participantsrsquo attitudes towardsengineering became more positive with more participants

reporting that they know what an engineer does These resultsare consistent with the open-ended response questionsResponses to the open-ended lsquolsquoWhat is an Engineerrsquorsquo prompton the What is Engineering post-test indicate that partici-pants gained an understanding of engineers as people whowork in teams and mass produce products

The scores on the What is Technology instrument weresignificantly higher on the post-test indicating that the campintervention had a positive impact on the participantsrsquo under-standings of technology As evident in the frequency counts andopen-ended responses many participants began camp with themisconception that technology only included items that requiredelectricity These results are consistent with the findings ofCunningham et al (2005) who found that students in grades1ndash5 tend to associate technology with objects requiringelectricity By the end of camp all participants reported thattechnology referred to any object that was manmade

Scores on the Engineering Content Test were significantlyhigher on the post-test indicating that the intervention had

0

20

40

60

80

100

S

elec

ted

Item

Pre

Post

Figure 3 Frequency of participantsrsquo responses for items on the What is Technology instrument Items that are considered technology

0

20

40

60

80

100

S

elec

ted

Item

Pre

Post

Figure 4 Frequency of participantsrsquo responses for items on the What is Engineering instrument Items associated with engineering

R Hammack et al Journal of Pre-College Engineering Education Research 17

8httpdxdoiorg1077712157-92881102

a positive impact on participantsrsquo content knowledge relatedto chemical engineering Although the median post-testscore was over 40 points higher than the pretest medianit was still only 57 which is below the 70 proficiencyscore used by the school where the camp was held Thissuggests that while participants increased their understandingof chemical engineering they still did not master all of thecontent covered in camp The short duration of camp maynot have provided enough time for students to fully masterthe new concepts

The range in frequencies on the What is Engineeringpretest point to participantsrsquo uncertainties about the activitiesof an engineer There was still a large range in the post-testresults indicating that participants still had some degree ofuncertainty about the work of engineers However there weresome positive changes worth noting Prior to participating incamp 68 of participants viewed engineers as people whowork as a team whereas all campers circled lsquolsquowork as ateamrsquorsquo on the post instrument All of the camp activitiesrequired that participants work as a team to solve problemswhich may have accounted for this change Recognizing that

engineers are not solitary loners but in fact work in teams totackle important real world problems could help combat thecommon negative stereotypes previously reported (Cavallo2007 Miller et al 2006) and may prevent more adole-scents form abandoning the STEM pipeline Additionallyparticipantsrsquo Engineering Content Test responses also indi-cated a positive shift in their attitudes towards engineering as acareer For instance only 26 of camp participants reportedthat they had considered becoming a chemical engineer priorto camp whereas 63 of participants reported that campencouraged them to consider chemical engineering as a career

Although the findings indicate an increase in the number ofstudent considering a career in engineering they still heldmany misconceptions about the work of engineers It isimportant to note that the engineering camp did not explicitlyaddress all misconceptions presented on the What isEngineering instrument However the camp did discuss thatengineers focus their work on the design and creative aspectsof construction but are not the actual builders This highlights atrend in our findings that the participantsrsquo views of engineeringshifted away from engineers as mere builders and a movetowards an understanding that engineers design solutions toproblems This trend is mirrored in the open-ended responsesas nearly all participants recognized that lsquolsquobuildingrsquorsquo was notan act of engineering by the end of the camp The only twolsquolsquobuildingrsquorsquo responses on the post-test were

lsquolsquoAn engineer is a person who builds a lot of one thingrsquorsquolsquolsquoSomeone who builds lots of stuffrsquorsquoIt is important to note that one major focus of the candy

airplane activity was the role engineers play in industry andmass production and the mass production of chemicals wasdiscussed during both the slime and film canister rocketactivities It is therefore not surprising that over one-half of theparticipants referred to mass production in their post-testresponses The two participants who wrote lsquolsquobuildrsquorsquo in their post

0102030405060708090

100

r

espo

ndin

g

Theme

Pre

Post

Figure 5 Frequency of themes for What is Engineering instrument

0

20

40

60

80

100

120

Electrical Man made Helps people

r

espo

ndin

g

Theme

Pre

Post

Figure 6 Frequency of themes for What is Technology instrument

18 R Hammack et al Journal of Pre-College Engineering Education Research

9httpdxdoiorg1077712157-92881102

responses may also have been referring to mass productionrather than simply building or constructing an object

Interestingly there are conflicting findings between thequantitative and qualitative responses on the What is

Engineering instrument Although the number of imagesstudents selected of engineers designing and improvingproducts increased students qualitative responses related todesigning and improving products decreased Because the

0

10

20

30

40

50

60

r

espo

ndin

g

Theme

Pre

Post

Figure 7 Frequency of themes for Engineering Content Test responses

Table 4Examples of participantsrsquo open-ended responses by theme

Theme Example Pre-program Responses Example Post-program Responses

How do you know something is technologyElectrical It creates energy and has circuit boards and stuff NAMan made Man invents technology eg bicycle subway Something that is technology is a man-made objectHelps people If it was invented by people to help other people NA

What is an engineerMass production NA An engineer creates multiple copies of a productBuilds Someone who builds something Someone who builds lots of stuffDesigns Somebody who designs and builds stuff An engineer designs some product and fins a way

to mass produce itImproves thingshelps others Someone who researches and develops innovations

to make this world a better and easier placeSomeone who improves inventions

Fixes things An engineer is someone who fixes or creates things An engineer is a person that fixes things andworks as a team

Team work NA An engineer is a person that fixes things andworks as a team

Uses tools I think it is someone who builds or designsthings and works with tools and differentkinds of machinery

NA

Uses math and science People who do math and science NA

What does a chemical engineer doI donrsquot know I donrsquot really know I canrsquot describe itMixesUses chemicals Mixes chemicals Someone who works with mixing and learning

about chemicalsManufacturingdesigns new

products or chemicalsWorks with chemistry to make new materials Helps manufacture chemicals

Experiments with chemicals A chemical engineer experiments with different chemicalsto see if they do something useful or harmful

Experiments with different chemicals to seewhat they do

Works with food NA Works with food

R Hammack et al Journal of Pre-College Engineering Education Research 19

10httpdxdoiorg1077712157-92881102

candy airplane slime and rocket activities all dealt with massproduction it is possible that participants concentrated on theaspect of mass production during their open-ended responsesHowever half of participantsrsquo journal prompts about the workof a chemical engineer referred to some aspect of designing orcreating a product on the post-test Additionally there wasa decrease in the qualitative responses related to lsquolsquohelpingpeoplersquorsquo on the What is Technology and What isEngineering instruments The decrease in balanced bythe increase in lsquolsquomanmadersquorsquo responses on the What isTechnology instrument and lsquolsquomass productionrsquorsquo responseson the What is Engineering instrument

Overall the findings indicate that the participants gainedan understanding that engineers are involved with the massproduction of products as well as their design andimprovement As such the decrease in studentsrsquo responsesin these categories is balanced by the increase in thenumber of responses that indicated engineers as massproducers of items Follow-up interviews with participantswould have been valuable as a means of providingadditional information as to how participantsrsquo perceptionsof engineers changed as a result of camp

Limitations

Although the results indicate a positive impact onparticipants it is not clear which components of the camp(hands-on activities interaction with a professional engi-neer combination of factors) contributed to this changeFollow-up interviews could have helped identify thesefactors Campers self-selected into the program and mayhave already had an above average interest in STEM Thiscoupled with the small sample size limits the general-izability of the findings Results were also limited by theinstruments used The What is Engineering instrumentuses a specific definition of engineering that is notuniversally accepted Additionally the validity of thecontent test has not been established and the test has onlybeen used with the 19 study participants Because we onlycollected and analyzed data immediately following thesummer camp our study only addresses short-termimpacts Future research should be conducted to determinelong-term impacts of engineering outreach programs

Conclusion and Implications

In answering our research questions we found thatparticipation in an engineering summer camp had a positiveimpact on middle school studentsrsquo understandings of whattechnology is and what engineers do evidenced by both thequantitative and qualitative data The open-ended questionssupported and added detail to the quantitative findingsproviding more evidence than if either data set had beenused independently Participantsrsquo attitudes towards mathe-matics and science were not significantly impacted by the

summer camp However their attitudes towards engineer-ing were significantly impacted The short duration of thecamp limited the quantity of information that could bepresented to participants with the current intervention focusingprimarily on engineering Although measurable impacts wereaccomplished during the week long summer camp long terminterventions such as the infusion of engineering activitieswithin the school curriculum could allow for attitudinalchanges across multiple STEM disciplines

The partnership between practicing teachers and engineer-ing faculty was important to the success of the camp Althoughthe extent to which co-teaching impacted outcomes was notmeasured the diverse backgrounds of the camp facilitatorsallowed for the combination of pedagogical knowledge andengineering content expertise to create rich engineeringexperiences that were developmentally appropriate for middleschool participants These experiences positively impactedparticipantsrsquo attitudes towards engineering revealing thebenefits of collaborative efforts between K-12 educators andengineering professionals Additional research in this area isneeded

References

Achieve Inc (2013) Next generation science standards WashingtonDC Achieve Inc

Bischoff P J (2006) The role of knowledge structures in the ability ofpreservice elementary teachers to diagnose a childrsquos understanding ofmolecular kinetics Science Education 90(5) 936ndash951 doi101002sce20155

Cavallo A (2007) Draw-a-ScientistMystery box Science and Children45(3) 37ndash41 Retrieved from httpwwwnstaorgpublicationsbrowse_journalsaspxaction5issueampid51025053sc07_045_03

Cavanaugh S (2007) Science camp Just for the girls Academic camps are onthe rise across the country including ones to get adolescent girls excitedabout the exploration of science Education Week 26(45) 26ndash28

Cheville A amp Bunting C (2011) Engineering students for the 21stcentury Student development through the curriculum Advances inEngineering Education 2(4) 1ndash36 Retrieved from httpadvancesaseeorgwp-contentuploadsvol02issue04papersaee-vol02-issue04-p10pdf

Committee on K-12 Engineering Education (2009) In L Katehi GPearson amp M Feder (Eds) Engineering in K-12 educationunderstanding the status and improving the prospects WashingtonDC The National Academies Press

Cunningham C Lachapelle C amp Lindgren-Streicher A (2005) Assessingelementary schools studentsrsquo conceptions of engineering and technologyProceedings of the 2005 American Society for Engineering EducationAnnual Conference amp Exposition Portland OR

DeJarnette N K (2012) Americarsquos children Providing early exposure toSTEM (science technology engineering and math) initiativesEducation 133(1) 77ndash84 Retrieved from httpwwwprojectinnovationbizeducation_2006html

Elam M E Donham B L amp Soloman S R (2012) An engineeringsummer program for underrepresented students from rural schooldistricts Journal of STEM Education 13(2) 35ndash44 Retrieved fromhttpojsjstemorgindexphpjournal5JSTEMamppage5articleampop5viewamppath5B5D51619amppath5B5D51437

Emerson R M Fretz R I amp Shaw L L (1995) Writing EthnographicFieldnotes Chicago IL University of Chicago Press

20 R Hammack et al Journal of Pre-College Engineering Education Research

11httpdxdoiorg1077712157-92881102

N Day 4 Students experimented with materials todetermine which reactants would make the bestpropellants for film canister rockets Students designeda rocket body to attach to their film canisters and testedtheir chosen propellants A chemical engineerrsquos role inthe development and testing of rocket propellants wasdiscussed as well as the importance of mass productionof propellants

Prior to each activity students were presented with aproblem scenario Then they had to complete a designchallenge to address the problem For example prior tothe candy airplane activity students were told that anaircraft manufacturer was planning to expand its opera-tions and was looking for a new airplane model and manu-facturing facility Students then designed a model aircraftto present to the manufacturer and then mass-producedtheir design Each scenario provided students with a realworld context in which engineers would work Uponcompletion of each activity facilitators discussed howstudents engaged in engineering design throughout theactivities they created and reiterated real world applica-tions of the concepts

Instruments

Researchers used a variety of instruments in order tomeasure the impact of the program on the participantsrsquoattitudes toward mathematics science and engineeringconceptions of technology and conceptions of engineeringThese measures are described in detail below

Attitudes to Mathematics Science and Engineering(AMSE) The AMSE is a 5-point Likert-scaled instrumentconsisting of 36 items that are categorized into 8 subscalesTable 1 provides a list and description of each subscaleincluding the range of scores Scores on the overall AMSEcan range from 36 to 180 with higher scores being indicativeof a more positive attitude toward mathematics science andengineering (Hirsch et al 2007) Illustrative items includelsquolsquoEngineers are just people who do a lot of math and sciencersquorsquoand lsquolsquoEngineers help make peoplersquos lives betterrsquorsquo

What is Technology Cunningham and colleagues(2005) developed the What is Technology instrument toassess elementary studentsrsquo conceptions of technology Theinstrument consists of images and names of 16 items andone open-ended item How do you know if something istechnology Participants circle the images they associatewith technology and complete the open-ended responseExample items are shoes cellular phone and bird Scoresprovide data related to participantsrsquo ability to correctlyidentify examples of technology which can help research-ers identify misconceptions held by participants

What is Engineering Cunningham and colleagues(2005) also developed the What is Engineering Instrumentwhich consists of 16 images and descriptions of people atwork and one open ended response What is an engineer Forthis instrument participants circle the images with which theyassociate the work engineers perform as a part of their jobsand complete the open-ended response question Illustrativeitems include improve bandages drive machines and work as

Figure 1 Engineering design process used throughout the summer camp

Figure 2 Flowchart used to explain the process in the popcorn activity

14 R Hammack et al Journal of Pre-College Engineering Education Research

5httpdxdoiorg1077712157-92881102

a team Scores provide data related to participantsrsquo ability tocorrectly identify tasks completed by engineers as they workwhich can help researchers identify misconceptions heldabout engineers

Engineering Content Test On the first day of the campparticipants were given a precamp content test developedby the chemical engineering faculty member The surveycontained questions about content specific to chemicalengineering confidences in participating in engineeringdesign and the following journal prompts lsquolsquoWhat does achemical engineer dorsquorsquo and lsquolsquoHave you thought aboutbecoming an engineerrsquorsquo On the last day of camp participantscompleted a post-camp content test and responded to thefollowing prompts lsquolsquoWhat does a chemical engineer dorsquorsquoand lsquolsquoDid camp encourage you to consider engineeringrsquorsquo

Data Collection and Analysis

Camp facilitators pretested all participants at thebeginning of the first day of camp and post-tested on thelast day of camp All students completed the AMSE Whatis an Engineer and What is Technology instruments andEngineering Content test

Quantitative Data Analysis Owing to the smallsample size and lack of a normal distribution of dataresearchers used nonparametric statistics for data analy-sis A Wilcoxon Signed Rank test was performed inSPSS version 210 for the content test score overallAMSE score and all AMSE subscales except thelsquolsquootherrsquorsquo subscale Individualsrsquo subscale scores weredetermined by summing all of the item responses for aparticular subscale Table 1 presents the possible rangesfor an individualrsquos subscale scores On the presurveytwo participants left one item blank from the negativeopinions of mathematics and science subscale On thepost-survey two participants left one item blank onefrom the negative opinions of mathematics and sciencesubscale and one from the interest in engineeringstereotypic aspects subscale To account for eachparticipantrsquos missing value the mean for each indivi-dualrsquos subscales were imputed for the missing item

For example items 4 12 13 21 and 29 comprise thenegative opinions of mathematics and science subscaleParticipant A left item 21 blank on the pretest Hisscores from items 4 12 13 and 29 were averaged andthe resulting value was assigned to item 21

Each participantrsquos What is Technology and What isEngineering responses were scored based on the percen-tage of correct images that were circled Pre and post-testpercentages were entered into SPSS and a WilcoxonSigned Rank analysis was performed Researchers thenexamined the frequencies to identify trends

Each participantrsquos Engineering Content Test was given ascore based on the percentage of items that were correctlyanswered Pre- and post-test percentages were entered intoSPSS and a Wilcoxon Signed Rank analysis was performed

Qualitative Data Analysis The open-ended responseitems on the What is Technology and What is Engineeringinstruments as well as the lsquolsquoWhat does a chemicalengineer dorsquorsquo question were printed onto cards andexamined independently by three researchers to developa coding scheme for the responses The researchersdiscussed the three coding schemes and consolidatedthem into a single coding scheme The cards wereindependently recoded using the consolidated codingscheme There was 95 agreement between the resultingcodes the three researchers discussed the differencesuntil 100 consensus was reached (Emerson Fretz ampShaw 1995)

Results

Attitudes to Mathematics Science and Engineering

Researchers calculated a Cronbachrsquos alpha (086) todetermine the internal consistency for the AMSE instru-ment this value was consistent with the value reported(085) by Hirsch et al (2007) Table 2 presents thedescriptive statistics and Wilcoxon Signed Rank testresults for the seven AMSE subscales Only the Engineersubscale showed a significant difference pre to post(Z 5 0003 p 001)

Table 1Attitudes Toward Mathematics Science and Engineering (AMSE) subscales descriptions and score ranges

Subscale

Subscale ranges

Description Min Max

Overall AMSE 36 180Interest in engineering stereotypic aspects (ISA) Studentsrsquo interest in engineering based on common stereotypes 7 21Interest in engineering non-stereotypic aspects (INSA) Studentsrsquo interest in engineering not based on common stereotypes 4 12Positive opinions of mathematics and science (PO) Studentsrsquo positive opinions towards math and science 5 15Negative opinions of mathematics and science (NO) Studentsrsquo negative opinions towards math and science 5 15Problem solving (PS) Studentsrsquo attitudes towards their problem solving abilities 4 12Technical skills (TS) Studentsrsquo attitudes towards their technical skills 4 12Engineer (E) Studentsrsquo understanding of the type of work engineers perform 5 15

R Hammack et al Journal of Pre-College Engineering Education Research 15

6httpdxdoiorg1077712157-92881102

What is Technology What is Engineering andEngineering Content

Table 3 presents the descriptive statistics and WilcoxonSigned Rank results for the What is Engineering What isTechnology and engineering content measures Pre- to post-test differences were significant for three measures (p 001)

Figure 3 illustrates the percentage of items selected ascorrect on the What is Technology pre and post measuresAn examination of these percentages indicated that manyparticipants associated technology with items that wereelectronic (100 selected TV and phones) or involved theuse of electricity (100 selected subways 95 selectedpower lines) and none of the participants associated techno-logy with living organisms (0 selected bird tree ordandelions) Fewer participants associated technology withnon-electrical manmade items however a greater percentageof participants selected items that may be associated withconstruction (58 selected bridge 63 selected house 84selected factory) than items not generally associated withconstruction (47 selected shoes cups and bandages 52selected books and bikes) On the What is Technology Post-test however participants clearly associated technology withitems that do not exist in nature (95 selected bandages100 selected shoes subway phone bridge TV cupfactory house power lines bicycle books)

Participantsrsquo responses to the What is Engineeringinstrument are displayed in Figure 4 An examination ofstudent responses indicates that seven of the nine itemsassociated with activities of an engineer (marked with anasterisk in Figure 4) increased at the conclusion of theengineering camp These items include verbs such as

improve develop design and create The other two items (ieDesign clean water figure how to track luggage) showed nochange in the number of responses however more than half ofthe students correctly selected these items Of the seven itemsnot associated with activities of an engineer fewer studentsselected that engineers construct buildings sell food and cleanteeth at the conclusion of camp There was an increase in thenumber of students who incorrectly selected driving machinesand repairing cars as acts of an engineer and over half stillchose that an engineer performs the act of installing wiringSurprisingly two participants held on to the misconceptionthat the work of an engineer includes arranging flowers

Open-Ended ResponsesThis section present results from the qualitative analysis

of an open-ended response from three data sources(1) What is an Engineer (2) What is Technology and(3) Engineering Content Test The percent occurrence ofemergent themes from each data source is presentedin Figures 5 6 and 7 On the What is Technology instrumentstudentsrsquo initial responses to lsquolsquoHow do you know something istechnologyrsquorsquo were categorized as electrical manmade orhelps people However at the end of the engineering camp allstudentsrsquo responses indicated an understanding that technologywas something that is manmade Additionally a large percen-tage of studentsrsquo initial responses to the prompt lsquolsquoWhat is anengineerrsquorsquo were categorized as someone who builds designsor improves things to help people On the post-test howeverthe percentage of responses falling within these categoriesdeclined greatly and a new theme (mass production) wasidentified in 68 of responses Studentsrsquo Engineering ContentTest responses indicated growth in their understanding of

Table 2Summary statistics for participantsrsquo (n519) scores on Attitudes Towards Mathematics Science and Engineering Scale (AMSE)

Subscale

Pretest Post-test

Z pMin Max Mdn Min Max Mdn

AMSE-Overall 81 139 111 78 140 114 20829 0407ISA 13 32 22 9 31 24 20156 0876INSA 5 18 10 4 19 10 20604 0546PO 5 23 17 8 23 16 21225 0220NO 5 13 9 5 21 8 20192 0848PS 6 20 12 6 16 13 20287 0774TS 10 20 14 9 20 13 0528 0528E 14 22 18 16 23 19 0003 0003

p 001

Table 3Summary statistics for participantrsquos scores on What is Engineering and What is Technology and Engineering Content instruments

Instrument n

Pretest Post-test

Z PMin Max Mdn Min Max Mdn

Engineering 19 375 9375 625 50 100 75 23285 0001Technology 19 4375 100 75 100 100 100 22818 0005Content 19 0 57 14 29 86 57 23456 0001

p 001

16 R Hammack et al Journal of Pre-College Engineering Education Research

7httpdxdoiorg1077712157-92881102

chemical engineers Examples of studentsrsquo open-endedresponses on the Engineering Content Test are presentedin Table 4 When asked what chemical engineers doresponses of lsquolsquoI donrsquot knowrsquorsquo decreased from 26 to 5by the end of the camp Additionally analysis ofstudentsrsquo open-ended responses indicated a shift in theirunderstanding from thinking that chemical engineersonly mix and use chemicals to engaging in more complexwork (ie experimenting) with chemicals

Discussion

Overall the results indicate that participants improved their(1) understanding of technology (2) chemical engineeringcontent knowledge and (3) attitudes towards engineering Ofthe seven AMSE subscales tested only the responses on theEngineer subscale changed significantly This indicates thatover the course of the camp participantsrsquo attitudes towardsengineering became more positive with more participants

reporting that they know what an engineer does These resultsare consistent with the open-ended response questionsResponses to the open-ended lsquolsquoWhat is an Engineerrsquorsquo prompton the What is Engineering post-test indicate that partici-pants gained an understanding of engineers as people whowork in teams and mass produce products

The scores on the What is Technology instrument weresignificantly higher on the post-test indicating that the campintervention had a positive impact on the participantsrsquo under-standings of technology As evident in the frequency counts andopen-ended responses many participants began camp with themisconception that technology only included items that requiredelectricity These results are consistent with the findings ofCunningham et al (2005) who found that students in grades1ndash5 tend to associate technology with objects requiringelectricity By the end of camp all participants reported thattechnology referred to any object that was manmade

Scores on the Engineering Content Test were significantlyhigher on the post-test indicating that the intervention had

0

20

40

60

80

100

S

elec

ted

Item

Pre

Post

Figure 3 Frequency of participantsrsquo responses for items on the What is Technology instrument Items that are considered technology

0

20

40

60

80

100

S

elec

ted

Item

Pre

Post

Figure 4 Frequency of participantsrsquo responses for items on the What is Engineering instrument Items associated with engineering

R Hammack et al Journal of Pre-College Engineering Education Research 17

8httpdxdoiorg1077712157-92881102

a positive impact on participantsrsquo content knowledge relatedto chemical engineering Although the median post-testscore was over 40 points higher than the pretest medianit was still only 57 which is below the 70 proficiencyscore used by the school where the camp was held Thissuggests that while participants increased their understandingof chemical engineering they still did not master all of thecontent covered in camp The short duration of camp maynot have provided enough time for students to fully masterthe new concepts

The range in frequencies on the What is Engineeringpretest point to participantsrsquo uncertainties about the activitiesof an engineer There was still a large range in the post-testresults indicating that participants still had some degree ofuncertainty about the work of engineers However there weresome positive changes worth noting Prior to participating incamp 68 of participants viewed engineers as people whowork as a team whereas all campers circled lsquolsquowork as ateamrsquorsquo on the post instrument All of the camp activitiesrequired that participants work as a team to solve problemswhich may have accounted for this change Recognizing that

engineers are not solitary loners but in fact work in teams totackle important real world problems could help combat thecommon negative stereotypes previously reported (Cavallo2007 Miller et al 2006) and may prevent more adole-scents form abandoning the STEM pipeline Additionallyparticipantsrsquo Engineering Content Test responses also indi-cated a positive shift in their attitudes towards engineering as acareer For instance only 26 of camp participants reportedthat they had considered becoming a chemical engineer priorto camp whereas 63 of participants reported that campencouraged them to consider chemical engineering as a career

Although the findings indicate an increase in the number ofstudent considering a career in engineering they still heldmany misconceptions about the work of engineers It isimportant to note that the engineering camp did not explicitlyaddress all misconceptions presented on the What isEngineering instrument However the camp did discuss thatengineers focus their work on the design and creative aspectsof construction but are not the actual builders This highlights atrend in our findings that the participantsrsquo views of engineeringshifted away from engineers as mere builders and a movetowards an understanding that engineers design solutions toproblems This trend is mirrored in the open-ended responsesas nearly all participants recognized that lsquolsquobuildingrsquorsquo was notan act of engineering by the end of the camp The only twolsquolsquobuildingrsquorsquo responses on the post-test were

lsquolsquoAn engineer is a person who builds a lot of one thingrsquorsquolsquolsquoSomeone who builds lots of stuffrsquorsquoIt is important to note that one major focus of the candy

airplane activity was the role engineers play in industry andmass production and the mass production of chemicals wasdiscussed during both the slime and film canister rocketactivities It is therefore not surprising that over one-half of theparticipants referred to mass production in their post-testresponses The two participants who wrote lsquolsquobuildrsquorsquo in their post

0102030405060708090

100

r

espo

ndin

g

Theme

Pre

Post

Figure 5 Frequency of themes for What is Engineering instrument

0

20

40

60

80

100

120

Electrical Man made Helps people

r

espo

ndin

g

Theme

Pre

Post

Figure 6 Frequency of themes for What is Technology instrument

18 R Hammack et al Journal of Pre-College Engineering Education Research

9httpdxdoiorg1077712157-92881102

responses may also have been referring to mass productionrather than simply building or constructing an object

Interestingly there are conflicting findings between thequantitative and qualitative responses on the What is

Engineering instrument Although the number of imagesstudents selected of engineers designing and improvingproducts increased students qualitative responses related todesigning and improving products decreased Because the

0

10

20

30

40

50

60

r

espo

ndin

g

Theme

Pre

Post

Figure 7 Frequency of themes for Engineering Content Test responses

Table 4Examples of participantsrsquo open-ended responses by theme

Theme Example Pre-program Responses Example Post-program Responses

How do you know something is technologyElectrical It creates energy and has circuit boards and stuff NAMan made Man invents technology eg bicycle subway Something that is technology is a man-made objectHelps people If it was invented by people to help other people NA

What is an engineerMass production NA An engineer creates multiple copies of a productBuilds Someone who builds something Someone who builds lots of stuffDesigns Somebody who designs and builds stuff An engineer designs some product and fins a way

to mass produce itImproves thingshelps others Someone who researches and develops innovations

to make this world a better and easier placeSomeone who improves inventions

Fixes things An engineer is someone who fixes or creates things An engineer is a person that fixes things andworks as a team

Team work NA An engineer is a person that fixes things andworks as a team

Uses tools I think it is someone who builds or designsthings and works with tools and differentkinds of machinery

NA

Uses math and science People who do math and science NA

What does a chemical engineer doI donrsquot know I donrsquot really know I canrsquot describe itMixesUses chemicals Mixes chemicals Someone who works with mixing and learning

about chemicalsManufacturingdesigns new

products or chemicalsWorks with chemistry to make new materials Helps manufacture chemicals

Experiments with chemicals A chemical engineer experiments with different chemicalsto see if they do something useful or harmful

Experiments with different chemicals to seewhat they do

Works with food NA Works with food

R Hammack et al Journal of Pre-College Engineering Education Research 19

10httpdxdoiorg1077712157-92881102

candy airplane slime and rocket activities all dealt with massproduction it is possible that participants concentrated on theaspect of mass production during their open-ended responsesHowever half of participantsrsquo journal prompts about the workof a chemical engineer referred to some aspect of designing orcreating a product on the post-test Additionally there wasa decrease in the qualitative responses related to lsquolsquohelpingpeoplersquorsquo on the What is Technology and What isEngineering instruments The decrease in balanced bythe increase in lsquolsquomanmadersquorsquo responses on the What isTechnology instrument and lsquolsquomass productionrsquorsquo responseson the What is Engineering instrument

Overall the findings indicate that the participants gainedan understanding that engineers are involved with the massproduction of products as well as their design andimprovement As such the decrease in studentsrsquo responsesin these categories is balanced by the increase in thenumber of responses that indicated engineers as massproducers of items Follow-up interviews with participantswould have been valuable as a means of providingadditional information as to how participantsrsquo perceptionsof engineers changed as a result of camp

Limitations

Although the results indicate a positive impact onparticipants it is not clear which components of the camp(hands-on activities interaction with a professional engi-neer combination of factors) contributed to this changeFollow-up interviews could have helped identify thesefactors Campers self-selected into the program and mayhave already had an above average interest in STEM Thiscoupled with the small sample size limits the general-izability of the findings Results were also limited by theinstruments used The What is Engineering instrumentuses a specific definition of engineering that is notuniversally accepted Additionally the validity of thecontent test has not been established and the test has onlybeen used with the 19 study participants Because we onlycollected and analyzed data immediately following thesummer camp our study only addresses short-termimpacts Future research should be conducted to determinelong-term impacts of engineering outreach programs

Conclusion and Implications

In answering our research questions we found thatparticipation in an engineering summer camp had a positiveimpact on middle school studentsrsquo understandings of whattechnology is and what engineers do evidenced by both thequantitative and qualitative data The open-ended questionssupported and added detail to the quantitative findingsproviding more evidence than if either data set had beenused independently Participantsrsquo attitudes towards mathe-matics and science were not significantly impacted by the

summer camp However their attitudes towards engineer-ing were significantly impacted The short duration of thecamp limited the quantity of information that could bepresented to participants with the current intervention focusingprimarily on engineering Although measurable impacts wereaccomplished during the week long summer camp long terminterventions such as the infusion of engineering activitieswithin the school curriculum could allow for attitudinalchanges across multiple STEM disciplines

The partnership between practicing teachers and engineer-ing faculty was important to the success of the camp Althoughthe extent to which co-teaching impacted outcomes was notmeasured the diverse backgrounds of the camp facilitatorsallowed for the combination of pedagogical knowledge andengineering content expertise to create rich engineeringexperiences that were developmentally appropriate for middleschool participants These experiences positively impactedparticipantsrsquo attitudes towards engineering revealing thebenefits of collaborative efforts between K-12 educators andengineering professionals Additional research in this area isneeded

References

Achieve Inc (2013) Next generation science standards WashingtonDC Achieve Inc

Bischoff P J (2006) The role of knowledge structures in the ability ofpreservice elementary teachers to diagnose a childrsquos understanding ofmolecular kinetics Science Education 90(5) 936ndash951 doi101002sce20155

Cavallo A (2007) Draw-a-ScientistMystery box Science and Children45(3) 37ndash41 Retrieved from httpwwwnstaorgpublicationsbrowse_journalsaspxaction5issueampid51025053sc07_045_03

Cavanaugh S (2007) Science camp Just for the girls Academic camps are onthe rise across the country including ones to get adolescent girls excitedabout the exploration of science Education Week 26(45) 26ndash28

Cheville A amp Bunting C (2011) Engineering students for the 21stcentury Student development through the curriculum Advances inEngineering Education 2(4) 1ndash36 Retrieved from httpadvancesaseeorgwp-contentuploadsvol02issue04papersaee-vol02-issue04-p10pdf

Committee on K-12 Engineering Education (2009) In L Katehi GPearson amp M Feder (Eds) Engineering in K-12 educationunderstanding the status and improving the prospects WashingtonDC The National Academies Press

Cunningham C Lachapelle C amp Lindgren-Streicher A (2005) Assessingelementary schools studentsrsquo conceptions of engineering and technologyProceedings of the 2005 American Society for Engineering EducationAnnual Conference amp Exposition Portland OR

DeJarnette N K (2012) Americarsquos children Providing early exposure toSTEM (science technology engineering and math) initiativesEducation 133(1) 77ndash84 Retrieved from httpwwwprojectinnovationbizeducation_2006html

Elam M E Donham B L amp Soloman S R (2012) An engineeringsummer program for underrepresented students from rural schooldistricts Journal of STEM Education 13(2) 35ndash44 Retrieved fromhttpojsjstemorgindexphpjournal5JSTEMamppage5articleampop5viewamppath5B5D51619amppath5B5D51437

Emerson R M Fretz R I amp Shaw L L (1995) Writing EthnographicFieldnotes Chicago IL University of Chicago Press

20 R Hammack et al Journal of Pre-College Engineering Education Research

11httpdxdoiorg1077712157-92881102

a team Scores provide data related to participantsrsquo ability tocorrectly identify tasks completed by engineers as they workwhich can help researchers identify misconceptions heldabout engineers

Engineering Content Test On the first day of the campparticipants were given a precamp content test developedby the chemical engineering faculty member The surveycontained questions about content specific to chemicalengineering confidences in participating in engineeringdesign and the following journal prompts lsquolsquoWhat does achemical engineer dorsquorsquo and lsquolsquoHave you thought aboutbecoming an engineerrsquorsquo On the last day of camp participantscompleted a post-camp content test and responded to thefollowing prompts lsquolsquoWhat does a chemical engineer dorsquorsquoand lsquolsquoDid camp encourage you to consider engineeringrsquorsquo

Data Collection and Analysis

Camp facilitators pretested all participants at thebeginning of the first day of camp and post-tested on thelast day of camp All students completed the AMSE Whatis an Engineer and What is Technology instruments andEngineering Content test

Quantitative Data Analysis Owing to the smallsample size and lack of a normal distribution of dataresearchers used nonparametric statistics for data analy-sis A Wilcoxon Signed Rank test was performed inSPSS version 210 for the content test score overallAMSE score and all AMSE subscales except thelsquolsquootherrsquorsquo subscale Individualsrsquo subscale scores weredetermined by summing all of the item responses for aparticular subscale Table 1 presents the possible rangesfor an individualrsquos subscale scores On the presurveytwo participants left one item blank from the negativeopinions of mathematics and science subscale On thepost-survey two participants left one item blank onefrom the negative opinions of mathematics and sciencesubscale and one from the interest in engineeringstereotypic aspects subscale To account for eachparticipantrsquos missing value the mean for each indivi-dualrsquos subscales were imputed for the missing item

For example items 4 12 13 21 and 29 comprise thenegative opinions of mathematics and science subscaleParticipant A left item 21 blank on the pretest Hisscores from items 4 12 13 and 29 were averaged andthe resulting value was assigned to item 21

Each participantrsquos What is Technology and What isEngineering responses were scored based on the percen-tage of correct images that were circled Pre and post-testpercentages were entered into SPSS and a WilcoxonSigned Rank analysis was performed Researchers thenexamined the frequencies to identify trends

Each participantrsquos Engineering Content Test was given ascore based on the percentage of items that were correctlyanswered Pre- and post-test percentages were entered intoSPSS and a Wilcoxon Signed Rank analysis was performed

Qualitative Data Analysis The open-ended responseitems on the What is Technology and What is Engineeringinstruments as well as the lsquolsquoWhat does a chemicalengineer dorsquorsquo question were printed onto cards andexamined independently by three researchers to developa coding scheme for the responses The researchersdiscussed the three coding schemes and consolidatedthem into a single coding scheme The cards wereindependently recoded using the consolidated codingscheme There was 95 agreement between the resultingcodes the three researchers discussed the differencesuntil 100 consensus was reached (Emerson Fretz ampShaw 1995)

Results

Attitudes to Mathematics Science and Engineering

Researchers calculated a Cronbachrsquos alpha (086) todetermine the internal consistency for the AMSE instru-ment this value was consistent with the value reported(085) by Hirsch et al (2007) Table 2 presents thedescriptive statistics and Wilcoxon Signed Rank testresults for the seven AMSE subscales Only the Engineersubscale showed a significant difference pre to post(Z 5 0003 p 001)

Table 1Attitudes Toward Mathematics Science and Engineering (AMSE) subscales descriptions and score ranges

Subscale

Subscale ranges

Description Min Max

Overall AMSE 36 180Interest in engineering stereotypic aspects (ISA) Studentsrsquo interest in engineering based on common stereotypes 7 21Interest in engineering non-stereotypic aspects (INSA) Studentsrsquo interest in engineering not based on common stereotypes 4 12Positive opinions of mathematics and science (PO) Studentsrsquo positive opinions towards math and science 5 15Negative opinions of mathematics and science (NO) Studentsrsquo negative opinions towards math and science 5 15Problem solving (PS) Studentsrsquo attitudes towards their problem solving abilities 4 12Technical skills (TS) Studentsrsquo attitudes towards their technical skills 4 12Engineer (E) Studentsrsquo understanding of the type of work engineers perform 5 15

R Hammack et al Journal of Pre-College Engineering Education Research 15

6httpdxdoiorg1077712157-92881102

What is Technology What is Engineering andEngineering Content

Table 3 presents the descriptive statistics and WilcoxonSigned Rank results for the What is Engineering What isTechnology and engineering content measures Pre- to post-test differences were significant for three measures (p 001)

Figure 3 illustrates the percentage of items selected ascorrect on the What is Technology pre and post measuresAn examination of these percentages indicated that manyparticipants associated technology with items that wereelectronic (100 selected TV and phones) or involved theuse of electricity (100 selected subways 95 selectedpower lines) and none of the participants associated techno-logy with living organisms (0 selected bird tree ordandelions) Fewer participants associated technology withnon-electrical manmade items however a greater percentageof participants selected items that may be associated withconstruction (58 selected bridge 63 selected house 84selected factory) than items not generally associated withconstruction (47 selected shoes cups and bandages 52selected books and bikes) On the What is Technology Post-test however participants clearly associated technology withitems that do not exist in nature (95 selected bandages100 selected shoes subway phone bridge TV cupfactory house power lines bicycle books)

Participantsrsquo responses to the What is Engineeringinstrument are displayed in Figure 4 An examination ofstudent responses indicates that seven of the nine itemsassociated with activities of an engineer (marked with anasterisk in Figure 4) increased at the conclusion of theengineering camp These items include verbs such as

improve develop design and create The other two items (ieDesign clean water figure how to track luggage) showed nochange in the number of responses however more than half ofthe students correctly selected these items Of the seven itemsnot associated with activities of an engineer fewer studentsselected that engineers construct buildings sell food and cleanteeth at the conclusion of camp There was an increase in thenumber of students who incorrectly selected driving machinesand repairing cars as acts of an engineer and over half stillchose that an engineer performs the act of installing wiringSurprisingly two participants held on to the misconceptionthat the work of an engineer includes arranging flowers

Open-Ended ResponsesThis section present results from the qualitative analysis

of an open-ended response from three data sources(1) What is an Engineer (2) What is Technology and(3) Engineering Content Test The percent occurrence ofemergent themes from each data source is presentedin Figures 5 6 and 7 On the What is Technology instrumentstudentsrsquo initial responses to lsquolsquoHow do you know something istechnologyrsquorsquo were categorized as electrical manmade orhelps people However at the end of the engineering camp allstudentsrsquo responses indicated an understanding that technologywas something that is manmade Additionally a large percen-tage of studentsrsquo initial responses to the prompt lsquolsquoWhat is anengineerrsquorsquo were categorized as someone who builds designsor improves things to help people On the post-test howeverthe percentage of responses falling within these categoriesdeclined greatly and a new theme (mass production) wasidentified in 68 of responses Studentsrsquo Engineering ContentTest responses indicated growth in their understanding of

Table 2Summary statistics for participantsrsquo (n519) scores on Attitudes Towards Mathematics Science and Engineering Scale (AMSE)

Subscale

Pretest Post-test

Z pMin Max Mdn Min Max Mdn

AMSE-Overall 81 139 111 78 140 114 20829 0407ISA 13 32 22 9 31 24 20156 0876INSA 5 18 10 4 19 10 20604 0546PO 5 23 17 8 23 16 21225 0220NO 5 13 9 5 21 8 20192 0848PS 6 20 12 6 16 13 20287 0774TS 10 20 14 9 20 13 0528 0528E 14 22 18 16 23 19 0003 0003

p 001

Table 3Summary statistics for participantrsquos scores on What is Engineering and What is Technology and Engineering Content instruments

Instrument n

Pretest Post-test

Z PMin Max Mdn Min Max Mdn

Engineering 19 375 9375 625 50 100 75 23285 0001Technology 19 4375 100 75 100 100 100 22818 0005Content 19 0 57 14 29 86 57 23456 0001

p 001

16 R Hammack et al Journal of Pre-College Engineering Education Research

7httpdxdoiorg1077712157-92881102

chemical engineers Examples of studentsrsquo open-endedresponses on the Engineering Content Test are presentedin Table 4 When asked what chemical engineers doresponses of lsquolsquoI donrsquot knowrsquorsquo decreased from 26 to 5by the end of the camp Additionally analysis ofstudentsrsquo open-ended responses indicated a shift in theirunderstanding from thinking that chemical engineersonly mix and use chemicals to engaging in more complexwork (ie experimenting) with chemicals

Discussion

Overall the results indicate that participants improved their(1) understanding of technology (2) chemical engineeringcontent knowledge and (3) attitudes towards engineering Ofthe seven AMSE subscales tested only the responses on theEngineer subscale changed significantly This indicates thatover the course of the camp participantsrsquo attitudes towardsengineering became more positive with more participants

reporting that they know what an engineer does These resultsare consistent with the open-ended response questionsResponses to the open-ended lsquolsquoWhat is an Engineerrsquorsquo prompton the What is Engineering post-test indicate that partici-pants gained an understanding of engineers as people whowork in teams and mass produce products

The scores on the What is Technology instrument weresignificantly higher on the post-test indicating that the campintervention had a positive impact on the participantsrsquo under-standings of technology As evident in the frequency counts andopen-ended responses many participants began camp with themisconception that technology only included items that requiredelectricity These results are consistent with the findings ofCunningham et al (2005) who found that students in grades1ndash5 tend to associate technology with objects requiringelectricity By the end of camp all participants reported thattechnology referred to any object that was manmade

Scores on the Engineering Content Test were significantlyhigher on the post-test indicating that the intervention had

0

20

40

60

80

100

S

elec

ted

Item

Pre

Post

Figure 3 Frequency of participantsrsquo responses for items on the What is Technology instrument Items that are considered technology

0

20

40

60

80

100

S

elec

ted

Item

Pre

Post

Figure 4 Frequency of participantsrsquo responses for items on the What is Engineering instrument Items associated with engineering

R Hammack et al Journal of Pre-College Engineering Education Research 17

8httpdxdoiorg1077712157-92881102

a positive impact on participantsrsquo content knowledge relatedto chemical engineering Although the median post-testscore was over 40 points higher than the pretest medianit was still only 57 which is below the 70 proficiencyscore used by the school where the camp was held Thissuggests that while participants increased their understandingof chemical engineering they still did not master all of thecontent covered in camp The short duration of camp maynot have provided enough time for students to fully masterthe new concepts

The range in frequencies on the What is Engineeringpretest point to participantsrsquo uncertainties about the activitiesof an engineer There was still a large range in the post-testresults indicating that participants still had some degree ofuncertainty about the work of engineers However there weresome positive changes worth noting Prior to participating incamp 68 of participants viewed engineers as people whowork as a team whereas all campers circled lsquolsquowork as ateamrsquorsquo on the post instrument All of the camp activitiesrequired that participants work as a team to solve problemswhich may have accounted for this change Recognizing that

engineers are not solitary loners but in fact work in teams totackle important real world problems could help combat thecommon negative stereotypes previously reported (Cavallo2007 Miller et al 2006) and may prevent more adole-scents form abandoning the STEM pipeline Additionallyparticipantsrsquo Engineering Content Test responses also indi-cated a positive shift in their attitudes towards engineering as acareer For instance only 26 of camp participants reportedthat they had considered becoming a chemical engineer priorto camp whereas 63 of participants reported that campencouraged them to consider chemical engineering as a career

Although the findings indicate an increase in the number ofstudent considering a career in engineering they still heldmany misconceptions about the work of engineers It isimportant to note that the engineering camp did not explicitlyaddress all misconceptions presented on the What isEngineering instrument However the camp did discuss thatengineers focus their work on the design and creative aspectsof construction but are not the actual builders This highlights atrend in our findings that the participantsrsquo views of engineeringshifted away from engineers as mere builders and a movetowards an understanding that engineers design solutions toproblems This trend is mirrored in the open-ended responsesas nearly all participants recognized that lsquolsquobuildingrsquorsquo was notan act of engineering by the end of the camp The only twolsquolsquobuildingrsquorsquo responses on the post-test were

lsquolsquoAn engineer is a person who builds a lot of one thingrsquorsquolsquolsquoSomeone who builds lots of stuffrsquorsquoIt is important to note that one major focus of the candy

airplane activity was the role engineers play in industry andmass production and the mass production of chemicals wasdiscussed during both the slime and film canister rocketactivities It is therefore not surprising that over one-half of theparticipants referred to mass production in their post-testresponses The two participants who wrote lsquolsquobuildrsquorsquo in their post

0102030405060708090

100

r

espo

ndin

g

Theme

Pre

Post

Figure 5 Frequency of themes for What is Engineering instrument

0

20

40

60

80

100

120

Electrical Man made Helps people

r

espo

ndin

g

Theme

Pre

Post

Figure 6 Frequency of themes for What is Technology instrument

18 R Hammack et al Journal of Pre-College Engineering Education Research

9httpdxdoiorg1077712157-92881102

responses may also have been referring to mass productionrather than simply building or constructing an object

Interestingly there are conflicting findings between thequantitative and qualitative responses on the What is

Engineering instrument Although the number of imagesstudents selected of engineers designing and improvingproducts increased students qualitative responses related todesigning and improving products decreased Because the

0

10

20

30

40

50

60

r

espo

ndin

g

Theme

Pre

Post

Figure 7 Frequency of themes for Engineering Content Test responses

Table 4Examples of participantsrsquo open-ended responses by theme

Theme Example Pre-program Responses Example Post-program Responses

How do you know something is technologyElectrical It creates energy and has circuit boards and stuff NAMan made Man invents technology eg bicycle subway Something that is technology is a man-made objectHelps people If it was invented by people to help other people NA

What is an engineerMass production NA An engineer creates multiple copies of a productBuilds Someone who builds something Someone who builds lots of stuffDesigns Somebody who designs and builds stuff An engineer designs some product and fins a way

to mass produce itImproves thingshelps others Someone who researches and develops innovations

to make this world a better and easier placeSomeone who improves inventions

Fixes things An engineer is someone who fixes or creates things An engineer is a person that fixes things andworks as a team

Team work NA An engineer is a person that fixes things andworks as a team

Uses tools I think it is someone who builds or designsthings and works with tools and differentkinds of machinery

NA

Uses math and science People who do math and science NA

What does a chemical engineer doI donrsquot know I donrsquot really know I canrsquot describe itMixesUses chemicals Mixes chemicals Someone who works with mixing and learning

about chemicalsManufacturingdesigns new

products or chemicalsWorks with chemistry to make new materials Helps manufacture chemicals

Experiments with chemicals A chemical engineer experiments with different chemicalsto see if they do something useful or harmful

Experiments with different chemicals to seewhat they do

Works with food NA Works with food

R Hammack et al Journal of Pre-College Engineering Education Research 19

10httpdxdoiorg1077712157-92881102

candy airplane slime and rocket activities all dealt with massproduction it is possible that participants concentrated on theaspect of mass production during their open-ended responsesHowever half of participantsrsquo journal prompts about the workof a chemical engineer referred to some aspect of designing orcreating a product on the post-test Additionally there wasa decrease in the qualitative responses related to lsquolsquohelpingpeoplersquorsquo on the What is Technology and What isEngineering instruments The decrease in balanced bythe increase in lsquolsquomanmadersquorsquo responses on the What isTechnology instrument and lsquolsquomass productionrsquorsquo responseson the What is Engineering instrument

Overall the findings indicate that the participants gainedan understanding that engineers are involved with the massproduction of products as well as their design andimprovement As such the decrease in studentsrsquo responsesin these categories is balanced by the increase in thenumber of responses that indicated engineers as massproducers of items Follow-up interviews with participantswould have been valuable as a means of providingadditional information as to how participantsrsquo perceptionsof engineers changed as a result of camp

Limitations

Although the results indicate a positive impact onparticipants it is not clear which components of the camp(hands-on activities interaction with a professional engi-neer combination of factors) contributed to this changeFollow-up interviews could have helped identify thesefactors Campers self-selected into the program and mayhave already had an above average interest in STEM Thiscoupled with the small sample size limits the general-izability of the findings Results were also limited by theinstruments used The What is Engineering instrumentuses a specific definition of engineering that is notuniversally accepted Additionally the validity of thecontent test has not been established and the test has onlybeen used with the 19 study participants Because we onlycollected and analyzed data immediately following thesummer camp our study only addresses short-termimpacts Future research should be conducted to determinelong-term impacts of engineering outreach programs

Conclusion and Implications

In answering our research questions we found thatparticipation in an engineering summer camp had a positiveimpact on middle school studentsrsquo understandings of whattechnology is and what engineers do evidenced by both thequantitative and qualitative data The open-ended questionssupported and added detail to the quantitative findingsproviding more evidence than if either data set had beenused independently Participantsrsquo attitudes towards mathe-matics and science were not significantly impacted by the

summer camp However their attitudes towards engineer-ing were significantly impacted The short duration of thecamp limited the quantity of information that could bepresented to participants with the current intervention focusingprimarily on engineering Although measurable impacts wereaccomplished during the week long summer camp long terminterventions such as the infusion of engineering activitieswithin the school curriculum could allow for attitudinalchanges across multiple STEM disciplines

The partnership between practicing teachers and engineer-ing faculty was important to the success of the camp Althoughthe extent to which co-teaching impacted outcomes was notmeasured the diverse backgrounds of the camp facilitatorsallowed for the combination of pedagogical knowledge andengineering content expertise to create rich engineeringexperiences that were developmentally appropriate for middleschool participants These experiences positively impactedparticipantsrsquo attitudes towards engineering revealing thebenefits of collaborative efforts between K-12 educators andengineering professionals Additional research in this area isneeded

References

Achieve Inc (2013) Next generation science standards WashingtonDC Achieve Inc

Bischoff P J (2006) The role of knowledge structures in the ability ofpreservice elementary teachers to diagnose a childrsquos understanding ofmolecular kinetics Science Education 90(5) 936ndash951 doi101002sce20155

Cavallo A (2007) Draw-a-ScientistMystery box Science and Children45(3) 37ndash41 Retrieved from httpwwwnstaorgpublicationsbrowse_journalsaspxaction5issueampid51025053sc07_045_03

Cavanaugh S (2007) Science camp Just for the girls Academic camps are onthe rise across the country including ones to get adolescent girls excitedabout the exploration of science Education Week 26(45) 26ndash28

Cheville A amp Bunting C (2011) Engineering students for the 21stcentury Student development through the curriculum Advances inEngineering Education 2(4) 1ndash36 Retrieved from httpadvancesaseeorgwp-contentuploadsvol02issue04papersaee-vol02-issue04-p10pdf

Committee on K-12 Engineering Education (2009) In L Katehi GPearson amp M Feder (Eds) Engineering in K-12 educationunderstanding the status and improving the prospects WashingtonDC The National Academies Press

Cunningham C Lachapelle C amp Lindgren-Streicher A (2005) Assessingelementary schools studentsrsquo conceptions of engineering and technologyProceedings of the 2005 American Society for Engineering EducationAnnual Conference amp Exposition Portland OR

DeJarnette N K (2012) Americarsquos children Providing early exposure toSTEM (science technology engineering and math) initiativesEducation 133(1) 77ndash84 Retrieved from httpwwwprojectinnovationbizeducation_2006html

Elam M E Donham B L amp Soloman S R (2012) An engineeringsummer program for underrepresented students from rural schooldistricts Journal of STEM Education 13(2) 35ndash44 Retrieved fromhttpojsjstemorgindexphpjournal5JSTEMamppage5articleampop5viewamppath5B5D51619amppath5B5D51437

Emerson R M Fretz R I amp Shaw L L (1995) Writing EthnographicFieldnotes Chicago IL University of Chicago Press

20 R Hammack et al Journal of Pre-College Engineering Education Research

11httpdxdoiorg1077712157-92881102

What is Technology What is Engineering andEngineering Content

Table 3 presents the descriptive statistics and WilcoxonSigned Rank results for the What is Engineering What isTechnology and engineering content measures Pre- to post-test differences were significant for three measures (p 001)

Figure 3 illustrates the percentage of items selected ascorrect on the What is Technology pre and post measuresAn examination of these percentages indicated that manyparticipants associated technology with items that wereelectronic (100 selected TV and phones) or involved theuse of electricity (100 selected subways 95 selectedpower lines) and none of the participants associated techno-logy with living organisms (0 selected bird tree ordandelions) Fewer participants associated technology withnon-electrical manmade items however a greater percentageof participants selected items that may be associated withconstruction (58 selected bridge 63 selected house 84selected factory) than items not generally associated withconstruction (47 selected shoes cups and bandages 52selected books and bikes) On the What is Technology Post-test however participants clearly associated technology withitems that do not exist in nature (95 selected bandages100 selected shoes subway phone bridge TV cupfactory house power lines bicycle books)

Participantsrsquo responses to the What is Engineeringinstrument are displayed in Figure 4 An examination ofstudent responses indicates that seven of the nine itemsassociated with activities of an engineer (marked with anasterisk in Figure 4) increased at the conclusion of theengineering camp These items include verbs such as

improve develop design and create The other two items (ieDesign clean water figure how to track luggage) showed nochange in the number of responses however more than half ofthe students correctly selected these items Of the seven itemsnot associated with activities of an engineer fewer studentsselected that engineers construct buildings sell food and cleanteeth at the conclusion of camp There was an increase in thenumber of students who incorrectly selected driving machinesand repairing cars as acts of an engineer and over half stillchose that an engineer performs the act of installing wiringSurprisingly two participants held on to the misconceptionthat the work of an engineer includes arranging flowers

Open-Ended ResponsesThis section present results from the qualitative analysis

of an open-ended response from three data sources(1) What is an Engineer (2) What is Technology and(3) Engineering Content Test The percent occurrence ofemergent themes from each data source is presentedin Figures 5 6 and 7 On the What is Technology instrumentstudentsrsquo initial responses to lsquolsquoHow do you know something istechnologyrsquorsquo were categorized as electrical manmade orhelps people However at the end of the engineering camp allstudentsrsquo responses indicated an understanding that technologywas something that is manmade Additionally a large percen-tage of studentsrsquo initial responses to the prompt lsquolsquoWhat is anengineerrsquorsquo were categorized as someone who builds designsor improves things to help people On the post-test howeverthe percentage of responses falling within these categoriesdeclined greatly and a new theme (mass production) wasidentified in 68 of responses Studentsrsquo Engineering ContentTest responses indicated growth in their understanding of

Table 2Summary statistics for participantsrsquo (n519) scores on Attitudes Towards Mathematics Science and Engineering Scale (AMSE)

Subscale

Pretest Post-test

Z pMin Max Mdn Min Max Mdn

AMSE-Overall 81 139 111 78 140 114 20829 0407ISA 13 32 22 9 31 24 20156 0876INSA 5 18 10 4 19 10 20604 0546PO 5 23 17 8 23 16 21225 0220NO 5 13 9 5 21 8 20192 0848PS 6 20 12 6 16 13 20287 0774TS 10 20 14 9 20 13 0528 0528E 14 22 18 16 23 19 0003 0003

p 001

Table 3Summary statistics for participantrsquos scores on What is Engineering and What is Technology and Engineering Content instruments

Instrument n

Pretest Post-test

Z PMin Max Mdn Min Max Mdn

Engineering 19 375 9375 625 50 100 75 23285 0001Technology 19 4375 100 75 100 100 100 22818 0005Content 19 0 57 14 29 86 57 23456 0001

p 001

16 R Hammack et al Journal of Pre-College Engineering Education Research

7httpdxdoiorg1077712157-92881102

chemical engineers Examples of studentsrsquo open-endedresponses on the Engineering Content Test are presentedin Table 4 When asked what chemical engineers doresponses of lsquolsquoI donrsquot knowrsquorsquo decreased from 26 to 5by the end of the camp Additionally analysis ofstudentsrsquo open-ended responses indicated a shift in theirunderstanding from thinking that chemical engineersonly mix and use chemicals to engaging in more complexwork (ie experimenting) with chemicals

Discussion

Overall the results indicate that participants improved their(1) understanding of technology (2) chemical engineeringcontent knowledge and (3) attitudes towards engineering Ofthe seven AMSE subscales tested only the responses on theEngineer subscale changed significantly This indicates thatover the course of the camp participantsrsquo attitudes towardsengineering became more positive with more participants

reporting that they know what an engineer does These resultsare consistent with the open-ended response questionsResponses to the open-ended lsquolsquoWhat is an Engineerrsquorsquo prompton the What is Engineering post-test indicate that partici-pants gained an understanding of engineers as people whowork in teams and mass produce products

The scores on the What is Technology instrument weresignificantly higher on the post-test indicating that the campintervention had a positive impact on the participantsrsquo under-standings of technology As evident in the frequency counts andopen-ended responses many participants began camp with themisconception that technology only included items that requiredelectricity These results are consistent with the findings ofCunningham et al (2005) who found that students in grades1ndash5 tend to associate technology with objects requiringelectricity By the end of camp all participants reported thattechnology referred to any object that was manmade

Scores on the Engineering Content Test were significantlyhigher on the post-test indicating that the intervention had

0

20

40

60

80

100

S

elec

ted

Item

Pre

Post

Figure 3 Frequency of participantsrsquo responses for items on the What is Technology instrument Items that are considered technology

0

20

40

60

80

100

S

elec

ted

Item

Pre

Post

Figure 4 Frequency of participantsrsquo responses for items on the What is Engineering instrument Items associated with engineering

R Hammack et al Journal of Pre-College Engineering Education Research 17

8httpdxdoiorg1077712157-92881102

a positive impact on participantsrsquo content knowledge relatedto chemical engineering Although the median post-testscore was over 40 points higher than the pretest medianit was still only 57 which is below the 70 proficiencyscore used by the school where the camp was held Thissuggests that while participants increased their understandingof chemical engineering they still did not master all of thecontent covered in camp The short duration of camp maynot have provided enough time for students to fully masterthe new concepts

The range in frequencies on the What is Engineeringpretest point to participantsrsquo uncertainties about the activitiesof an engineer There was still a large range in the post-testresults indicating that participants still had some degree ofuncertainty about the work of engineers However there weresome positive changes worth noting Prior to participating incamp 68 of participants viewed engineers as people whowork as a team whereas all campers circled lsquolsquowork as ateamrsquorsquo on the post instrument All of the camp activitiesrequired that participants work as a team to solve problemswhich may have accounted for this change Recognizing that

engineers are not solitary loners but in fact work in teams totackle important real world problems could help combat thecommon negative stereotypes previously reported (Cavallo2007 Miller et al 2006) and may prevent more adole-scents form abandoning the STEM pipeline Additionallyparticipantsrsquo Engineering Content Test responses also indi-cated a positive shift in their attitudes towards engineering as acareer For instance only 26 of camp participants reportedthat they had considered becoming a chemical engineer priorto camp whereas 63 of participants reported that campencouraged them to consider chemical engineering as a career

Although the findings indicate an increase in the number ofstudent considering a career in engineering they still heldmany misconceptions about the work of engineers It isimportant to note that the engineering camp did not explicitlyaddress all misconceptions presented on the What isEngineering instrument However the camp did discuss thatengineers focus their work on the design and creative aspectsof construction but are not the actual builders This highlights atrend in our findings that the participantsrsquo views of engineeringshifted away from engineers as mere builders and a movetowards an understanding that engineers design solutions toproblems This trend is mirrored in the open-ended responsesas nearly all participants recognized that lsquolsquobuildingrsquorsquo was notan act of engineering by the end of the camp The only twolsquolsquobuildingrsquorsquo responses on the post-test were

lsquolsquoAn engineer is a person who builds a lot of one thingrsquorsquolsquolsquoSomeone who builds lots of stuffrsquorsquoIt is important to note that one major focus of the candy

airplane activity was the role engineers play in industry andmass production and the mass production of chemicals wasdiscussed during both the slime and film canister rocketactivities It is therefore not surprising that over one-half of theparticipants referred to mass production in their post-testresponses The two participants who wrote lsquolsquobuildrsquorsquo in their post

0102030405060708090

100

r

espo

ndin

g

Theme

Pre

Post

Figure 5 Frequency of themes for What is Engineering instrument

0

20

40

60

80

100

120

Electrical Man made Helps people

r

espo

ndin

g

Theme

Pre

Post

Figure 6 Frequency of themes for What is Technology instrument

18 R Hammack et al Journal of Pre-College Engineering Education Research

9httpdxdoiorg1077712157-92881102

responses may also have been referring to mass productionrather than simply building or constructing an object

Interestingly there are conflicting findings between thequantitative and qualitative responses on the What is

Engineering instrument Although the number of imagesstudents selected of engineers designing and improvingproducts increased students qualitative responses related todesigning and improving products decreased Because the

0

10

20

30

40

50

60

r

espo

ndin

g

Theme

Pre

Post

Figure 7 Frequency of themes for Engineering Content Test responses

Table 4Examples of participantsrsquo open-ended responses by theme

Theme Example Pre-program Responses Example Post-program Responses

How do you know something is technologyElectrical It creates energy and has circuit boards and stuff NAMan made Man invents technology eg bicycle subway Something that is technology is a man-made objectHelps people If it was invented by people to help other people NA

What is an engineerMass production NA An engineer creates multiple copies of a productBuilds Someone who builds something Someone who builds lots of stuffDesigns Somebody who designs and builds stuff An engineer designs some product and fins a way

to mass produce itImproves thingshelps others Someone who researches and develops innovations

to make this world a better and easier placeSomeone who improves inventions

Fixes things An engineer is someone who fixes or creates things An engineer is a person that fixes things andworks as a team

Team work NA An engineer is a person that fixes things andworks as a team

Uses tools I think it is someone who builds or designsthings and works with tools and differentkinds of machinery

NA

Uses math and science People who do math and science NA

What does a chemical engineer doI donrsquot know I donrsquot really know I canrsquot describe itMixesUses chemicals Mixes chemicals Someone who works with mixing and learning

about chemicalsManufacturingdesigns new

products or chemicalsWorks with chemistry to make new materials Helps manufacture chemicals

Experiments with chemicals A chemical engineer experiments with different chemicalsto see if they do something useful or harmful

Experiments with different chemicals to seewhat they do

Works with food NA Works with food

R Hammack et al Journal of Pre-College Engineering Education Research 19

10httpdxdoiorg1077712157-92881102

candy airplane slime and rocket activities all dealt with massproduction it is possible that participants concentrated on theaspect of mass production during their open-ended responsesHowever half of participantsrsquo journal prompts about the workof a chemical engineer referred to some aspect of designing orcreating a product on the post-test Additionally there wasa decrease in the qualitative responses related to lsquolsquohelpingpeoplersquorsquo on the What is Technology and What isEngineering instruments The decrease in balanced bythe increase in lsquolsquomanmadersquorsquo responses on the What isTechnology instrument and lsquolsquomass productionrsquorsquo responseson the What is Engineering instrument

Overall the findings indicate that the participants gainedan understanding that engineers are involved with the massproduction of products as well as their design andimprovement As such the decrease in studentsrsquo responsesin these categories is balanced by the increase in thenumber of responses that indicated engineers as massproducers of items Follow-up interviews with participantswould have been valuable as a means of providingadditional information as to how participantsrsquo perceptionsof engineers changed as a result of camp

Limitations

Although the results indicate a positive impact onparticipants it is not clear which components of the camp(hands-on activities interaction with a professional engi-neer combination of factors) contributed to this changeFollow-up interviews could have helped identify thesefactors Campers self-selected into the program and mayhave already had an above average interest in STEM Thiscoupled with the small sample size limits the general-izability of the findings Results were also limited by theinstruments used The What is Engineering instrumentuses a specific definition of engineering that is notuniversally accepted Additionally the validity of thecontent test has not been established and the test has onlybeen used with the 19 study participants Because we onlycollected and analyzed data immediately following thesummer camp our study only addresses short-termimpacts Future research should be conducted to determinelong-term impacts of engineering outreach programs

Conclusion and Implications

In answering our research questions we found thatparticipation in an engineering summer camp had a positiveimpact on middle school studentsrsquo understandings of whattechnology is and what engineers do evidenced by both thequantitative and qualitative data The open-ended questionssupported and added detail to the quantitative findingsproviding more evidence than if either data set had beenused independently Participantsrsquo attitudes towards mathe-matics and science were not significantly impacted by the

summer camp However their attitudes towards engineer-ing were significantly impacted The short duration of thecamp limited the quantity of information that could bepresented to participants with the current intervention focusingprimarily on engineering Although measurable impacts wereaccomplished during the week long summer camp long terminterventions such as the infusion of engineering activitieswithin the school curriculum could allow for attitudinalchanges across multiple STEM disciplines

The partnership between practicing teachers and engineer-ing faculty was important to the success of the camp Althoughthe extent to which co-teaching impacted outcomes was notmeasured the diverse backgrounds of the camp facilitatorsallowed for the combination of pedagogical knowledge andengineering content expertise to create rich engineeringexperiences that were developmentally appropriate for middleschool participants These experiences positively impactedparticipantsrsquo attitudes towards engineering revealing thebenefits of collaborative efforts between K-12 educators andengineering professionals Additional research in this area isneeded

References

Achieve Inc (2013) Next generation science standards WashingtonDC Achieve Inc

Bischoff P J (2006) The role of knowledge structures in the ability ofpreservice elementary teachers to diagnose a childrsquos understanding ofmolecular kinetics Science Education 90(5) 936ndash951 doi101002sce20155

Cavallo A (2007) Draw-a-ScientistMystery box Science and Children45(3) 37ndash41 Retrieved from httpwwwnstaorgpublicationsbrowse_journalsaspxaction5issueampid51025053sc07_045_03

Cavanaugh S (2007) Science camp Just for the girls Academic camps are onthe rise across the country including ones to get adolescent girls excitedabout the exploration of science Education Week 26(45) 26ndash28

Cheville A amp Bunting C (2011) Engineering students for the 21stcentury Student development through the curriculum Advances inEngineering Education 2(4) 1ndash36 Retrieved from httpadvancesaseeorgwp-contentuploadsvol02issue04papersaee-vol02-issue04-p10pdf

Committee on K-12 Engineering Education (2009) In L Katehi GPearson amp M Feder (Eds) Engineering in K-12 educationunderstanding the status and improving the prospects WashingtonDC The National Academies Press

Cunningham C Lachapelle C amp Lindgren-Streicher A (2005) Assessingelementary schools studentsrsquo conceptions of engineering and technologyProceedings of the 2005 American Society for Engineering EducationAnnual Conference amp Exposition Portland OR

DeJarnette N K (2012) Americarsquos children Providing early exposure toSTEM (science technology engineering and math) initiativesEducation 133(1) 77ndash84 Retrieved from httpwwwprojectinnovationbizeducation_2006html

Elam M E Donham B L amp Soloman S R (2012) An engineeringsummer program for underrepresented students from rural schooldistricts Journal of STEM Education 13(2) 35ndash44 Retrieved fromhttpojsjstemorgindexphpjournal5JSTEMamppage5articleampop5viewamppath5B5D51619amppath5B5D51437

Emerson R M Fretz R I amp Shaw L L (1995) Writing EthnographicFieldnotes Chicago IL University of Chicago Press

20 R Hammack et al Journal of Pre-College Engineering Education Research

11httpdxdoiorg1077712157-92881102

chemical engineers Examples of studentsrsquo open-endedresponses on the Engineering Content Test are presentedin Table 4 When asked what chemical engineers doresponses of lsquolsquoI donrsquot knowrsquorsquo decreased from 26 to 5by the end of the camp Additionally analysis ofstudentsrsquo open-ended responses indicated a shift in theirunderstanding from thinking that chemical engineersonly mix and use chemicals to engaging in more complexwork (ie experimenting) with chemicals

Discussion

Overall the results indicate that participants improved their(1) understanding of technology (2) chemical engineeringcontent knowledge and (3) attitudes towards engineering Ofthe seven AMSE subscales tested only the responses on theEngineer subscale changed significantly This indicates thatover the course of the camp participantsrsquo attitudes towardsengineering became more positive with more participants

reporting that they know what an engineer does These resultsare consistent with the open-ended response questionsResponses to the open-ended lsquolsquoWhat is an Engineerrsquorsquo prompton the What is Engineering post-test indicate that partici-pants gained an understanding of engineers as people whowork in teams and mass produce products

The scores on the What is Technology instrument weresignificantly higher on the post-test indicating that the campintervention had a positive impact on the participantsrsquo under-standings of technology As evident in the frequency counts andopen-ended responses many participants began camp with themisconception that technology only included items that requiredelectricity These results are consistent with the findings ofCunningham et al (2005) who found that students in grades1ndash5 tend to associate technology with objects requiringelectricity By the end of camp all participants reported thattechnology referred to any object that was manmade

Scores on the Engineering Content Test were significantlyhigher on the post-test indicating that the intervention had

0

20

40

60

80

100

S

elec

ted

Item

Pre

Post

Figure 3 Frequency of participantsrsquo responses for items on the What is Technology instrument Items that are considered technology

0

20

40

60

80

100

S

elec

ted

Item

Pre

Post

Figure 4 Frequency of participantsrsquo responses for items on the What is Engineering instrument Items associated with engineering

R Hammack et al Journal of Pre-College Engineering Education Research 17

8httpdxdoiorg1077712157-92881102

a positive impact on participantsrsquo content knowledge relatedto chemical engineering Although the median post-testscore was over 40 points higher than the pretest medianit was still only 57 which is below the 70 proficiencyscore used by the school where the camp was held Thissuggests that while participants increased their understandingof chemical engineering they still did not master all of thecontent covered in camp The short duration of camp maynot have provided enough time for students to fully masterthe new concepts

The range in frequencies on the What is Engineeringpretest point to participantsrsquo uncertainties about the activitiesof an engineer There was still a large range in the post-testresults indicating that participants still had some degree ofuncertainty about the work of engineers However there weresome positive changes worth noting Prior to participating incamp 68 of participants viewed engineers as people whowork as a team whereas all campers circled lsquolsquowork as ateamrsquorsquo on the post instrument All of the camp activitiesrequired that participants work as a team to solve problemswhich may have accounted for this change Recognizing that

engineers are not solitary loners but in fact work in teams totackle important real world problems could help combat thecommon negative stereotypes previously reported (Cavallo2007 Miller et al 2006) and may prevent more adole-scents form abandoning the STEM pipeline Additionallyparticipantsrsquo Engineering Content Test responses also indi-cated a positive shift in their attitudes towards engineering as acareer For instance only 26 of camp participants reportedthat they had considered becoming a chemical engineer priorto camp whereas 63 of participants reported that campencouraged them to consider chemical engineering as a career

Although the findings indicate an increase in the number ofstudent considering a career in engineering they still heldmany misconceptions about the work of engineers It isimportant to note that the engineering camp did not explicitlyaddress all misconceptions presented on the What isEngineering instrument However the camp did discuss thatengineers focus their work on the design and creative aspectsof construction but are not the actual builders This highlights atrend in our findings that the participantsrsquo views of engineeringshifted away from engineers as mere builders and a movetowards an understanding that engineers design solutions toproblems This trend is mirrored in the open-ended responsesas nearly all participants recognized that lsquolsquobuildingrsquorsquo was notan act of engineering by the end of the camp The only twolsquolsquobuildingrsquorsquo responses on the post-test were

lsquolsquoAn engineer is a person who builds a lot of one thingrsquorsquolsquolsquoSomeone who builds lots of stuffrsquorsquoIt is important to note that one major focus of the candy

airplane activity was the role engineers play in industry andmass production and the mass production of chemicals wasdiscussed during both the slime and film canister rocketactivities It is therefore not surprising that over one-half of theparticipants referred to mass production in their post-testresponses The two participants who wrote lsquolsquobuildrsquorsquo in their post

0102030405060708090

100

r

espo

ndin

g

Theme

Pre

Post

Figure 5 Frequency of themes for What is Engineering instrument

0

20

40

60

80

100

120

Electrical Man made Helps people

r

espo

ndin

g

Theme

Pre

Post

Figure 6 Frequency of themes for What is Technology instrument

18 R Hammack et al Journal of Pre-College Engineering Education Research

9httpdxdoiorg1077712157-92881102

responses may also have been referring to mass productionrather than simply building or constructing an object

Interestingly there are conflicting findings between thequantitative and qualitative responses on the What is

Engineering instrument Although the number of imagesstudents selected of engineers designing and improvingproducts increased students qualitative responses related todesigning and improving products decreased Because the

0

10

20

30

40

50

60

r

espo

ndin

g

Theme

Pre

Post

Figure 7 Frequency of themes for Engineering Content Test responses

Table 4Examples of participantsrsquo open-ended responses by theme

Theme Example Pre-program Responses Example Post-program Responses

How do you know something is technologyElectrical It creates energy and has circuit boards and stuff NAMan made Man invents technology eg bicycle subway Something that is technology is a man-made objectHelps people If it was invented by people to help other people NA

What is an engineerMass production NA An engineer creates multiple copies of a productBuilds Someone who builds something Someone who builds lots of stuffDesigns Somebody who designs and builds stuff An engineer designs some product and fins a way

to mass produce itImproves thingshelps others Someone who researches and develops innovations

to make this world a better and easier placeSomeone who improves inventions

Fixes things An engineer is someone who fixes or creates things An engineer is a person that fixes things andworks as a team

Team work NA An engineer is a person that fixes things andworks as a team

Uses tools I think it is someone who builds or designsthings and works with tools and differentkinds of machinery

NA

Uses math and science People who do math and science NA

What does a chemical engineer doI donrsquot know I donrsquot really know I canrsquot describe itMixesUses chemicals Mixes chemicals Someone who works with mixing and learning

about chemicalsManufacturingdesigns new

products or chemicalsWorks with chemistry to make new materials Helps manufacture chemicals

Experiments with chemicals A chemical engineer experiments with different chemicalsto see if they do something useful or harmful

Experiments with different chemicals to seewhat they do

Works with food NA Works with food

R Hammack et al Journal of Pre-College Engineering Education Research 19

10httpdxdoiorg1077712157-92881102

candy airplane slime and rocket activities all dealt with massproduction it is possible that participants concentrated on theaspect of mass production during their open-ended responsesHowever half of participantsrsquo journal prompts about the workof a chemical engineer referred to some aspect of designing orcreating a product on the post-test Additionally there wasa decrease in the qualitative responses related to lsquolsquohelpingpeoplersquorsquo on the What is Technology and What isEngineering instruments The decrease in balanced bythe increase in lsquolsquomanmadersquorsquo responses on the What isTechnology instrument and lsquolsquomass productionrsquorsquo responseson the What is Engineering instrument

Overall the findings indicate that the participants gainedan understanding that engineers are involved with the massproduction of products as well as their design andimprovement As such the decrease in studentsrsquo responsesin these categories is balanced by the increase in thenumber of responses that indicated engineers as massproducers of items Follow-up interviews with participantswould have been valuable as a means of providingadditional information as to how participantsrsquo perceptionsof engineers changed as a result of camp

Limitations

Although the results indicate a positive impact onparticipants it is not clear which components of the camp(hands-on activities interaction with a professional engi-neer combination of factors) contributed to this changeFollow-up interviews could have helped identify thesefactors Campers self-selected into the program and mayhave already had an above average interest in STEM Thiscoupled with the small sample size limits the general-izability of the findings Results were also limited by theinstruments used The What is Engineering instrumentuses a specific definition of engineering that is notuniversally accepted Additionally the validity of thecontent test has not been established and the test has onlybeen used with the 19 study participants Because we onlycollected and analyzed data immediately following thesummer camp our study only addresses short-termimpacts Future research should be conducted to determinelong-term impacts of engineering outreach programs

Conclusion and Implications

In answering our research questions we found thatparticipation in an engineering summer camp had a positiveimpact on middle school studentsrsquo understandings of whattechnology is and what engineers do evidenced by both thequantitative and qualitative data The open-ended questionssupported and added detail to the quantitative findingsproviding more evidence than if either data set had beenused independently Participantsrsquo attitudes towards mathe-matics and science were not significantly impacted by the

summer camp However their attitudes towards engineer-ing were significantly impacted The short duration of thecamp limited the quantity of information that could bepresented to participants with the current intervention focusingprimarily on engineering Although measurable impacts wereaccomplished during the week long summer camp long terminterventions such as the infusion of engineering activitieswithin the school curriculum could allow for attitudinalchanges across multiple STEM disciplines

The partnership between practicing teachers and engineer-ing faculty was important to the success of the camp Althoughthe extent to which co-teaching impacted outcomes was notmeasured the diverse backgrounds of the camp facilitatorsallowed for the combination of pedagogical knowledge andengineering content expertise to create rich engineeringexperiences that were developmentally appropriate for middleschool participants These experiences positively impactedparticipantsrsquo attitudes towards engineering revealing thebenefits of collaborative efforts between K-12 educators andengineering professionals Additional research in this area isneeded

References

Achieve Inc (2013) Next generation science standards WashingtonDC Achieve Inc

Bischoff P J (2006) The role of knowledge structures in the ability ofpreservice elementary teachers to diagnose a childrsquos understanding ofmolecular kinetics Science Education 90(5) 936ndash951 doi101002sce20155

Cavallo A (2007) Draw-a-ScientistMystery box Science and Children45(3) 37ndash41 Retrieved from httpwwwnstaorgpublicationsbrowse_journalsaspxaction5issueampid51025053sc07_045_03

Cavanaugh S (2007) Science camp Just for the girls Academic camps are onthe rise across the country including ones to get adolescent girls excitedabout the exploration of science Education Week 26(45) 26ndash28

Cheville A amp Bunting C (2011) Engineering students for the 21stcentury Student development through the curriculum Advances inEngineering Education 2(4) 1ndash36 Retrieved from httpadvancesaseeorgwp-contentuploadsvol02issue04papersaee-vol02-issue04-p10pdf

Committee on K-12 Engineering Education (2009) In L Katehi GPearson amp M Feder (Eds) Engineering in K-12 educationunderstanding the status and improving the prospects WashingtonDC The National Academies Press

Cunningham C Lachapelle C amp Lindgren-Streicher A (2005) Assessingelementary schools studentsrsquo conceptions of engineering and technologyProceedings of the 2005 American Society for Engineering EducationAnnual Conference amp Exposition Portland OR

DeJarnette N K (2012) Americarsquos children Providing early exposure toSTEM (science technology engineering and math) initiativesEducation 133(1) 77ndash84 Retrieved from httpwwwprojectinnovationbizeducation_2006html

Elam M E Donham B L amp Soloman S R (2012) An engineeringsummer program for underrepresented students from rural schooldistricts Journal of STEM Education 13(2) 35ndash44 Retrieved fromhttpojsjstemorgindexphpjournal5JSTEMamppage5articleampop5viewamppath5B5D51619amppath5B5D51437

Emerson R M Fretz R I amp Shaw L L (1995) Writing EthnographicFieldnotes Chicago IL University of Chicago Press

20 R Hammack et al Journal of Pre-College Engineering Education Research

11httpdxdoiorg1077712157-92881102

a positive impact on participantsrsquo content knowledge relatedto chemical engineering Although the median post-testscore was over 40 points higher than the pretest medianit was still only 57 which is below the 70 proficiencyscore used by the school where the camp was held Thissuggests that while participants increased their understandingof chemical engineering they still did not master all of thecontent covered in camp The short duration of camp maynot have provided enough time for students to fully masterthe new concepts

The range in frequencies on the What is Engineeringpretest point to participantsrsquo uncertainties about the activitiesof an engineer There was still a large range in the post-testresults indicating that participants still had some degree ofuncertainty about the work of engineers However there weresome positive changes worth noting Prior to participating incamp 68 of participants viewed engineers as people whowork as a team whereas all campers circled lsquolsquowork as ateamrsquorsquo on the post instrument All of the camp activitiesrequired that participants work as a team to solve problemswhich may have accounted for this change Recognizing that

engineers are not solitary loners but in fact work in teams totackle important real world problems could help combat thecommon negative stereotypes previously reported (Cavallo2007 Miller et al 2006) and may prevent more adole-scents form abandoning the STEM pipeline Additionallyparticipantsrsquo Engineering Content Test responses also indi-cated a positive shift in their attitudes towards engineering as acareer For instance only 26 of camp participants reportedthat they had considered becoming a chemical engineer priorto camp whereas 63 of participants reported that campencouraged them to consider chemical engineering as a career

Although the findings indicate an increase in the number ofstudent considering a career in engineering they still heldmany misconceptions about the work of engineers It isimportant to note that the engineering camp did not explicitlyaddress all misconceptions presented on the What isEngineering instrument However the camp did discuss thatengineers focus their work on the design and creative aspectsof construction but are not the actual builders This highlights atrend in our findings that the participantsrsquo views of engineeringshifted away from engineers as mere builders and a movetowards an understanding that engineers design solutions toproblems This trend is mirrored in the open-ended responsesas nearly all participants recognized that lsquolsquobuildingrsquorsquo was notan act of engineering by the end of the camp The only twolsquolsquobuildingrsquorsquo responses on the post-test were

lsquolsquoAn engineer is a person who builds a lot of one thingrsquorsquolsquolsquoSomeone who builds lots of stuffrsquorsquoIt is important to note that one major focus of the candy

airplane activity was the role engineers play in industry andmass production and the mass production of chemicals wasdiscussed during both the slime and film canister rocketactivities It is therefore not surprising that over one-half of theparticipants referred to mass production in their post-testresponses The two participants who wrote lsquolsquobuildrsquorsquo in their post

0102030405060708090

100

r

espo

ndin

g

Theme

Pre

Post

Figure 5 Frequency of themes for What is Engineering instrument

0

20

40

60

80

100

120

Electrical Man made Helps people

r

espo

ndin

g

Theme

Pre

Post

Figure 6 Frequency of themes for What is Technology instrument

18 R Hammack et al Journal of Pre-College Engineering Education Research

9httpdxdoiorg1077712157-92881102

responses may also have been referring to mass productionrather than simply building or constructing an object

Interestingly there are conflicting findings between thequantitative and qualitative responses on the What is

Engineering instrument Although the number of imagesstudents selected of engineers designing and improvingproducts increased students qualitative responses related todesigning and improving products decreased Because the

0

10

20

30

40

50

60

r

espo

ndin

g

Theme

Pre

Post

Figure 7 Frequency of themes for Engineering Content Test responses

Table 4Examples of participantsrsquo open-ended responses by theme

Theme Example Pre-program Responses Example Post-program Responses

How do you know something is technologyElectrical It creates energy and has circuit boards and stuff NAMan made Man invents technology eg bicycle subway Something that is technology is a man-made objectHelps people If it was invented by people to help other people NA

What is an engineerMass production NA An engineer creates multiple copies of a productBuilds Someone who builds something Someone who builds lots of stuffDesigns Somebody who designs and builds stuff An engineer designs some product and fins a way

to mass produce itImproves thingshelps others Someone who researches and develops innovations

to make this world a better and easier placeSomeone who improves inventions

Fixes things An engineer is someone who fixes or creates things An engineer is a person that fixes things andworks as a team

Team work NA An engineer is a person that fixes things andworks as a team

Uses tools I think it is someone who builds or designsthings and works with tools and differentkinds of machinery

NA

Uses math and science People who do math and science NA

What does a chemical engineer doI donrsquot know I donrsquot really know I canrsquot describe itMixesUses chemicals Mixes chemicals Someone who works with mixing and learning

about chemicalsManufacturingdesigns new

products or chemicalsWorks with chemistry to make new materials Helps manufacture chemicals

Experiments with chemicals A chemical engineer experiments with different chemicalsto see if they do something useful or harmful

Experiments with different chemicals to seewhat they do

Works with food NA Works with food

R Hammack et al Journal of Pre-College Engineering Education Research 19

10httpdxdoiorg1077712157-92881102

candy airplane slime and rocket activities all dealt with massproduction it is possible that participants concentrated on theaspect of mass production during their open-ended responsesHowever half of participantsrsquo journal prompts about the workof a chemical engineer referred to some aspect of designing orcreating a product on the post-test Additionally there wasa decrease in the qualitative responses related to lsquolsquohelpingpeoplersquorsquo on the What is Technology and What isEngineering instruments The decrease in balanced bythe increase in lsquolsquomanmadersquorsquo responses on the What isTechnology instrument and lsquolsquomass productionrsquorsquo responseson the What is Engineering instrument

Overall the findings indicate that the participants gainedan understanding that engineers are involved with the massproduction of products as well as their design andimprovement As such the decrease in studentsrsquo responsesin these categories is balanced by the increase in thenumber of responses that indicated engineers as massproducers of items Follow-up interviews with participantswould have been valuable as a means of providingadditional information as to how participantsrsquo perceptionsof engineers changed as a result of camp

Limitations

Although the results indicate a positive impact onparticipants it is not clear which components of the camp(hands-on activities interaction with a professional engi-neer combination of factors) contributed to this changeFollow-up interviews could have helped identify thesefactors Campers self-selected into the program and mayhave already had an above average interest in STEM Thiscoupled with the small sample size limits the general-izability of the findings Results were also limited by theinstruments used The What is Engineering instrumentuses a specific definition of engineering that is notuniversally accepted Additionally the validity of thecontent test has not been established and the test has onlybeen used with the 19 study participants Because we onlycollected and analyzed data immediately following thesummer camp our study only addresses short-termimpacts Future research should be conducted to determinelong-term impacts of engineering outreach programs

Conclusion and Implications

In answering our research questions we found thatparticipation in an engineering summer camp had a positiveimpact on middle school studentsrsquo understandings of whattechnology is and what engineers do evidenced by both thequantitative and qualitative data The open-ended questionssupported and added detail to the quantitative findingsproviding more evidence than if either data set had beenused independently Participantsrsquo attitudes towards mathe-matics and science were not significantly impacted by the

summer camp However their attitudes towards engineer-ing were significantly impacted The short duration of thecamp limited the quantity of information that could bepresented to participants with the current intervention focusingprimarily on engineering Although measurable impacts wereaccomplished during the week long summer camp long terminterventions such as the infusion of engineering activitieswithin the school curriculum could allow for attitudinalchanges across multiple STEM disciplines

The partnership between practicing teachers and engineer-ing faculty was important to the success of the camp Althoughthe extent to which co-teaching impacted outcomes was notmeasured the diverse backgrounds of the camp facilitatorsallowed for the combination of pedagogical knowledge andengineering content expertise to create rich engineeringexperiences that were developmentally appropriate for middleschool participants These experiences positively impactedparticipantsrsquo attitudes towards engineering revealing thebenefits of collaborative efforts between K-12 educators andengineering professionals Additional research in this area isneeded

References

Achieve Inc (2013) Next generation science standards WashingtonDC Achieve Inc

Bischoff P J (2006) The role of knowledge structures in the ability ofpreservice elementary teachers to diagnose a childrsquos understanding ofmolecular kinetics Science Education 90(5) 936ndash951 doi101002sce20155

Cavallo A (2007) Draw-a-ScientistMystery box Science and Children45(3) 37ndash41 Retrieved from httpwwwnstaorgpublicationsbrowse_journalsaspxaction5issueampid51025053sc07_045_03

Cavanaugh S (2007) Science camp Just for the girls Academic camps are onthe rise across the country including ones to get adolescent girls excitedabout the exploration of science Education Week 26(45) 26ndash28

Cheville A amp Bunting C (2011) Engineering students for the 21stcentury Student development through the curriculum Advances inEngineering Education 2(4) 1ndash36 Retrieved from httpadvancesaseeorgwp-contentuploadsvol02issue04papersaee-vol02-issue04-p10pdf

Committee on K-12 Engineering Education (2009) In L Katehi GPearson amp M Feder (Eds) Engineering in K-12 educationunderstanding the status and improving the prospects WashingtonDC The National Academies Press

Cunningham C Lachapelle C amp Lindgren-Streicher A (2005) Assessingelementary schools studentsrsquo conceptions of engineering and technologyProceedings of the 2005 American Society for Engineering EducationAnnual Conference amp Exposition Portland OR

DeJarnette N K (2012) Americarsquos children Providing early exposure toSTEM (science technology engineering and math) initiativesEducation 133(1) 77ndash84 Retrieved from httpwwwprojectinnovationbizeducation_2006html

Elam M E Donham B L amp Soloman S R (2012) An engineeringsummer program for underrepresented students from rural schooldistricts Journal of STEM Education 13(2) 35ndash44 Retrieved fromhttpojsjstemorgindexphpjournal5JSTEMamppage5articleampop5viewamppath5B5D51619amppath5B5D51437

Emerson R M Fretz R I amp Shaw L L (1995) Writing EthnographicFieldnotes Chicago IL University of Chicago Press

20 R Hammack et al Journal of Pre-College Engineering Education Research

11httpdxdoiorg1077712157-92881102

responses may also have been referring to mass productionrather than simply building or constructing an object

Interestingly there are conflicting findings between thequantitative and qualitative responses on the What is

Engineering instrument Although the number of imagesstudents selected of engineers designing and improvingproducts increased students qualitative responses related todesigning and improving products decreased Because the

0

10

20

30

40

50

60

r

espo

ndin

g

Theme

Pre

Post

Figure 7 Frequency of themes for Engineering Content Test responses

Table 4Examples of participantsrsquo open-ended responses by theme

Theme Example Pre-program Responses Example Post-program Responses

How do you know something is technologyElectrical It creates energy and has circuit boards and stuff NAMan made Man invents technology eg bicycle subway Something that is technology is a man-made objectHelps people If it was invented by people to help other people NA

What is an engineerMass production NA An engineer creates multiple copies of a productBuilds Someone who builds something Someone who builds lots of stuffDesigns Somebody who designs and builds stuff An engineer designs some product and fins a way

to mass produce itImproves thingshelps others Someone who researches and develops innovations

to make this world a better and easier placeSomeone who improves inventions

Fixes things An engineer is someone who fixes or creates things An engineer is a person that fixes things andworks as a team

Team work NA An engineer is a person that fixes things andworks as a team

Uses tools I think it is someone who builds or designsthings and works with tools and differentkinds of machinery

NA

Uses math and science People who do math and science NA

What does a chemical engineer doI donrsquot know I donrsquot really know I canrsquot describe itMixesUses chemicals Mixes chemicals Someone who works with mixing and learning

about chemicalsManufacturingdesigns new

products or chemicalsWorks with chemistry to make new materials Helps manufacture chemicals

Experiments with chemicals A chemical engineer experiments with different chemicalsto see if they do something useful or harmful

Experiments with different chemicals to seewhat they do

Works with food NA Works with food

R Hammack et al Journal of Pre-College Engineering Education Research 19

10httpdxdoiorg1077712157-92881102

candy airplane slime and rocket activities all dealt with massproduction it is possible that participants concentrated on theaspect of mass production during their open-ended responsesHowever half of participantsrsquo journal prompts about the workof a chemical engineer referred to some aspect of designing orcreating a product on the post-test Additionally there wasa decrease in the qualitative responses related to lsquolsquohelpingpeoplersquorsquo on the What is Technology and What isEngineering instruments The decrease in balanced bythe increase in lsquolsquomanmadersquorsquo responses on the What isTechnology instrument and lsquolsquomass productionrsquorsquo responseson the What is Engineering instrument

Overall the findings indicate that the participants gainedan understanding that engineers are involved with the massproduction of products as well as their design andimprovement As such the decrease in studentsrsquo responsesin these categories is balanced by the increase in thenumber of responses that indicated engineers as massproducers of items Follow-up interviews with participantswould have been valuable as a means of providingadditional information as to how participantsrsquo perceptionsof engineers changed as a result of camp

Limitations

Although the results indicate a positive impact onparticipants it is not clear which components of the camp(hands-on activities interaction with a professional engi-neer combination of factors) contributed to this changeFollow-up interviews could have helped identify thesefactors Campers self-selected into the program and mayhave already had an above average interest in STEM Thiscoupled with the small sample size limits the general-izability of the findings Results were also limited by theinstruments used The What is Engineering instrumentuses a specific definition of engineering that is notuniversally accepted Additionally the validity of thecontent test has not been established and the test has onlybeen used with the 19 study participants Because we onlycollected and analyzed data immediately following thesummer camp our study only addresses short-termimpacts Future research should be conducted to determinelong-term impacts of engineering outreach programs

Conclusion and Implications

In answering our research questions we found thatparticipation in an engineering summer camp had a positiveimpact on middle school studentsrsquo understandings of whattechnology is and what engineers do evidenced by both thequantitative and qualitative data The open-ended questionssupported and added detail to the quantitative findingsproviding more evidence than if either data set had beenused independently Participantsrsquo attitudes towards mathe-matics and science were not significantly impacted by the

summer camp However their attitudes towards engineer-ing were significantly impacted The short duration of thecamp limited the quantity of information that could bepresented to participants with the current intervention focusingprimarily on engineering Although measurable impacts wereaccomplished during the week long summer camp long terminterventions such as the infusion of engineering activitieswithin the school curriculum could allow for attitudinalchanges across multiple STEM disciplines

The partnership between practicing teachers and engineer-ing faculty was important to the success of the camp Althoughthe extent to which co-teaching impacted outcomes was notmeasured the diverse backgrounds of the camp facilitatorsallowed for the combination of pedagogical knowledge andengineering content expertise to create rich engineeringexperiences that were developmentally appropriate for middleschool participants These experiences positively impactedparticipantsrsquo attitudes towards engineering revealing thebenefits of collaborative efforts between K-12 educators andengineering professionals Additional research in this area isneeded

References

Achieve Inc (2013) Next generation science standards WashingtonDC Achieve Inc

Bischoff P J (2006) The role of knowledge structures in the ability ofpreservice elementary teachers to diagnose a childrsquos understanding ofmolecular kinetics Science Education 90(5) 936ndash951 doi101002sce20155

Cavallo A (2007) Draw-a-ScientistMystery box Science and Children45(3) 37ndash41 Retrieved from httpwwwnstaorgpublicationsbrowse_journalsaspxaction5issueampid51025053sc07_045_03

Cavanaugh S (2007) Science camp Just for the girls Academic camps are onthe rise across the country including ones to get adolescent girls excitedabout the exploration of science Education Week 26(45) 26ndash28

Cheville A amp Bunting C (2011) Engineering students for the 21stcentury Student development through the curriculum Advances inEngineering Education 2(4) 1ndash36 Retrieved from httpadvancesaseeorgwp-contentuploadsvol02issue04papersaee-vol02-issue04-p10pdf

Committee on K-12 Engineering Education (2009) In L Katehi GPearson amp M Feder (Eds) Engineering in K-12 educationunderstanding the status and improving the prospects WashingtonDC The National Academies Press

Cunningham C Lachapelle C amp Lindgren-Streicher A (2005) Assessingelementary schools studentsrsquo conceptions of engineering and technologyProceedings of the 2005 American Society for Engineering EducationAnnual Conference amp Exposition Portland OR

DeJarnette N K (2012) Americarsquos children Providing early exposure toSTEM (science technology engineering and math) initiativesEducation 133(1) 77ndash84 Retrieved from httpwwwprojectinnovationbizeducation_2006html

Elam M E Donham B L amp Soloman S R (2012) An engineeringsummer program for underrepresented students from rural schooldistricts Journal of STEM Education 13(2) 35ndash44 Retrieved fromhttpojsjstemorgindexphpjournal5JSTEMamppage5articleampop5viewamppath5B5D51619amppath5B5D51437

Emerson R M Fretz R I amp Shaw L L (1995) Writing EthnographicFieldnotes Chicago IL University of Chicago Press

20 R Hammack et al Journal of Pre-College Engineering Education Research

11httpdxdoiorg1077712157-92881102

candy airplane slime and rocket activities all dealt with massproduction it is possible that participants concentrated on theaspect of mass production during their open-ended responsesHowever half of participantsrsquo journal prompts about the workof a chemical engineer referred to some aspect of designing orcreating a product on the post-test Additionally there wasa decrease in the qualitative responses related to lsquolsquohelpingpeoplersquorsquo on the What is Technology and What isEngineering instruments The decrease in balanced bythe increase in lsquolsquomanmadersquorsquo responses on the What isTechnology instrument and lsquolsquomass productionrsquorsquo responseson the What is Engineering instrument

Overall the findings indicate that the participants gainedan understanding that engineers are involved with the massproduction of products as well as their design andimprovement As such the decrease in studentsrsquo responsesin these categories is balanced by the increase in thenumber of responses that indicated engineers as massproducers of items Follow-up interviews with participantswould have been valuable as a means of providingadditional information as to how participantsrsquo perceptionsof engineers changed as a result of camp

Limitations

Although the results indicate a positive impact onparticipants it is not clear which components of the camp(hands-on activities interaction with a professional engi-neer combination of factors) contributed to this changeFollow-up interviews could have helped identify thesefactors Campers self-selected into the program and mayhave already had an above average interest in STEM Thiscoupled with the small sample size limits the general-izability of the findings Results were also limited by theinstruments used The What is Engineering instrumentuses a specific definition of engineering that is notuniversally accepted Additionally the validity of thecontent test has not been established and the test has onlybeen used with the 19 study participants Because we onlycollected and analyzed data immediately following thesummer camp our study only addresses short-termimpacts Future research should be conducted to determinelong-term impacts of engineering outreach programs

Conclusion and Implications

In answering our research questions we found thatparticipation in an engineering summer camp had a positiveimpact on middle school studentsrsquo understandings of whattechnology is and what engineers do evidenced by both thequantitative and qualitative data The open-ended questionssupported and added detail to the quantitative findingsproviding more evidence than if either data set had beenused independently Participantsrsquo attitudes towards mathe-matics and science were not significantly impacted by the

summer camp However their attitudes towards engineer-ing were significantly impacted The short duration of thecamp limited the quantity of information that could bepresented to participants with the current intervention focusingprimarily on engineering Although measurable impacts wereaccomplished during the week long summer camp long terminterventions such as the infusion of engineering activitieswithin the school curriculum could allow for attitudinalchanges across multiple STEM disciplines

The partnership between practicing teachers and engineer-ing faculty was important to the success of the camp Althoughthe extent to which co-teaching impacted outcomes was notmeasured the diverse backgrounds of the camp facilitatorsallowed for the combination of pedagogical knowledge andengineering content expertise to create rich engineeringexperiences that were developmentally appropriate for middleschool participants These experiences positively impactedparticipantsrsquo attitudes towards engineering revealing thebenefits of collaborative efforts between K-12 educators andengineering professionals Additional research in this area isneeded

References

Achieve Inc (2013) Next generation science standards WashingtonDC Achieve Inc

Bischoff P J (2006) The role of knowledge structures in the ability ofpreservice elementary teachers to diagnose a childrsquos understanding ofmolecular kinetics Science Education 90(5) 936ndash951 doi101002sce20155

Cavallo A (2007) Draw-a-ScientistMystery box Science and Children45(3) 37ndash41 Retrieved from httpwwwnstaorgpublicationsbrowse_journalsaspxaction5issueampid51025053sc07_045_03

Cavanaugh S (2007) Science camp Just for the girls Academic camps are onthe rise across the country including ones to get adolescent girls excitedabout the exploration of science Education Week 26(45) 26ndash28

Cheville A amp Bunting C (2011) Engineering students for the 21stcentury Student development through the curriculum Advances inEngineering Education 2(4) 1ndash36 Retrieved from httpadvancesaseeorgwp-contentuploadsvol02issue04papersaee-vol02-issue04-p10pdf

Committee on K-12 Engineering Education (2009) In L Katehi GPearson amp M Feder (Eds) Engineering in K-12 educationunderstanding the status and improving the prospects WashingtonDC The National Academies Press

Cunningham C Lachapelle C amp Lindgren-Streicher A (2005) Assessingelementary schools studentsrsquo conceptions of engineering and technologyProceedings of the 2005 American Society for Engineering EducationAnnual Conference amp Exposition Portland OR

DeJarnette N K (2012) Americarsquos children Providing early exposure toSTEM (science technology engineering and math) initiativesEducation 133(1) 77ndash84 Retrieved from httpwwwprojectinnovationbizeducation_2006html

Elam M E Donham B L amp Soloman S R (2012) An engineeringsummer program for underrepresented students from rural schooldistricts Journal of STEM Education 13(2) 35ndash44 Retrieved fromhttpojsjstemorgindexphpjournal5JSTEMamppage5articleampop5viewamppath5B5D51619amppath5B5D51437

Emerson R M Fretz R I amp Shaw L L (1995) Writing EthnographicFieldnotes Chicago IL University of Chicago Press

20 R Hammack et al Journal of Pre-College Engineering Education Research

11httpdxdoiorg1077712157-92881102

Fralick B Kearn J Thompson S amp Lyons J (2009) How middleschoolers draw engineers and scientists Journal of Science Educationand Technology 18(1) 60ndash73 doi101007s10956-008-9133-3

Hall C Dickerson J Batts D Kauffmann P amp Bosse M (2011) Arewe missing opportunities to encourage interest in STEM fieldsJournal of Technology Education 23(1) 32ndash46 Retrieved from wwwfilesericedgovfulltextEJ965337pdf

Hammack R amp High K (2014) Effects of an after school engineeringmentoring program on middle school girlsrsquo perceptions of engineersJournal of Women and Minorities in Science and Engineering 20(1)11ndash20 doi101615JWomenMinorScienEng2014006726

Hirsch L Carpinelli J D Kimmel H Rockland R amp Bloom J(2007) The differential effects of preengineering curricula on middleschool studentsrsquo attitudes to knowledge of engineering careersProceedings of the 37th Annual ASEEIEEE Frontiers in EducationConference Milwaukee WI

International Technology Education Association (2000) Standards forTechnological Literacy Content for the Study of Technology RestonVA ITEA

Jones M G Howe A amp Rua M J (2000) Gender differences instudentsrsquo experiences interests and attitudes toward science andscientists Science Education 84(2) 180ndash192 doi101002(SICI)1098-237X(200003)8423C180AID-SCE33E30CO2-X

Karatas F O Micklos A amp Bodner G M (2011) Sixth-grade studentsrsquoviews of the nature of engineering and images of engineers Journal ofScience and Education Technology 20 123ndash135 doi 101007s10956-010-9239-2

Knight M amp Cunningham C (2004) Draw an engineer test (DAET)Development of a tool to investigate studentsrsquo ideas about engineersand engineering Proceedings of the 2004 American Society forEngineering Education Annual Conference amp Exposition Salt LakeCity UT

Lent R W Brown S D amp Hackett G (1994) Toward a unifying socialcognitive theory of career and academic interest choice andperformance Journal of Vocational Behavior 45(1) 79ndash122 doi101006jvbe19941027

Maltese A V amp Tai R H (2010) Eyeballs in the fridge Sources ofearly interest in science International Journal of Science Education32(5) 669ndash685 doi 10108009500690902792385

Mason C L Kahle J B amp Gardner A L (1991) Draw-A-Scientisttest Future implications School Science and Mathematics 91(5)193ndash198

Metin D amp Leblebicioglu G (2011) How did a science camp affectchildrenrsquos conceptions of science Asia-Pacific Forum on ScienceLearning and Teaching 12(1) doi105296jsev2i11348

Miller P H Blessing J amp Schwartz S (2006) Gender differences inhigh-school studentsrsquo views about science International Journal ofScience Education 28(4) 363ndash381 doi 10108009500690500277664

Mooney M A amp Laubach T A (2002) Adventure engineering A designcentered inquiry based approach to middle grade science andmathematics education Journal of Engineering Education 91(3)309ndash318 doi 101002j2168-98302002tb00708x

Nadelson L S amp Callahan J (2011) A comparison of two engineeringoutreach programs for adolescents Journal of STEM Education 12(1-2)43ndash54 Retrieved from httpojsjstemorgindexphpjournal5JSTEMamppage5articleampop5viewamppath5B5D51527amppath5B5D51348

National Academy of Engineering and National Research Council (2009)Engineering in k-12 education Understanding the status andimproving the prospects Washington DC National Academies Press

National Science Foundation (1983) Educating Americans for the twenty-first century Report of the national science board commission onprecollege education in mathematics science and technologyWashington DC National Science Foundation

Nugent G Barker B Grandgenett N amp Adamchuk V I (2010)Impact of robotics and geospatial technology interventions on youthSTEM learning and attitudes Journal of Research on Technology inEducation 42(4) 391ndash408 Retrieved from httpwebbebscohostcomargolibraryokstateeduehostpdfviewervid59ampsid50f03328f-43a8-4ea1-9f20-0c507cf340sessionmgr113amphid5125

Oliver M amp Venville G (2011) An exploratory case study of Olympiadstudentsrsquo attitudes towards and passion for science InternationalJournal of Science Education 33(16) 2295ndash2322 doi 101080095006932010550654

Osborne J Simon S amp Collins S (2003) Attitudes towardsscience A review of the literature and its implications InternationalJournal of Science Education 25(9) 1049ndash1079 doi 1010800950069032000032199

Plant E A Baylor A L Doerr C amp Rosenberg-Kima R (2009)Changing middle-school studentsrsquo attitudes and performance regardingengineering with computer-based social models Computer Education53 209ndash215 doi 101016jcompedu200901013

Rahm J Moore J C amp Martel-Reny M-P (2005) The role ofafterschool and community science programs in the lives of urbanyouth School Science amp Mathematics 105(6) 283 doi 101111j1949-85942005tb18129x

Robbins M E amp Schoenfisch M H (2005) An interactive analyticalchemistry summer camp for middle school girls Journal of ChemicalEducation 82(10) 1486ndash1488 doi 101021ed082p1486

Robinson M amp Kenny B (2003) Engineering literacy in high schoolstudents Bulletin of Science Technology amp Society 23(2) 95ndash101doi 1011770270467603251300

Shulman L S (1986) Those who understand Knowledge growth inteaching Educational Researcher 15(2) 4ndash14 Retrieved from httpwwwjstororgstable1175860

Stake J E amp Mares K R (2001) Science enrichment programs forgifted high school girls and boys Predictor of program impact onscience confidence and motivation Journal of Research in ScienceTeaching 38(10) 1065ndash1088 doi101002tea10001

Sweigart C A amp Landrum T J (2015) The impact of number of adultson instruction Implications for co-teaching Preventing SchoolFailure 59(1) 22ndash29 doi 1010801045988X2014919139

US Bureau of Labor Statistics (2014) STEM 101Intro to tomorrowrsquosjobs Occupational Outlook Quarterly 1ndash12 Retrieved from wwwblsgovooq

US Department of Commerce Economics and Statistics Administration(2011) STEM Good jobs now and for the future (ESA Issue Brief No03-11) Washington DC US Government Printing Office

Weinberg A E Basile C G amp Albright L (2011) The effect of anexperiential learning program on middle school studentsrsquo motivationtoward mathematics and science Research in Middle Level Education35(3) 1ndash12 Retrieved from httpwwwamleorgportals0pdfrmlermle_vol35_no3pdf

Wyss V L Heulskamp D amp Siebert C J (2012) Increasing middleschool student interest in STEM careers with videos of scientistsInternational Journal of Environmental amp Science Education 7(4)501ndash522 Retrieved from httpfilesericedgovfulltextEJ997137pdf

Zambo D amp Zambo R (2006) Using thought bubble pictures to assessstudentsrsquo feelings about mathematics Mathematics Teaching in theMiddle School 12(1) 14ndash21 Retrieved from httpwwwjstororgstable41164017

R Hammack et al Journal of Pre-College Engineering Education Research 21

12httpdxdoiorg1077712157-92881102