at SciVerse ScienceDirect

Journal of Environmental Management 128 (2013) 941e948

Contents lists available

Journal of Environmental Management

journal homepage: www.elsevier .com/locate/ jenvman

Application of Delphi-AHP methods to select the priorities of WEEEfor recycling in a waste management decision-making toolq

Mincheol Kim a, Yong-Chul Jang b,*, Seunguk Lee b

aDepartment of Management Information Systems, Jeju National University, Jeju 690-756, South KoreabDepartment of Environmental Engineering, Chungnam National University, Daejeon 305-764, South Korea

a r t i c l e i n f o

Article history:Received 28 March 2013Received in revised form22 June 2013Accepted 30 June 2013Available online 24 July 2013

Keywords:AHPDelphi methodElectronic wasteRecyclingWEEE

q Summary: This paper presents the application odetermine the priorities of waste electrical and electrecycling through the national waste managementKorea.* Corresponding author. Tel.: þ82 42 821 6674; fax

E-mail addresses: mck1292@jejunu.ac.kr (M. Kim),yahoo.com (Y.-C. Jang), sw2007@cnu.ac.kr (S. Lee).

0301-4797/$ e see front matter Crown Copyright � 2http://dx.doi.org/10.1016/j.jenvman.2013.06.049

a b s t r a c t

The management of waste electrical and electronic equipment (WEEE) or electronic waste (e-waste) hasbecome a major issue of concern for solid waste communities due to the large volumes of waste beinggenerated from the consumption of modern electrical and electronic products. In 2003, Korea introducedthe extended producer responsibility (EPR) system to reduce the amount of electronic products to bedisposed and to promote resource recovery from WEEE. The EPR currently regulates a total of 10 elec-trical and electronic products. This paper presents the results of the application of the Delphi method andanalytical hierarchy process (AHP) modeling to the WEEE management tool in the policy-making pro-cess. Specifically, this paper focuses on the application of the Delphi-AHP technique to determine theWEEE priority to be included in the EPR system. Appropriate evaluation criteria were derived using theDelphi method to assess the potential selection and priority among electrical and electronic productsthat will be regulated by the EPR system. Quantitative weightings from the AHP model were calculated toidentify the priorities of electrical and electronic products to be potentially regulated. After applying allthe criteria using the AHP model, the results indicate that the top 10 target recycling products for theexpansion of the WEEE list were found to be vacuum cleaners, electric fans, rice cookers, large freezers,microwave ovens, water purifiers, air purifiers, humidifiers, dryers, and telephones in order from the firstto last. The proposed Delphi-AHP method can offer a more efficient means of selecting WEEE thansubjective assessment methods that are often based on professional judgment or limited available data.By providing WEEE items to be regulated, the proposed Delphi-AHP method can eliminate uncertaintyand subjective assessment and enable WEEE management policy-makers to identify the priority ofpotential WEEE. More generally, the work performed in this study is an example of how Delphi-AHPmodeling can be used as a decision-making process tool in WEEE management.

Crown Copyright � 2013 Published by Elsevier Ltd. All rights reserved.

1. Introduction

Once electrical and electronic products reach the end of theiruseful life, they become waste electrical and electronic equipment(WEEE), which is often referred to as electronic waste (or e-waste inshort). WEEE streams encompass a wide range of electrical andelectronic waste products, including home appliances (e.g., re-frigerators, washing machines, air conditioners); information

f the Delphi-AHP method toronic equipment (WEEE) fordecision-making process in

: þ82 42 822 5610.gogator@cnu.ac.kr, gatorycj@

013 Published by Elsevier Ltd. All

technology and telecommunication equipment (e.g., personalcomputers, notebook computers, printers, copying equipment,calculators, facsimiles, telephones, mobile phones); consumerelectronic devices (e.g., televisions, radios, video cameras, audioequipment); and other household electrical and electronic equip-ment (e.g., vacuum cleaners, toasters, coffee machines, hair dryers,watches, irons). WEEE is one of the fastest growing solid wastestreams in many countries (Babu et al., 2007). The generation ofWEEE has increased in quantity and variety due to thewide use andcommon replacement of electronic devices in this moderntechnology-driven society. As the life cycles of electrical and elec-tronic products are becoming shorter, the quantity of WEEE isexpected to increase in solid waste streams. Thus, proper man-agement of WEEE has become a major concern for solid wasteprofessionals because of the large growth of the waste stream aswell as the presence of myriad toxic materials within it (e.g., lead,

rights reserved.

M. Kim et al. / Journal of Environmental Management 128 (2013) 941e948942

cadmium, mercury, polybrominated diphenyl esters) (Huo et al.,2007; Wong et al., 2007; Scharnhorst et al., 2007; Hidy et al., 2011).

In response to growing concern over WEEE, many countries areworking to establish proper treatment and recycling processes toreduce the quantity of WEEE that is disposed and to recover valu-able resources. Unlike municipal solid waste, WEEE managementsystems have not been well established in most countries. Manyrecent studies have been published to address potential problemsassociated with WEEE management (He et al., 2006; Liu et al.,2006; Musson et al., 2006; Babu et al., 2007; Kahhat et al., 2008;Yang et al., 2008; Jang, 2010; Hidy et al., 2011).

In 2002, the Korea Ministry of Environment (Korea MOE)modified the Act on the Promotion of Conservation and Recycling ofResources for effective collection and recycling of waste materials(Korea MOE, 2002). The Act went into effect in 2003, and Koreaintroduced an extended producer responsibility (EPR) regulationfor packaging materials and electronic devices. The EPR requiresproducers to take more responsibility for managing the environ-mental impacts of their products throughout their life cycles. In2007, the Act on the Resource Recycling of Waste Electrical Elec-tronic Equipment (WEEE) and End-of-life Vehicles (ELVs) wasenacted (Korea MOE, 2007). The Korea WEEE Act regulates sixhazardous substances (lead, mercury, cadmium, hexavalent chro-mium, polybrominated biphenyls, polybrominated diphenyl esters)in electronics that disturb recycling and increase environmentalimpacts upon disposal, similar to the EU RoHS (the restriction of theuse of certain hazardous substances) Directive (EU Directive,2002a). However, the categories and types of WEEE regulated bythe WEEE Act are still very limited and include only 10 electricaland electronic devices (i.e., refrigerators, washing machines, tele-visions, air conditioners, computers, printers, facsimiles, audioequipment, copy machines, and mobile phones). The EU WEEEdirective consists of 10 categories with a total of 96 devices or typesof equipment including large and small home appliances, infor-mation technology and telecommunications equipment, lightingequipment, medical devices, automatic dispensers, and otherscategories (EU Directive, 2002b). Unlike the European WEEEdirective, current management policies and regulations in Koreahave focused on large home appliances and limited IT products(e.g., computers, printers, mobile phones). The categories and itemsregulated by the EPR system under the WEEE Act need to beexpanded, including small home appliances (e.g., vacuum cleaners,coffee makers) and other IT products (e.g., MP3 players, gameplayers).

This study was intended to assess the potential of waste elec-tronic devices or products that may become target WEEE forrecycling and management in Korea. Four WEEE categories wereselected for the evaluation process: new WEEE including large-sized home appliances, information and communication technol-ogy (ICT) devices, small and medium-sized household electronicdevices, and audio and video equipment. Although these types ofwaste devices or equipmentmay not comprise a large proportion ofWEEE streams, they merit special attention due to concerns overthe hazards they pose to the environment upon disposal (Aizawaet al., 2008). Whether these devices or equipment should beincluded in WEEE regulations may depend on economic, environ-mental, and social factors such as the presence of proper recyclingtechnology, generation rates (or emission rates), potential eco-nomic values, the presence of toxic chemicals and valuable metals,and the potential environmental impacts of the devices.

There is an urgent need to identify the ranking of new targetWEEE for mandatory recycling and management in Korea. How-ever, selecting an appropriate WEEE to be regulated can be achallenging and subjective task for waste management policy-makers, especially since waste management professionals and

policy-makers often lack precise and objective decision-makingprocedures and evaluation criteria. Therefore, integrating objec-tive and quantitative tools into the evaluation procedure enablesdecision-makers to efficiently and objectively identify the mostappropriate WEEE to be recycled. In order to rank the target de-vices, we introduced the Delphi method and analytical hierarchyprocess (AHP) as a tool in the policy-making process for WEEErecycling. AHPmodelingwas used to determine the criteriaweightsand establish an evaluation model. The Delphi method wasemployed to identify objective evaluation criteria for selectingWEEE to be recycled.

A number of studies have examined the adoption of the AHPmethod for environmental management. Randall et al. (2004)adopted the AHP method for management of surplus elementalmercury in the US. Cram et al. (2006) used the AHP method toprioritize the same vegetation types based on endemic species. Hsuet al. (2008) utilized the AHP method to select the proper com-panies for medical waste disposal using interviews with medicalwaste experts by the Delphi method. Chang et al. (2009) combinedthe geographic information system (GIS) and fuzzy AHP method tosearch for the most appropriate distribution strategy in Taipei,Taiwan. Wang et al. (2009) used the AHP method to lower thecomplexity of waste management systems in order to select theappropriate solid waste landfill site. Lin et al. (2010) adopted theAHP method to determine the relative priorities of the addition ofnew mandatory recycled waste in Taiwan.

2. Background of WEEE management in Korea

The republic of Korea, with a population of approximately 50million people, has experienced high environmental pollutionloadings due to its rapid industrial development over the pastseveral decades. The Korea’s rapid economic growth, combinedwith its thriving electronics and information technology industry,fueled by such Korean electronics companies as SAMSUNG and LG,has become the major driving force for the expansion of domesticmarkets for electrical and electronic products. Recent statisticsshow that as of 2011, more than 52 million mobile phones, 24million televisions, 17 million refrigerators, and 12.8 million com-puters were in use in Korea (KPE, 2012).

Determining life spans of electronic devices is central to esti-mating the potential rate ofWEEE generation. Estimates are usuallybased on domestic demand for electronic devices and their averagelife span (i.e., the length of the time between the initial purchase ofan electronic device and the time it completes its useful life). Lifespans vary depending upon the type of device, economic andmarket conditions, age, and cultural behavior. It was estimated thatthe average life spans of the devices studied were 7.8 years forrefrigerators, 7.8 years for washing machines, 7.4 years for televi-sions, and 6.0 years for air-conditioners. Shorter life spans werefound for personal computers (4.0 years), notebook computers (4.0years), printers (4.3 years), and mobile phones (2.4 years) (Jang andLim, 2007; Jang, 2010).

By combining sales data (or domestic demand) with life spanassumptions for each product, the total estimated number of units(or sometimes the corresponding weight) of a particular electronicdevice that is retired for waste management via reuse and refur-bishment, recycling, export, or disposal (i.e., the amount of WEEEretired) can be estimated. The estimated amount of WEEE gener-ated is determined by the sum of the retired amount of WEEEsubtracted from the volume of WEEE that is reused, loaned, orstored at households. Based on the assumptions above, the retiredand generated rate of WEEE in Korea can be estimated. Forexample, in this study, it was estimated that among 52 millionmobile phones in use, more than 22 million mobile phones were

Table 1Target recycling rates and accomplishment of WEEE recycling by the EPR system in Korea.

Type of WEEE 2009 2010 2011

Domesticdemand

EPR targetrate

Amount recycledby producers

Domesticdemand

EPR targetrate

Amount recycledby producers

Domesticdemand

EPR targetrate

Amount recycledby producers

Refrigerators 222,474 45,830 58,636 234,430 51,809 64,618 231,792 57,948 62,568Washing

machines95,470 24,918 26,046 107,136 29,356 29,215 97,884 27,897 27,885

Televisions 74,214 11,874 18,544 86,300 16,397 21,491 73,821 15,502 19,585Air-conditioners 126,979 2921 2887 128,790 3091 3064 146,862 3525 4060Computers 47,605 5284 8383 54,571 6712 9790 55,506 7771 7141Audio

equipment4901 760 685 4667 793 711 4157 769 788

Mobile phones 3,206 635 629 3537 778 731 3302 759 619Copying

machines4636 617 588 5732 814 994 5682 852 1002

Facsimiles 468 57 117 525 70 129 365 55 70Printers 11,929 1420 1938 15,097 1963 2462 14,606 2191 2298Total 591,882 94,316 118,453 640,785 111,783 133,205 633,977 117,269 126,016

Recycling target rates by EPR and the amount of WEEE recycled in Korea between 2009 and 2011.Source: Korea KEC, 2012

M. Kim et al. / Journal of Environmental Management 128 (2013) 941e948 943

retired in 2011 only, while more than 8 million of the mobilephones that were generated could be reused or recycled.

Table 1 presents the domestic demand, EPR target rate, andactual recycling rates of WEEE between 2009 and 2011 in Koreaunder the EPR system. The Korea MOE determine the annualmandatory recycling rate of each product, based on target recyclingrates over the previous years, the amount of electrical and elec-tronic products shipped from the warehouse, and recycling marketconditions. The annual amount of EPR mandatory recycling rate isdetermined by the annual amount of electrical and electronicproducts that are sold in domestic market multiplied by the annualmandatory recycling rate. For example, in 2011, the Korea MOEdetermined that the amounts of recycling EPR target rates were57,948 tons for refrigerators, 27,897 tons for washing machines,15,502 tons for televisions, and 3,525 tons for air conditioners. Theamounts of most WEEE products producers recycled generallyexceeded their mandatory target rates.

Fig. 1 shows the trends of recycling rates ofWEEE between 2006and 2011. The quantity of WEEE recycled generally increased overthe years because of the increased EPR target rate and regulation.The amount of WEEE recycled in 2011 was approximately 5.4%lower than the amount of WEE recycled in 2010. When compared

Fig. 1. Recycling rates of WEEE in K

with the average WEEE recycling rate (6.6 kg/person/yr) ofadvanced EU countries in 2010, the recycling rate (2.8 kg/person/yr)of WEEE in Korea as of 2011 is still much lower (EC, 2013).

In recent years, the national government and local governmentshave employed various methods to recycle WEEE. These includethe establishment of collection systems for diverse e-waste devices,setting long-term national target rates for recycling of WEEE,assigning mandatory collection rates to electronics retailers anddistributors, planning for the expansion of mandatory WEEErecycling by the EPR system, increasing the role of local govern-ment in WEEE collection, and developing a recycling technologyroadmap for WEEE. If these WEEE collection and recycling effortsare improved, the amount of WEEE that is recycled will continuallyincrease over time. Starting in 2012, some municipalities started tocollect small and medium-sized WEEE in curbside collection con-tainers free of charge in residential areas. In some municipalities,there is a free door-to-door collection service available for house-holds who call on-line service to pick up large, obsolete householdappliances by producers. In 2012, mandatory collection rates wereassigned to larger electronics retailers and distributors for recyclingof the 10 EPR items regulated by the Korea MOE. In 2013, therequired collection rates for retailers range from 0.6% of air

orea between 2006 and 2011.

M. Kim et al. / Journal of Environmental Management 128 (2013) 941e948944

conditioners to 25% of televisions that are sold to consumers (KoreaMOE, 2013). In addition, there is growing interest in urban miningfrom WEEE in Korea due to rapidly increasing prices stemmingfrom the depletion of valuable and rare metals (e.g., palladium,lithium, tin, cobalt, nickel, indium, titanium, neodymium, tantalum,and other metals) (Lee et al., 2007).

3. Methodology

In this study, the methodology included gathering data associ-ated with annual domestic demand for (or sales of) selected homeappliances and electronic devices, site visits, questionnaire surveysfor the Delphi-AHP method, interviews and conversations, and areview of available literature. Site visits to locations includingWEEE recycling centers and facilities, regulatory agencies, and theKorea Association of Electronics & Environment (Korea AEE), aproducer responsibility organization (PRO), were made to supportand supplement information gathered by the surveys. Interviewsand conversations with environmental regulatory agencies, recy-cling industry experts, and PRO committees were conducted toobtain the details of recent progress and development associatedwith WEEE management. Available literature was also obtainedand examined to compare WEEE collection and recycling rates inKorea with other countries and regions. An attempt was made toselect the priority types of WEEE to be targeted for mandatoryrecycling in the EPR system in Korea. In addition to adopting theDelphi method to identify evaluation criteria, this study used theAHP to establish a model for selecting the priority devices. Thetheoretical approaches and backgrounds that were adopted aredescribed below.

3.1. Delphi method

In this study, we used the Delphi and AHP methods to selectproducts for the expansion and promotion of WEEE recycling inorder to present priority rankings based on selected evaluationcriteria. Our goal was to use the criteria and results as a basis todevelop regulatory measures for the selection of products for in-clusion in the expansion and maximization of WEEE recycling. Weconducted an expert survey for appropriate evaluation criteria us-ing the Delphi method, interviewing a total of 10 experts fromgovernment, academia, and industry sectors in order to prioritizeproducts for WEEE recycling. Based on the evaluation criteriaselected for priority ranking, the AHP method was used to deter-mine the relative importance of each product. We also collectedevidence from expert interviews, site visits, and the availableliterature on generation rates (waste volume or emission rate),recycling rates, valuable metal content, and ease of establishing acollection system.

To determine the evaluation criteria to prioritize the products,we applied the Delphi method. The Delphi method is used torepeatedly obtain expert opinions until there is a comprehensiveconsensus on selecting projects, predicting problems, and resolvingproblems (Delbecq et al., 1975). In contrast to the commonly usedmethod of brainstorming, in which experts convene in one place toreach a consensus, the Delphi method overcomes the disadvantageof an outspoken person or collective group thinking dominating theoutcome by allowing experts to respond anonymously. In thisstudy, we primarily used e-mail to obtain opinions and conductedexpert surveys twice (1st expert survey and 2nd expert survey).Specifically, we analyzed the coefficient of variation (CV) for theexpert surveys and content validity ratio (CVR). In these evaluationcriteria, when the CV value is less than 0.5, additional surveys arestopped (Dajani et al., 1979). The CVR developed by Lawshe (1975)and recalculated by Wilson et al. (2012) measures agreement

among survey raters as to how essential a particular factor or itemis. The CVR ranges from þ1 to �1. A higher positive value is used asan indicator that survey experts were in agreement that a factor oritemwas essential. Generally, a CVR that is greater than 0.29 can beconsidered to be an appropriate evaluation level. The coefficient ofvariation is the ratio of the standard deviation to the mean. Usingthe CV can make it easier to compare the overall precision of thedata obtained, as shown in Equation (1):

Content Validity Ratio ðCVRÞ ¼ NE� N=2N=2

(1)

where Ne¼ the number of survey experts indicating that a factor oritem is “essential” and N ¼ the total number of survey experts.

3.2. AHP method

When the environment is more complex and more factors needto be considered in decision-making process, it becomes difficult toselect appropriate alternatives. Hence, in this study, we introducedthe concept of hierarchical pair-wise comparison by applying theAHP method. By using AHP, the decision-making process can bedivided into several hierarchical levels, and by using a pairwisecomparison at each level, a decision can be made based on theknowledge and experience of many experts (Saaty, 1996). AHP as-sesses the priorities of multiple alternatives under various valua-tion criteria.

AHP calculates the relative importance of decision-makingthrough stratification and class analysis. If the population to bestudied is composed of many people, standards and periods, AHPis useful because it can separate these criteria during analysis.AHP uses class analysis and problem stratification to determinerelative importance (Saaty, 1996). The classes of decision-makingare determined-in the first phase, which may be the mostimportant step in AHP application. The second phase collectsevaluation data using pairwise comparison between decision-making factors, and also draws the matrix through pairwisecomparison in a subclass that is dedicated to accomplishing thegoals of each factor.

The next phase uses amatrix of pairwise comparison to estimatethe weights and relative importance of the determination factorswithin each class. The w1, w2, w3, and wn are calculated, indicatingthe effects and preference of valuation standards c1, c2, c3 and cn byusing the aij value acquired during the pairwise comparison. Saaty(1996) proposed the eigenvalue method as a weight evaluation, asshown in Equation (2):

A0$W 0 ¼ lmax$W 0 (2)

where A is the square matrix resulting from pairwise comparison,lmax is the maximum eigenvalue, and w is the eigenvector.

The measurement of consistency is developed using two char-acteristics. One is that matrix A has greater consistency as lmax getscloser to n. The other is that lmax values are always bigger than orthe same as n, as shown in Equations (3) and (4):

Consistency Index ðCIÞ ¼ lmax � nn� 1

(3)

Consistency Ratio ðCRÞ ¼ CIðConsistency IndexÞRIðRandom IndexÞ (4)

The random index is determined by the size of n. In cases whenthe consistency is perfect, lmax is equal to n so that CI becomes0 and CR is 0. On the contrary, as the consistency of judgments gets

Table 2CV and CVR results of final evaluation criteria.

Criteria CV CVR

High waste generation rate or waste volume 0.14 0.80Low recycling cost 0.24 0.40Availability of recycling technology 0.15 0.56High valuable metal content 0.20 0.80Established collection system 0.13 0.40Similarity to current Extended Producer’s

Responsibility (EPR) item0.21 0.40

Similarity to current plastic disposal feecharge items

0.31 0.40Fig. 3. Relative importance of 1st evaluation criterion.

Fig. 4. Relative importance of 2nd evaluation criterion.

M. Kim et al. / Journal of Environmental Management 128 (2013) 941e948 945

lower, lmax becomes bigger than n and the CI and CR are both largerthan 0. Saaty (1996) advised that CR is consistent in cases where itis smaller than or equal to 0.1. If it exceeds 0.1, the pairwise com-parison needs to be done again or the questionnaire has to berevised.

In the last phase, the relative weights of decision-making factorsare integrated to evaluate the total ranks of several different alter-natives. The total importance of vectors that determine the prior-ities of alternatives at the bottom of the list are computed to achievethe purposes of determination at the top, whichmakes it possible tocombine the weights of each class acquired in the third phase. Theweighting equation of alternatives is shown in Equation (5):

ui ¼X�

uj�ðmijÞ (5)

ui : Total weight of the alternative i, uj : Relative weight of thevaluation standard j, mij : Weight of the alternative i to the valuationstandard j

In this study, we used the AHPmethod to collect expert opinionsfrom the Korea MOE, academia, and industry sectors in the field ofWEEE management and recycling. The AHP survey was adminis-tered to the experts; the results were analyzed and presented usingExpert Choice 2000 software. If the importance of a specific eval-uation criterion changes, the priority of product selection also canvary. Therefore, to determine the effect of the evaluation criteria onthe results, we also performed sensitivity analysis for each crite-rion. These methods improve the efficiency of the decision-makingprocess.

4. Results and discussion

4.1. Evaluation criteria by the Delphi method

We surveyed 10 experts in the field of WEEE recycling twiceusing the Delphi method in order to determine the evaluationcriteria for prioritizing recycling of waste electronic and electricalproducts. Twelve criteria were selected by the first Delphi survey.

Fig. 2. Overall hierarchical structure of the AHP framework.

Seven final criteria were selected by the second Delphi survey,including high waste generation rate (or emission rate), low recy-cling cost, availability of recycling technology, established collec-tion system, high valuable metal content (recovery potential),similarity to products regulated by current EPR items, and simi-larity to current plastic disposal fee charge items regulated by theKorea MOE. The CV and CVR results of the seven final criteria areshown in Table 2. Each evaluation criterion met the required levelsof CV (less than 0.5) and CVR (greater than 0.29). Based on theresults, similar criteria were grouped together hierarchically ac-cording to primary and secondary criteria, as shown in Fig. 2. In thehierarchical structure, the overall goal was to identify the priority ofmandatory recycling items to be targeted. Level 1 (1st evaluationcriteria) represents the four evaluation criteria (emission rate,recycling benefit, similarity to current EPR items, and similarproducts in the current plastic disposal fee charging system). Sub-criteria (2nd evaluation criteria) under the recycling benefit includeavailability of recycling technology, higher valuable metal recovery,and establishment of a feasible collection system.

4.2. Results of AHP analysis

It is difficult to determine the priority ranking (importance) ofeach evaluation criterion by brainstorming or using the Delphi

Table 3Products to be evaluated by the AHP method.

Category Products

Large home appliances Refrigerator, dryer, oven, water cooler/heater,freezer

IT and communicationdevices

Mobile (cellular) phone, electronic dictionary,scanner, telephone

Audio/video equipment Television, camera, DVD, projector, portable GPSdevice, MP3 player, portable multimedia player,electric musical instruments, game console, hometheater system

Medium-sized or smallconsumer electronics

Vacuum cleaner, microwave oven, electric ricecooker, heating fan, electric fan, air purifier,humidifier, dehumidifier, bidet, electrical mixer,coffee maker, electric iron, table lamp or stand

Fig. 5. Priority of WEEE to be recycled with relative importance.

Table 4Priority ranking among products considered for expanded recycling.

Rank By allfactors

By wastegenerationrate oremission rate

Wastegenerationrate (by mass)a

Wastegenerationrate (by unit)a

1 Vacuumcleaner

Vacuumcleaner

Vacuum cleaner Telephone

2 Electric fan Electric fan Electric mixer Vacuum cleaner3 Electric rice

cookerElectric ricecooker

Electric fan Electric mixer

4 Freezer Microwaveoven

Electric ricecooker

Electric fan

5 Microwaveoven

Telephone Telephone Electric ricecooker

6 Waterpurifier

Humidifier Humidifier Humidifier

7 Air purifier Freezer Microwaveoven

Microwaveoven

8 Humidifier Camera Coffee maker Electric iron9 Kitchen

dryerElectric iron

10 Telephone Waterpurifier

a Source: Kim et al., 2008.

M. Kim et al. / Journal of Environmental Management 128 (2013) 941e948946

method. However, by using AHP analysis, the relative importance ofeach evaluation criterion can be quantified and can ultimately leadto the selection of alternatives (in this study, the priority ranking ofalternative products to be recycled). The relative importance ofeach evaluation criterion was determined, as shown in Fig. 3. Theresults showed that the emission rate or generation rate was themost important factor with a weighted percentage of 46.0%, fol-lowed by the recycling benefit factor (20.7%) and similarity toproducts currently regulated by the EPR item (19.5%). Therefore, inorder to minimize the effort and time spent in the process ofprioritizing the products, the factor to be considered first is theemission rate or generation rate. Fig. 4 shows the relative impor-tance of the secondary evaluation criteria that were determined tobe subordinate to recycling benefit, using the AHP method. Similarto the results listed above, the collection system is a very importantfactor (45.5%), followed by metal recovery (30.8%) and recyclingtechnology (23.8%).

Before starting the AHP method, the Korea MOE suggested 32potential WEEE candidates to be prioritized during expert in-terviews, as shown in Table 3. Among these, three EPR products(i.e., refrigerators, TVs, and mobile or cellular phones) were alsoincluded in expert surveys for the relative comparison of productswith futuremandatory target recycling items. Based on the primaryand secondary selection evaluation, the results of the priorityranking of the products are listed in Fig. 5. After being evaluated forrelative importance, the 32 products in Fig. 5 were ranked ac-cording to their overall priority among recycling items. In additionto the three current WEEE that have been targeted for recycling(refrigerators, TV, and mobile phones), the top ten priority items tobe added to themandatory target recycling EPR list include vacuumcleaners, electric fans, electric rice cookers, large freezers, micro-wave ovens, water purifiers, air purifiers, humidifiers, kitchendryers, and telephones in order (from the first to the last).

Table 4 presents the top ten priority ranking results based on thefactor only, with all factors considered. The emission rate or gen-eration rate was considered to be the most important factor in this

analysis. The results show that when only the emission rate orgeneration rate is considered, it is optimal to include vacuumcleaners, electric fans, electric rice cookers, microwave ovens,telephones, humidifiers, freezers, camera, electric irons, and watercoolers/heaters (in order from first to last) in the expansion ofrecycling target items. When all the factors are considered, the topfive priority items include vacuum cleaners, electric fans, electricrice cookers, freezers, and microwave ovens.

Table 4 also shows the top 10 priority items by the “generationrate or emission rate” factor, comparing them with actual genera-tion rates of small-size electronics and appliances (in order fromfirst to last) by unit and mass reported by other researchers (Kim

Fig. 6. Results of sensitivity analysis with respect to changes of relative importance of evaluation criteria.

M. Kim et al. / Journal of Environmental Management 128 (2013) 941e948 947

et al., 2008). It is found that the order of the top 10 products in ourstudy is very similar to that of the generation rates of small appli-ances reported by Kim et al. (2008) (Table 4).

4.3. Results of sensitivity analysis

To improve the decision-making process, sensitivity analysiswas used to analyze the way changes in the importance of factorsaffect on the results. For example, if the importance of the recyclingbenefit factor increases from 20.7% to 67.8%, the priority ranking ofproducts to be included in the expansion of target recycling itemsalso changes, as shown in Fig. 6. Vacuum cleaners, large freezers,electric rice cookers, microwave ovens, water purifiers, electricfans, dryers, air purifiers, electric ovens, and humidifiers have beenidentified as the top ten items in order from first to last. Thus, thepriority ranking results have also changed due to the change in theimportance of the factors.

In the coming years, the Korea MOE plans to add 16 new cate-gories ofWEEE to bemandatorily recycled by the EPR system, basedon the results of this study as well as a series of discussions amongstakeholders (e.g., producers, NGOs, the recycling industry, andgovernment). The target items to be considered for mandatoryrecycling include water purifiers, air purifiers, vacuum cleaners,microwave ovens, bidets, electric rice cookers, heating fans, electricfans, humidifiers, electric irons, electric cookers, kitchen dryers,video cassettes, electric mixers, food waste drying machines, andwater softeners.

5. Conclusion

In an ever-changing modern electronic and information tech-nology society, numerous electrical and electronic devices aregenerated for consumption and eventual disposal. In order toeffectively increaseWEEE recycling rates imposed by theWEEE Act,the target WEEE list should be gradually expanded, based on socialinfra-structure and consensus among stakeholders. However,selecting an appropriate WEEE to be regulated for recycling andproper treatment is frequently a challenging and subjective taskbecausewastemanagement decision-makers often lack precise andobjective data and evaluation criteria.

In this study, we used the Delphi method and AHP to determinethe priority of target recycling products for the expansion andpromotion of WEEE recycling. Expert surveys were performed todetermine the evaluation criteria and expand the list of EPRproducts to be mandatorily recycled. Evaluation criteria were firstselected based on the results of the primary and secondary Delphisurveys; the results were classified hierarchically, grouped bysimilarities. The primary evaluation criteria included generationrate or emission rate (46.0%), followed by recycling benefit (20.7%),and similarity to products currently regulated by the EPR system(19.5%). The secondary evaluation criteria were collection system(45.4%), followed by valuable metal content (30.8%) and recyclingtechnologies (23.8%). Applying the above criteria using the AHP, theresults indicate that the top 10 target recycling products for theexpansion of the WEEE list were found to be vacuum cleaners,electric fans, electric rice cookers, large freezers, microwave ovens,

M. Kim et al. / Journal of Environmental Management 128 (2013) 941e948948

water purifiers, air purifiers, humidifiers, kitchen dryers, andstandard telephones in order from first to last.

The application of Delphi and AHP modeling proved to be anefficient tool for the WEEE decision-making process. Integratingquantitative methods into the evaluation procedure enableddecision-makers to determine WEEE priorities for recyclingobjectively and efficiently. Recent WEEE regulation efforts alongwith better recycling technology would increase WEEE collectionand recycling rates through diverse collection programs, encourageproducers to develop more environmentally sustainable products,and require producers to take extended responsibility for therecycling of their products. Finally, it should be noted that therelative importance of evaluation criteria used inWEEE analysis canchange due to better collection systems, innovations, and newrecycling technologies. Therefore, this study’s evaluation criteriaand the factors used to determine its relative importance should beupdated and further refined for evaluation in waste managementarena.

Acknowledgments

This work was financially supported by the Korea Ministry ofEnvironment (Korea MOE) as a ‘Knowledge-based waste to energyrecycling human resource development project’.

References

Aizawa, H., Yoshida, H., Sakai, S., 2008. Current results and future perspective forJapanese recycling of home appliances. Resour. Conserv. Recy. 52, 1339e1410.

Babu, B., Parandu, A., Basha, C., 2007. Electrical and electronic waste: a globalenvironmental problem. Waste Manage. Res. 25, 307e318.

Chang, N.-B., Chang, Y.-H., Chen, H.-W., 2009. Fair fund distribution for a municipalincinerator using GIS-based fuzzy analytic hierarchy process. J. Environ.Manage. 90, 441e454.

Cram, S., Sommer, I., Morales, L., Oropeza, O., Carmona, E., González-Medrano, F.,2006. Suitability of the vegetation types in Mexico’s Tamaulipas state for thesiting of hazardous waste treatment plants. J. Environ. Manage. 80, 13e24.

Dajani, J., Sincoff, M., Talley, W., 1979. Stability and agreement criteria for thetermination of Delphi studies. Technol. Forecast. Soc. 13, 83e90.

Delbecq, A.L., van de Ven, A.H., Gustavson, D.H., 1975. Group Techniques for Pro-gram Planning: a Guide to Nominal Group and Delphi Processes. Scott Fores-man and Company, Glenview, Illinois.

European Commission (EC), 2013. Environmental Data Centre on Waste Electricaland Electronic Waste (WEEE). <http://www.epp.eurostat.ec.europa.eu>(accessed 14.01.13.).

EU Directive, 2002a. 2002/96/EC of the European Parliament and of the Council of27 January 2003 on the restriction of the use of certain hazardous substances(RoHS) in waste electrical and electronic equipment. Off. J. L037, 0019e0023.

EU Directive, 2002b. 2002/96/EC of the European Parliament and of the Council of27 January 2003 on waste electrical and electronic equipment (WEEE)-JointDeclaration of the European Parliament, the Council and the Commissionrelating to Article 9. Off. J. L037, 0024e0039.

He, W., Li, G., Ma, X., Wang, H., Huang, J., Xu, M., Huang, C., 2006. WEEE re-covery strategies and the WEEE treatment status in China. J. Hazard. Mater.136, 502e512.

Hidy, G., Alcorn, W., Clarke, R., Smith, D., Thomas, V., 2011. Environmental issuesand management strategies for waste electronic and electrical equipment. J. AirWaste Manage. 61, 990e995.

Hsu, P.-F., Wu, C.-R., Li, Y.-T., 2008. Selection of infectious medical waste disposalfirms by using the analytic hierarchy process and sensitivity analysis. WasteManage. 28, 1386e1394.

Huo, X., Peng, L., Xu, X., Zheng, L., Qiu, B., Qi, Z., Zhang, B., Han, D., Piao, Z., 2007.Elevated blood lead levels of children in Guiyu, an electronic waste recyclingtown in China. Environ. Health Persp. 115, 1113.

Jang, Y., 2010. Waste electrical and electronic equipment (WEEE) management inKorea: generation, collection, and recycling systems. J. Mater. Cycles WasteManage. 12, 283e294.

Jang, Y., Lim, S., 2007. National Survey of E-waste by Material Flow Analysis. FinalReport Submitted to Korean Association of Electronics Environment, Korea (inKorean).

Kahhat, R., Kim, J., Xu, M., Allenby, B., Williams, E., Zhang, P., 2008. Exploring e-waste management systems in the United States. Resour. Conserv. Recy. 52,955e964.

Kim, I., Cho, B., Kim, S., Lee, S., Kim, A., 2008. A Study on Current Trend in Gener-ation and Recycling of Small Household Appliance. Final Report Submitted toKorean Association of Electronics Environment, Korea (in Korean).

Korea Ministry of Environment (Korea MOE), 2002. The Act on the Promotion ofConservation and Recycling of Resources. Korea MOE.

Korea Ministry of Environment (Korea MOE), 2007. The Act on the ResourceRecycling of Waste Electrical Electronic Equipment (WEEE) and End-of-lifeVehicles (ELVs).

Korea Ministry of Environment (Korea MOE), 2013. Collection and Recycling TargetRate of WEEE in 2013. Korea MOE regulation No 2012e251.

Korea Environmental Coporation (KEC), 2012. Annual Report of Recycling of WEEEin Electric and Electronic Equipment and End-of-Life Vehicle in Eco-Assurance(EcoAS). Korea (in Korean).

Korea Power Exchange (KPE), 2012. Penetration Rate of Home Appliances and Be-haviors of Korean Consumers’ Survey for Electricity. Annual report, KPE. (inKorean).

Lawshe, C.H., 1975. A quantitative approach to content validity. Personnel Psychol.28, 563e575.

Lee, J.-C., Song, H.T., Yoo, J.-M., 2007. Present status of the recycling of waste elec-trical and electronic equipment in Korea. Resour. Conserv. Recy. 50, 380e397.

Lin, C., Wen, L., Tsai, Y., 2010. Applying decision-making tools to national e-wasterecycling policy: an example of analytic hierarchy process. Waste Manage. 30,863e869.

Liu, X., Tanaka, M., Matsui, Y., 2006. Electrical and electronic waste management inChina: progress and the barriers to overcome. Waste Manage. Res. 24, 92e101.

Musson, S., Vann, K., Jang, Y., Mutha, S., Jordan, A., Pearson, B., Townsend, T., 2006.RCRA toxicity characterization of discarded electronic devices. Environ. Sci.Technol. 40, 2721e2726.

Randall, P., Brown, L., Deschaine, L., Dimarzio, J., Kaiser, G., Vierow, J., 2004. Appli-cation of the analytic hierarchy process to compare alternatives for the long-term management of surplus mercury. J. Environ. Manage. 71, 35e43.

Saaty, T.L., 1996. The Analytic Hierarchy Process: Planning, Priority Setting, ResourceAllocation. RWS Publications, Pittsburgh.

Scharnhorst, W., Ludwig, C., Wochele, J., Jolliet, O., 2007. Heavy metal partitioningfrom electronic scrap during thermal End-of-Life treatment. Sci. Total Environ.373, 576e584.

Wang, G., Qin, L., Li, G., Chen, L., 2009. Landfill site selection using spatial infor-mation technologies and AHP: a case study in Beijing, China. J. Environ. Manage.90, 2414e2421.

Wilson, F., Pan, W., Schumsky, D., 2012. Recalculation of the critical values forLawshe’s content validity ratio. Meas. Eval. Couns. Dev. 45, 197e210.

Wong, M., Wu, S., Deng, W., Yu, X., Luo, Q., Leung, A., Wong, C., Luksemburg, W.,Wong, A., 2007. Export of toxic chemicalsea review of the case of uncontrolledelectronic-waste recycling. Environ. Pollut. 149, 131e140.

Yang, J., Lu, B., Xu, C., 2008. WEEE flow and mitigating measures in China. WasteManage. 28, 1589e1597.