S08 - Hung et al, 2010

8/10/2019 S08 - Hung et al, 2010

1/11

Evaluating a national science and technology program using the human capitaland relational asset perspectives

Chia-Liang Hung a,*, Jerome Chih-Lung Chou b, Hung-Wei Roan a

a Department of Information Management, National Chi Nan University, 54561, Puli, Nantou county, Taiwanb Department of Information Management, Hwa-Hsia Institute of Technology, 23568, Taipei, Taiwan

1. Introduction

In the ever-changing 21st century, most industrialized coun-

tries allocate resources in pursuit of frontier technologies in order

to retain their economic leadership in the face of global

competition. Taiwan, the Silicon Island, has also invested large

sums in science and technology since 1998, through the initiation

of nine national science and technology programs (henceforth

abbreviated as NSTPs) in the diverse areas of telecommunications,

digital archives, digital learning, hazards mitigation, nanotechnol-

ogy, biotechnology and pharmaceuticals, agricultural biotechnol-

ogy, genomic medicine, and system on chip (NSC, 2007a).

Specifically, the NT$24 billion (about US$0.73 billion) budget of

the Taiwan National Telecommunication Program (henceforth

abbreviated as NTP) accounts for 22.2% of the total NT$108 billion

(about US$3.3 billion) in NSTP investments. Fig. 1shows the NTP

organizational chart, which represents the funding flows mainly

coming from four governmental ministries and one government-

based telecom corporation; the supervision direction is coordinat-

ed by a special program committee. In reality, most research

projects are organized and conducted in the universities and

several national-level R&D institutes. Other NSTP programs were

also orchestratedby similar structures. However,in thecase of NTP

outcome, few Taiwan telecom vendors have managed to dominate

the global telecommunications industry for an extended period

outside of Nokia (Finland), Ericsson (Sweden), Qualcomm and

Motorola (USA), DoCoMo (Japan), and Samsung (Korea). Although

the number of patents sponsored by NTP through 19982006

totalled to 925(NSC,2007b), only 225of thesepatents were closely

related to telecommunications technology (i.e., those of the H04

group by the USPTO classification scheme), implying a mere 27.6%

of the total number of NTP patents. Even worse, the patent impact

of NTP has generally been low. According to the patent impact

identification method of Thomas and Breitzman (2006), hot

Evaluation and Program Planning 33 (2010) 487497

A R T I C L E I N F O

Article history:

Received 31 August 2009Received in revised form 28 January 2010

Accepted 31 January 2010

A B S T R A C T

The purpose of this research is to evaluate the performance of the National Science and Technology

Program (NSTP) by targeting the Taiwan National Telecommunication Program (NTP) initiated in 1998.

The Taiwan telecommunications industry has prospered, currently occupying key positions in global

markets even though NTP seldom contributes positively to patent citation performance. Hence, the

authors of this study investigate the qualitative perspective of intellectual capital rather than

quantitative technological indices. The currentstudyfocuses on both human capitaland relational assets

through surveys of 53 principal investigators of NTP projects and 63 industrial R&D managers of

telecommunicationscorporations in the Taiwan market. Resultsshow thatNSTP memberquality andthe

flow of employment are good indicators of human capital and that both perform better than the middle

value in the case of Taiwan NTP. In addition, we find that industrial participants are more likely to share

R&D resources than other academic researchers with higher intention of co-publishing, co-funding, and

sharing equipments and facilities. The industrial NTP participants also have higher expectations

regarding achieving advanced technology breakthroughs in contrast to non-NTP industrial interviewees.

Moreover, industrial participants with greaterindustryuniversity cooperation intensityindeed obtain a

particular advantage, that is, greater knowledge acquisition from other fields related to the effect of

knowledge spillovers through the particular NSTP linkage. Accordingly, from the perspectives of human

capital and relational assets, the authors conclude by articulating the importance of absorptive capacityresulting from good human capital and knowledge spillover contributed by relational assets within

governmental technology policy and NSTP programming.

2010 Elsevier Ltd. All rights reserved.

* Corresponding author. Tel.: +886 49 2910960x4640; fax: +886 49 2915205.

E-mail address: c [email protected](C.-L. Hung).

Contents lists available atScienceDirect

Evaluation and Program Planning

j o u r n a l h o m e p a g e : w w w . e l s e v i e r . c o m / l o c a t e / e v a l p r o g p l a n

0149-7189/$ see front matter 2010 Elsevier Ltd. All rights reserved.

doi:10.1016/j.evalprogplan.2010.01.003
mailto:[email protected]://www.sciencedirect.com/science/journal/01497189http://dx.doi.org/10.1016/j.evalprogplan.2010.01.003http://dx.doi.org/10.1016/j.evalprogplan.2010.01.003http://www.sciencedirect.com/science/journal/01497189mailto:[email protected]

8/10/2019 S08 - Hung et al, 2010

2/11

patents with high impacts usually have a minimum of 10 recentcitations. There have only been 12 such hot NTP patents, mostly

produced by the Industrial Technology Research Institute (ITRI)

and the Institute for Information Industry (III) of Taiwan, both of

which are Taiwan government-sponsored research organizations.

In fact, over 73% of NTP participants had low-impact patents with

zero citations. However,the real performance of NTPshould not be

solely evaluated based on the perspective of number of patents

issued or hot patents, because Taiwan ICT manufacturers continue

to occupy important positions in the global marketplace. For

example, Taiwan is the largest wafer foundry, CD, DVD, and Mask

ROM production center, and second largest in terms of IC design,

WLAN, xDSL CPE, Cable CPE, and routers (NSC, 2007a). Therefore, a

robust measurement of NTP performance should include other

dimensions such as the quality of human resources trained at NTPand the innovation and entrepreneurship resulting from the

relational assets within the industry-university cooperation

network. The purpose of this paper is to discover the real value

of the NSTPs by constructing a non-patent evaluation framework

associated with the case of NTP.

2. Evaluation framework

Lepori (2006) points out that the national R&Dbudget forNSTPs

often plays the role of input indicator for future science and

technology performance. Hence, the Organization for Economic

Cooperation Development developed a Frascati Manual that

clearly identifies several key R&D sources (OECD, 2002). However,

verifying the expenditure flow, especially the flows to researchpersonnel who actually embody the output knowledge, is even

more vital than focusing on R&D input (Stewart, 1997). If the

nationalR&D input flows mostly to education, training, instruction,

and mentoring of researchers, then growth in the capability for

technology innovation and exploitation will be expected. Stewart

(1997) argues that R&D input in research personnel as an

intellectual investment will create the following three types of

future competitive advantage: embodied human capital, internal

structural assets, and external customer assets. Bontis (1998)

further specifies that the third type of competitive advantage is

both the most sustainable and the most difficult to imitate,

followed by embodied human capital. Therefore, these two types of

competitive advantages will be the focus of our assessment of the

NSTP evaluation framework.

2.1. Perspective of human capital

Lanzi (2007) measured the quality of research human capital

based on the three elements of basic skill, professional capability,

and continuous learning abilities.Wu (2006)also emphasizes that

the real quality of research personnel is often reflected in job

salary, job status, and problem solving capability. It is important to

considerthe absorptive capability of human resources in order to

be better equipped to acquire knowledge and raise the possibility

of invention in the future (Cohen & Levinthal, 1990).Kundu and

Kumar (2006)claim that special industrial training and develop-

ment programs actually help employees achieve corporate goals

and personal career objectives as well as improve their productiv-

ity. Mann and Robertson (2006) further suggest that training

evaluation programs should not focus only on training satisfactionand planned behavior but also on the enhancement of employees

learning capabilities and organizational contribution aftertraining.

In addition, the notion of perceived high quality of human

capital also requires the appreciation of demanding employers

(Cozzens, 1997). A correct match that does not waste the talent

pool can only perform to the standards of the actual quality of

human capital. Therefore, Luwel (2005) finds that theflow of talent

is a good indication of knowledge diffusion and industrial

development resulting from national programs. It is important

for most newly industrialized or less-developed countries to focus

their limited national resources on diffusion-oriented technology

policy for expertise training and knowledge diffusion widely

(Chiang, 1990). According to Callon, Laredo, and Mustar (1997),

intermediaries who can transfer, translate, and disseminateinformation to bridge the poles of science, technology, and market

are also required throughout in the value exploitation process of

innovation. Further, Wu (2006)emphasizes that the appropriate

flow of talent toward emerging industries can only contribute to

one of the national program objectivesindustrial enhancement

and upgrading.

2.2. Perspective of relational assets

Beyond measuring the quality and flow of human capital from

NSTP, the relational asset thatBontis (1998)specifies as the most

importantadvantage in terms of the external customerasset is also

a useful indicator of NSTP research personnel performance. Dalpe

(1993) and Georghiou (1998) both propose that the number of

Fig. 1. Taiwan NTP organizational chart and fundingsupervision relationship.

C.-L. Hung et al./ Evaluation and Program Planning 33 (2010) 487497488

8/10/2019 S08 - Hung et al, 2010

3/11

publishing co-authors, theamount of mutual R&Dfunding, and the

intensity of sharing equipment and research data are possible

indicators for measuring the potential of relational assets.

Callon et al. (1997)argue that a technology-economic network

modelis necessaryto meet the emerging challenges when bringing

together scientists from different fields, engineers, marketing and

financial agents, and end user representatives. According to

Bozeman, Dietz, and Gaughan (2001), scientific and technical

human capital includes both human capital endowments, such as

formal education and training, and social relations and network

ties that bind scientists and the users of science together. In fact,the latter argument ofBozeman et al. (2001) articulated that the

potential assets value resulting from specific cooperative relation-

ships range from science and technology research to market

research. Consequently, the concept of knowledge value alliance

presents a framework as an alternative focus for thoroughly

understanding and evaluating scientific and technical work

(Rogers & Bozeman, 2001). In fact, industrial participants usually

place greater emphasis on the development of relational assets

through connections to the knowledge alliance network of

government-sponsored technology programs. Sakakibara (1997)

argues that the ex ante perceived benefits will determine the

degree of involvement within the NSTPs; in contrast, the ex post

participation gains will influence the possibility of further

realization of relational assets woven among the industry

university cooperation networks.

As a result, the current authors elaboratethe NSTP performance

evaluation framework from the perspective of intellectual assets,

namely, human capital and relational assets (Fig. 2). In this

framework, human capital is depicted through two groups of

subindices: one group measuring the quality of research personnel

as judged by industrial employers, and the other relating to the

fitness of the talent flow to the optimum position as evaluated by

NSTP project investigators. In addition, the authors divide

relational assets into two groups, one pertaining to the degree

of R&D resource sharing within the NSTP, while the other to the

perceived participation benefits associated with NSTP.

3. Research design

3.1. Development of measurement items

According to the proposed NSTP evaluation framework, the

authors developed four constructs of performance measurement

as shown in Table 1. The first construct measures the quality ofR&D talent. Lanzi (2007) and Kundu and Kumar (2006) both

suggest that a higher level of willingness of the firm to expend

resources on recruitment, salary, and opportunities for promotion

implies a higher quality of employees.Cohen and Levinthal (1990)

and Mann and Robertson (2006) propose that highly talented

individuals reflect higher performance and higher capability (or

absorptive capacity) to acquire knowledge and achieve organiza-

tional objectives. Wu (2006) argues that highly talented people are

usually assigned to R&D-oriented tasks, management of innova-

tion, or organizational problem solving.

Fig. 2. The NSTP evaluation framework.

Table 1

Measurement items of the NSTP evaluation framework.

Measurement items Literature

Human capital Quality of R&D talent Recruitment willingness Lanzi (2007)

Salary level Kundu and Kumar (2006)

Promotion opportunity Kundu and Kumar (2006)

Performance

Knowledge acquisition capability Cohen and Levinthal (1990),

Mann and Robertson (2006)Achievement of organization objectives

R&D-oriented tasks

Innovation management Wu (2006)

Organizational problem solving

Flow of R&D talents Personnel spillover into domestic industries Luwel (2005),Chiang (1990)

Personnel spillover into NSTP-related technological fields

Expertise imported from NSTP Cozzens (1997)

External brokers of expertise Callon et al. (1997)

Relat ional assets R&D resources sharing C o-p ub lish ing t he research r esu lt s Dalpe (1993), Georghiou (1998)

Cooperation for commercialization

Licensing between participants

Technical meetings and conferences Callon et al. (1997)

Reorganization for new research agendas Brown (1997)

Research co-funding

Experimental data sharing

Sharing equipment and facilities

Perceived participation benefits Researcher training Sakakibara (1997)

Breakthrough in a critical technology

Increasing awareness on the importance of R&D

Accelerated development of the technology

Decrease in private R&D budget

Acquisition of knowledge from other participants Rogers and Bozeman (2001),

Bozeman et al. (2001)Establishment of an ongoing informational network

Commercialization of a product or process

Increase in the number of patent applications

A superior competitive position

Establishment of a standard in a target industry

C.-L. Hung et al. / Evaluation and Program Planning 33 (2010) 487497 489

8/10/2019 S08 - Hung et al, 2010

4/11

The second construct concerns the measurement items of NSTP

R&D talent pertaining to industrial demanders. Luwel (2005)

identifies indicators from the supply side of expertise and lays

emphasis on the R&D personnel flowing into the domestic

industries and working in NSTP-related technological fields. The

trained expertise matching the demand of industrial development

is often thekey purpose of diffusion-based technology policyin the

newly industrialized countries (Chiang, 1990). Contrastingly,

Cozzens (1997) argues that the R&D talent flows standing on

the viewpoint of the demand side, and observes the degree of

demand for external expertise and talent brokers/hunters, along

with the recruitment willingness of industrial demanders, which is

also an important mechanism to facilitate further knowledge

integration for technology commercialization (Callon et al., 1997).

The third construct relates to the measurement items regarding

R&D resource sharing. Georghiou (1998) identifies as pertinent the

amount of licensing between NSTP participants, the number of

research meetings and conferences, an ongoing channel for sharing

research experience and innovative ideas, and the possibility of

reorganizing research agendas based on continuous cooperation.

Further, Dalpe (1993)emphasizes the intensity of co-funding, co-

publishing, and commercializing behaviors, while Brown (1997)

and Callon et al. (1997) focus on the establishment of sharing

mechanisms pertaining to the periodic communique, experimen-tal data, work-in-process output, and equipment and facilities. The

fourth construct concerns the participation incentive in terms of

perceived participation benefits, articulated predominantly by

Sakakibara (1997)and supported byRogers and Bozeman (2001)

and Bozeman et al. (2001) in the knowledge acquisition and

information linkage.

Correct sampling of respondents is necessary for obtaining valid

data that can verify the measurement construct. Therefore, this

study focused on the following two sample sources of respondents:

NSTP project investigators with regard to the construct of R&D

resource sharing and the supply side of R&D talent flow, and

industrial participants involved in the NSTP in terms of measuring

all constructs other than the supply side of R&D talent flow. In this

study, the authors apply the case of Taiwan NTP for furthervalidation of the measurement constructs.

3.2. Research hypotheses

Apart from measuring the quantity of researchers,Lanzi (2007)

argues that researcher quality should also be considered as an

indicator of NSTP performance. A quality of NSTP manpower

higher than that of non-NSTP manpower may be attributable to

particular NSTP training and development programs. Further, Wu

(2006) elaborates on human quality within the socio-economic

dimension. Talent trained by NSTP will be often given more

challenging tasks. Kundu and Kumar (2006) also argue that

talented people trained by NSTP are usually expected by industrial

employers to have greater capabilities in terms of achievingcorporate objectives. As a result, high performance NSTP research-

ers may receive higher salaries and more opportunity for

promotion. Hence, the authors articulate the following hypothesis

to reflect the human quality of NSTP performance:

H1. From the perspective of human capital, NSTP performance is

positively correlated withhigher quality of R&D talent as evaluated

by industrial employers.

According toCozzens (1997), the knowledge pool contributed

by government-sponsored NSTPs is an important supply channel

for aggregating talented people in technological fields where

demand for them exists. Wu (2006) emphasizes the allocation

efficiency of human resources and claims that the fit of education,

training, and development of researchers to industrial require-

ments is critical to NSTP achievements for speeding the creation of

the new industry. In addition, Luwel (2005) observes that

researcher flow into a specific industry implies the intended

diffusion of technological knowledge, which is usually the main

purpose for intervention by the government to drive industry

upgrades and enhancement as supported by governmental NSTPs.

Hence, the authors articulate the following hypothesis to evaluate

the fit of the talent flow:

H2. From the perspective of human capital, NSTP performance is

positively correlated with the appropriate flow of talent to the

targeted technology.

Moreover,Dalpe (1993)observed obstacles that hinder univer-

sities and industrial corporations in terms of deterring information

exchanges, the former emphasizing academic publishing and the

latter,commercializationand market positioning.If NSTPs establish

cooperative mechanisms that invite greater research communica-

tion and resource sharing among participants, the cooperative

relationship will promote the effect of technology spillover and

thereby stimulate emerging businesses. Georghiou (1998) argues

that good cooperative experience betweenuniversities and industry

implies a good fit of research topics, information, results, andpartnerships. Moreover, such special linkages are often a valuable

relational asset in commercializing frontier technologies or even

acceleratingnew business ventures through spinoffs, joint ventures,

or strategic alliances. Hence, the authors propose the following

hypothesis to reflect the value of NSTP relational assets:

H3. From theperspective of relational assets, NSTP performance is

positively correlated with the higher incentives associated with

R&D resource sharing between NSTP participants.

Sakakibara (1997)argues that the high perceived participation

value of involvement with NSTPs induces industrial players to

accept invitations of NSTP and co-fund industryuniversity

cooperative projects. Moreover, increasingly small gaps between

the ex ante perceived values and ex postactual gains in otherwords, higher satisfaction on the part of project participants

extend the relationship and increase the opportunity for future

cooperation. A comparison of the participation before-and-after

gap is a necessary aspect of the questionnaire design. Hence, the

authors articulate two hypotheses, H4 and H5, to reflect the value

of relational assets:


positively correlated with higher perceived benefit resulting from

participation in the NSTP.


positively correlated with higher satisfaction resulting from coop-

eration with the NSTP.

Hypotheses H1H4 will be verified using factor analysis in

order to ensure measurement validity for the two constructs,

human capital and relational assets. Then, the researchers will use

the t-test approach to compare the mean value of each

measurement item with the middle value of the measurement

scale in order to verify whether NSTPs do pursue a higher level of

performance. The middlevalueof this surveyis thevalue of four for

the Likert-type measurement scale ranging from one to seven in

the research questionnaire. For hypothesis H5, the authors will use

a paired t-testcomparison to verify whether industrial participants

have a before-and-after gap, so as to reflect the satisfaction

associated with participating in NSTP and to predict the

industrialization or commercialization potential of future collabo-

ration based on the particular relational linkage.


8/10/2019 S08 - Hung et al, 2010

5/11

3.3. Data collection

First, the authors designed a questionnaire consisting of all

measurement items associated with the NSTP evaluation frame-

work as Table 1. Then all the questions were pre-tested by the

professors involved in the NTP of Taiwan. The revised version is

shown as Table A-1 in the Appendix. An identical web-based

questionnaire was also developed using Microsoft Access, ASP, and

FrontPage software tools. Next, the authors sent the paper-based

questionnaire as well as theURL of the web-basedquestionnaire to

231 NTP project investigators and 250 key industrial telecommu-

nications players, including telecom equipment and service

vendors in Taiwan. The respondents were free to complete either

version of the questionnaire; those who completed the paper-

based version mailed their responses, while completed online

versions were directly e-mailed to the researchers. The collection

process lasted from May to June 2008. The total number of valid

responses included 63 industrial players and 53 NTP project

investigators from universities or research organizations, with

respective response rates of 26.8% and 23.4%. Most of the 63

industrial players were telecommunication manufacturers (53

respondents), while only 31 industrial respondents had partici-

pated in theNTP industryuniversity collaboration. A profile of the

respondents is given inTable 2.When faced with a low responserate during data collection, the

researchers made several attempts of follow-up tracing and called

for responses of academic and industrial interviewees. However,

they were not obligated to answer the questionnaire. In particular,

some academic professors replied that they were only responsible

to the program funders, National Science Council of Taiwan, and

not to unrelated bystanders or other researchers. They were rather

reluctant to be measured. Similarly, several industrial interviewees

refused to answer for fear of leaking the firms recruitment policy

and the industrial cooperative relationship. However, the authors

further checked the respondent profile and found that most

academic responses were provided by main top-tier universities

such as National Taiwan University, National Chiao-Tung Univer-

sity, National Tsing-Hua University, National Cheng-Kung Univer-sity, and National Central University. On the other hand, the

industrial respondents ranged from the most important national-

level R&D organization in Taiwan, namely, Industrial Technology

Research Institute, to top equipment manufacturers of Internetand

Wi-Fi, key mobile phone OEM/ODMvendors, and five major mobile

network operators, who had been enthusiastic supporters of the

M-Taiwan mobile application program, a sub-program of NTP. In

addition, the surveyed telecom service providers, that is, the

mobile operators and ISP providers, are usually bigger than the

telecommunication manufacturers. The latter have been horizon-

tally disintegrated into multiple specialized vendors from chip

designing, fabrication and packaging to testing, assembling, and

firmware, since the global telecommunication liberalization

(Steinbock, 2002). Thus, the unbalanced size of respondentsbetween service and manufacturing sector is inevitable. As a

result, the sampling size and the number of service responses are

both comparatively smaller than those of the manufacturing

sector. Hence, the authors judged the collected data for further

analysis by their representativeness even though the response rate

was unexpectedly low.

4. Data analysis and discussion

4.1. Validity of measurement

The authors conducted a factor analysis using the principal

component method with Varimax rotation to verify the measure-

ment validity (Sharma, 2005). The SPSS 13.0 statistical package

was used. The KMO (Kaiser-Meyer-Olkin) test demonstrates the

value of 0.525 and 0.752 for human capital and relational assets

respectively, which both show the acceptable level of sampling

data for factoring (Sharma, 2005). In addition, the Bartletts test

also reveals significantly high correlations greater than zero

(p < .0000) between measurement items, implying the correlation

matrix of present data is appropriate for factoring (Sharma, 2005).

Table 3shows the significant classification of four factors. The two

factors in the upper half ofTable 3indicating the quality and flow

of R&D talent explain 52.69% of the measurement variance of

human capital, while the other two factors in the lower half of

Table 3 pertaining to R&D resource sharing and perceived

participation benefits reflect 61.42% of the variance associated

with measuring relational assets. Furthermore, the authors appliedthe SAS Factor procedure for residual correlations analysis to

assess the estimated factor solutions. The derived overall RMSE

(Root-Mean-Square Errors) values are 0.098 and 0.084 for human

capital and relational assets respectively. Thus, the estimated

factor models can be considered to be appropriate due to small

RMSE values (Sharma, 2005). The results indeed fit the four

measurement constructs of NSTP performance evaluation elabo-

rated by the authors.

4.2. Evaluation of human capital

The authors then applied the human capital construct to

evaluate the performance of the NTP case. The t-test method tests

each measurement item to discover whether themean exceeds themedian value. Table 4 shows the t-test results. All the measure-

ments pertaining to human capital significantly exceeded the

median value, except the item associated with external brokers of

expertise. This may indicate that the employment matching

process/talent brokerage instituted by the NTP must, from the

industrial viewpoint, be improved. Nevertheless, the quality of

R&D talent as perceived by industrial demanders is apparently

high enough to be a high priority. To induce NTP-trained talent to

flow into telecommunications jobs within the domestic industry,

higher salaries and greater promotion opportunities are offered,

and more challenging tasks assigned.Further,this higherquality of

employment reflects the strategic purposes of governmental

support for national-level science and technology research. It also

reflects the point of the absorptive capability of human resourcesin order to be better equipped to scan the technology frontier,

facilitate international technology transfers, and raise the possi-

bility of invention in the future (Cohen & Levinthal, 1990).

Therefore, the data analysis supports hypotheses H1 and H2

(mostly).

The authors further investigated whether the different types of

industrial firms differ in terms of their quality perceptions of

talented individuals. As shown in Table 5, NTP industrial

participants generally had higher demand for importing tele-

com-related expertise from NTP than from non-NTP participants.

Further, industrial telecommunication demanders were classified

into manufacturers, such as equipment vendors, and service

providers, such as telecom operators, after which the effects of

each industrial role were compared. The sole significant difference

Table 2

Profile of respondents.

Classification of respondents Ratio (%)

Academic researchers vs. industrial players 46:54

Manufacturers vs. service providers 84:16

NTP vs. non-NTP participants 49:51

Awareness vs. unawareness of NTP 83:17

Recruiting vs. non-recruiting from NTP 49:51

In-sourcing vs. outsourcing expertise 48:52


8/10/2019 S08 - Hung et al, 2010

6/11

Table 3

Factor analysis of measurements.

Measurement items Factor 1 Factor 2

Human capital Qual ity of R&D tale nt !Recruitment willingness 0.21743 0.13278

!Salary level 0.53171 0.01397

!Promotion opportunity 0.94296 0.00516

!Performance 0.88577 0.08467

!Knowledge acquisition capability 0.79836 0.05305

!Achievement of organization objectives 0.75323 0.00363

!

R&D-oriented tasks 0.66741 0.35654! Innovation management 0.82513 0.04334

!Organizational problem solving 0.73197 0.12184

Flow of R&D talent !Personnel spillover into domestic industries 0.33276 0.52349

!Personnel spillover into NSTP-related technological fields 0.14367 0.79101

!Expertise imported from NSTP 0.11469 0.65244

!External brokers of expertise 0.23114 0.53257

l= 5.0698 1.7795

Cumulative= 0.5269

Relat ional assets R&D resour ce sh aring !Co-publishing the research results 0.01380 0.84081

!Cooperation for commercialization 0.06053 0.79822

!Licensing between participants 0.09704 0.84883

!Technical meetings and conferences 0.01283 0.91801

!Reorganization for new research agendas 0.08755 0.51310

!Research co-funding 0.12238 0.70920

!Experimental data sharing 0.09648 0.86604

!Sharing equipment and facilities 0.07005 0.73502

Perceived participation benefits !Researcher training 0.75682 0.12574!Breakthrough in a critical technology 0.80906 0.01510

! Increasing awareness on the importance of R&D 0.74163 0.01649

!Accelerated development of the technology 0.63615 0.03476

!Decrease in private R&D budget 0.72444 0.16188

!Acquisition of knowledge from other participants 0.55589 0.05792

!Establishment of an ongoing informational network 0.79688 0.10719

!Commercialization of a product or process 0.83355 0.00612

! Increase in the number of patent applications 0.83901 0.06382

!A superior competitive position 0.88560 0.04610

!Establishment of a standard in a target industry 0.86924 0.02878

l= 6.6521 5.0173

Cumulative= 0.6142

Table 4Significance of human capital performance.

Measurement items Mean t-value+ Hypothesis

Quality of R&D talents !Recruitment willingness 6.0 16.70** H1 (supported)

!Salary level 5.1 4.89**

!Promotion opportunity 5.4 6.78**

!Performance 5.1 5.02**

!Knowledge acquisition capability 5.6 8.32**

!Achievement of organization objectives 5.2 5.33**

!R&D-oriented tasks 4.9 3.59**

! Innovation management 5.3 6.07**

!Organizational problem solving 5.1 4.25**

Flow of R&D talents !Personnel spillover into domestic industries 6.0 17.02** H2 (mostly supported)

!Personnel spi llover into NST P-related technologi cal fields 4.7 2.2 3*

!Expertise imported from NSTP 4.3 2.04*

!External brokers of expertise 3.0 4.81**

* p

8/10/2019 S08 - Hung et al, 2010

7/11

is shown in Table 5, where both industrial participants, whether or

not they had ever worked with the NTP, evaluated the quality of

NTP talents equally, exclusive of salary and promotion opportu-

nities. In reality, according to the information from NTP project

investigators, most of the trained talent flows into the manufactur-ing sector (manufacturing vs. service: 61.3% vs. 38.7%), even

though bothtelecom service providers and manufacturers demand

expertise from NTPequally, andbothstated theirintention to place

NTP-trained talent in challenging positions such as R&D, innova-

tion, and organizational problem solving. Manufacturing firms

generally provide higher salaries and more promotion opportu-

nities. This stronger incentive appears to explain why the majority

of trained talent flows into the manufacturing sector and

contributes to Taiwans flourishing global share of broadband

device manufacturing.

4.3. Evaluation of relational assets

In the same way, the authors adopted the verified measurementconstruct of relational assets to evaluate the performance of the

NTP case andalso employed t-tests to determine whether the score

for each measurement item exceeded the middle value. Table 6

shows the statistical results. Only the perceived participation

benefit measurements significantly exceeded the median value.

Therefore, hypothesis H4, which argues that greater perceivedparticipation benefits enhance the value of relational assets within

the NSTP, is supported. Each of the measurement items regarding

participation benefits was significant. Further, these results

indicate that the spillover effect remains a top priority for

government-sponsored research programs aimed at technology

diffusion and industry position upgrading, as found in Georghiou

(1998). Contrastingly, the sharing of R&D resources between NTP

partners was non-significant. This unusual finding may be due to

the fact that nearly half of the data was collected from university

professors, who primarily receive government-sponsored research

grants and often concentrate on academic publishing rather than

diffusion of technology to industry. For this reason, the authors

further scrutinized the questionnaire data and only verified the

significance of 31 industrial responses from those who hadpreviously participated in the NTP; these results are shown in

Table 6

Significance of relational assets performance.

Measurement items Mean t-value+ Hypothesis

R&D resource sharing !Co-publishing the research results 3.8 0.89

!Cooperation for commercialization 3.4 1.99

!Licensing between participants 3.5 1.73

!Technical meetings and conferences 4.2 0.71

!Reorganization for new research agendas 4.4 1.52

!Research co-funding 3.1 3.16

!

Experimental data sharing 4.2 1.00!Sharing equipment and facilities 3.2 3.24

Perceived participation benefits !Researcher training 5.1 7.15** H4 (supported)

!Breakthrough in a critical technology 5.2 8.75**

! Increasing awareness on the importance of R&D 5.8 13.00**

!Accelerated development of the technology 5.0 5.08**

!Decrease in private R&D budget 5.5 9.64**

!Acquisition of knowledge from other participants 5.2 7.10**

!Establishment of an ongoing informational network 5.5 8.43**

!Commercialization of a product or process 5.3 7.58**

! Increase in the number of patent applications 5.5 8.43**

!A superior competitive position 5.8 12.34**

!Establishment of a standard in a target industry 5.5 10.4**

** p

8/10/2019 S08 - Hung et al, 2010

8/11

Table 7. The sharing of R&D resources between NTP partners from

the perspective of NTP industrial participants revealed significance

only pertaining to the measurement items of co-funding, co-

publishing, and sharing of equipment and facilities. As a result, the

findings partially support hypothesis H3.

Additionally, the authors investigated whether the different

types of industrial participants differed in terms of their perceived

NTP participation benefits. The results are shown inTable 8. Thet-

test verification shows that the perceived possibility of a

breakthrough in critical technology is one significant inducer for

NTP participants because of higher expected benefits. If we focus

only on NTP participants, the before-and-after paired comparisons

of gaps between the expected benefits and actual gains were non-

significant except for the inadequacy of accelerated technologydevelopment. This implies that NTP participants generally

experience a satisfactory level of cooperation with partners from

university professors or researchers of national R&D organizations

as they strive to achieve advanced telecommunications technolo-

gies. Hence, hypothesis H5 is also supported. However, achieve-

ment in the accelerated development of technology was lower

than we expected. This situationmay have been caused by a lack of

common focus in the universityindustry cooperation; professors

in Taiwan universities usually concentrate on the time to publish,

while industrial participants emphasize the speed to commercial-

ization for global competition. Further, the research found that

those industrial participants who engaged in a higher degree of

R&D resource sharing actually experience greater acquisition of

knowledge from other participants. In addition, another t-testcomparison also revealed that industrial manufacturing roles are

associated with higher benefits from knowledge acquisition than

telecom service providers. The capability of knowledge acquisition

indeed enhances the new product development in integrating

complements and skills (Mohr, Sengupta, & Slater, 2005). Hence,

this may indicate that NTP performance facilitates Taiwan telecom

manufacturers to quickly configure all telecom systems and thus

rapidly supply mobile devices and broadband equipments to

markets, which in turn improves their global standing.

4.4. Further data exploration among measurement constructs

Finally, the authors explored the relationship between the NSTP

performance evaluations. Significant and meaningful findings are

shown inTables 9 and 10. When focusing on the demand flow of

industrial participants, those who expressed greater willingness to

share R&D resources also indicated a greater demand for expertise

from NTP. By the same token, the t-test comparison for the supply

flow of R&D talent showed that academic researchers who share

more experimental data or licenses with NTP partners also offer

project-trained members more opportunities for employment

within telecom-related technological industries. Moreover, the

regression analysis inTable 10revealed some meaningful findings

between the perceived participation benefits and the quality of theNTP-trained R&D talent. Greater satisfaction in terms of increased

awareness of R&D importance was associated with greater

willingness to recruit NTP-trained talent. At the same time, greater

satisfaction in terms of the acquisition of knowledge from other

participants was associated with greater willingness to recruit

NTP-trained talent as well as a higher valuation for performance,

and greater likelihood of assigning challenging innovation

management tasks to new recruits. In addition, satisfaction with

the increase in the number of patent applications leads to

employees being evaluated higher in terms of achieving organiza-

tional objectives. Lastly, greater satisfaction with the commercial-

ization of a product or process leads to a greater possibility of

raising employee salaries due to high performance. Thus, the

higher salaries attract more R&D talent to the industry, especiallyto industrial employers who participate in the NSTP and

experience satisfaction through industry-university cooperation.

Accordingly, Table 11 summarizes the findings from verifica-

tions of and comparisons among the measurement constructs of

the NSTP evaluation framework. InTable 11, the arrows between

the measurement constructs indicate the potential influences.

Consequently, if the NSTPs embed more incentives for interaction

and sharing behaviors between project partners, especially among

industrial participants and university professors, the resulting

Table 9

Significance of relation between resource sharing and flow of R&D talent.

Resource sharing Flow of R&D talents Mean t-value+

!High- vs. low-degree of resource sharing by industrial NTP participants Demand flow: expertise imported from NSTP 4.52 vs. 3.96 2.187*

!High- vs. low-degree of Experimental data sharing by NTP project investigators Supply flow: personnel spillover into

NSTP-related technological fields

5.08 vs. 4.28 1.845*

!High- vs. low-degree of cross-licensing between industryuniversity cooperation 5.03 vs. 4.17 4.557**

* p

8/10/2019 S08 - Hung et al, 2010

9/11

trained human capital, enriched with sharing and knowledge

acquisition abilities, will be greatly demanded in the industry. It

appears that the NSTPs should promote circulation of knowledge

even though outside employment brokerages remain inadequate

when faced with highly immature and uncertain technology.

Moreover, if the NSTP office initiates coordinating mechanisms to

facilitate industry-university cooperation, participant satisfaction

may increase, and quality evaluations of NSTP-trained talent will

certainly increase. Consequently, the higher quality of human

capital will enhance the possibility of higher salaries during

recruitment and thereby contribute toward promoting NSTPexpertise across the industry. Hence, the important diffusion

purpose of most NSTP policies can be achieved successfully.

5. Conclusions

The current study adopts a perspective different from the

conventional quantitative approach of directly counting output in

terms of patent applications and citations to evaluate a researchand

development program. Using the perspectives of human capital and

relational assets, it explains how the Taiwan NTP is successful

despite its lack of outstanding technological patents. In this study,

the authors emphasize the intellectual capital on and between

the trained talents themselves. The quality and employment flow of

human capital trained by the NSTP is the first measurement of

concern pertaining to the on-talent dimension; the other is the

between-talent measurement dimension focusing on the relational

assets that can be attributed to sharing R&D resources and the

experience of participation benefits. The research in the NTP case

generally supports the on-talent aspect; the quality of the NTP-

trained human capitalis perceived better than themiddlevalue (the

indifferent point), which results in the manufacturing industry

offering higher salaries to NTP-trained individuals. NTP members

also effectively flow into the Taiwan telecommunications industry,

especially to the manufacturing sector. Yet, the authors show that

the construct of ex ante perceived participation benefits is moreeffective than the behavior of sharing R&D resources for evaluation

of relational assets. The measurement results of perceived

participation benefits indicate that NTP industrial participants are

more eager to achieve breakthroughs in critical technologies by

cooperating with academic researchers than participants not

involved in NTP. Even though most industrial NTP participants

eventually becomedisappointedwith the acceleratingdevelopment

of technology associated with commercial applications, they are

generally satisfied with the collaboration experience because of the

very small expectation gap pre- and post-participation. In addition,

industrial participants with higher industry-university cooperation

intensity gain a particular advantage, namely, greater knowledge

acquisition from various other fields due to knowledge spillover.

Taiwantelecommunications manufacturers stress the acquisition of

Table 11

Summary of findings.

Note: The arrows imply possible relationships between measurement constructs.


8/10/2019 S08 - Hung et al, 2010

10/11

knowledge from collaborative partners to improve and accelerate

the integration of telecommunications system products (e.g.,

wireless and broadband devices), and capture global markets. It is

thus reasonable that Taiwan telecommunications manufacturers

have a greater willingness to offer higher salaries and more

promotion opportunities to NTP-trained talents in contrast to

telecom service providers.

Accordingly, in terms of the national technology policy,

measuring performance according to the perspectives of human

capital and relational assets highlights the importance of the

increase in absorptive capacity and knowledge spillover contrib-

uted by NSTP, as opposed to the conventional quantitative method

thatconsiders patent analysis and scientificimpact. Thus, targeting

the performance criteria associated with human capital and

relational assets engendered by the government-sponsored NSTP

is more appropriate than investigating the technological output,

which only pertains to a few highly developed countries.

Furthermore, based on the disappointment with accelerating

application development of targeted technology among the

industrial participants, which corresponds with the results of

Sakakibara (1997)for the Japanese government-sponsored NSTP,

Taiwans NSTP program should not restrict itself to technological

breakthroughs, but should also consider the fields of marketing

research, regulation reform, infrastructure restructuring, andeducation as well as other fields that complement and nourish

application innovation and knowledge diffusion associated with

new technology. In fact, Georghiou (1998) suggests that the

coordination function is the most important criterion when

evaluating the performance of any scientific program. Hence, the

NSTP technology policy design should emphasize liaison mechan-

isms to harmonize the objectives of the participants, thereby

creating a space where heterogeneous relational alliances can be

formed between public and industrial laboratories, between

industrial competitors, or between firms with complementary

resources. Further, while the academic sector usually pursues

personal scientific achievements,the industrial sectorfocuses on the

potential for commercialization according to a strict cost-benefit

analysis. Hence, policies that can help to bridge this gap areimperative. For example, university researchers couldreceive career

compensation pertaining to their role in carrying out applied

research, or a tax reduction could be offered to industrial NSTP co-

funders for their support in pioneering research. In addition,

intermediary roles in marketing, public relations, and finance

should also be initiated based on market demand or new ventures

that attract the interests of industrial practitioners even when the

dominant purpose of NSTP focuses mainly on the technology

advancement. If carried out, NSTP participant satisfaction will

improve as recruitment of NSTP-trained members increases due to

their higher quality evaluations. Consequently, a coordination or

liaison mechanism installed in the NSTP that supports program

interaction will then support increasing returns in terms of

absorptive capability and technology spillover, thus efficientlyachieving the goal of a diffusion-oriented technology policy.

The limits of the current study may be found in the problem of

generalization. The targeting case, telecommunications, is an

applied technology area that lacks a strong component of basic

research, unlike nanotechnology or biotechnology. For future

research, broadening the sample space encompassing different

orientations of NSTP is necessary to justify and adjust the

measurement constructs articulated in this study.

Acknowledgements

The authors wish to acknowledge thefinancial support from the

BRIDGE project of National Science Council of Taiwan under grant

no. NSC-96-3114-P-260-001-Y.

Appendix A

Table A-1 Questionnaire of intellectual-based NSTP evaluation

framework.

Questions on quality of R&D talent (1 = not at all, 7 = very much)1. Does (Did) your company (ever) like to recruit the member ever

trained by National Telecommunication Program (NTP)?

2. Does (Did) your company (ever) give a higher level of salary to

the employee trained by NTP?3. Does (Did) your company (ever) give a higher promotion

opportunity to the employee trained by NTP?

4. Does (Did) your company (ever) get a higher level of

performance from the employee trained by NTP?

5. Do (Did) you (ever) think that the employee trained by NTP

possesses a higher level of knowledge acquisition capability?

6. Do(Did) you (ever) think thatthe employee trained by NTP has a

higher possibility of achieving the organization objectives?

7. Does (Did) your company (ever) assign a higher proportion of

R&D-oriented tasks to the employee trained by NTP?

8. Do (Did) you (ever) think that the employee trained by

NTP possesses a higher capability of managing innovation

process?

9. Do (Did) you (ever) think that the employee trained by NTP

possesses a higher capability to solve organizational problems?

Questions on flow of R&D talent (1 = not at all, 7 = very much)

10. Do you think that most of your NTP project members flowed

into the domestic industry?

11. Do you think that most of your NTP project members flowing

into the domestic industry targeted the telecom-related seg-

ments?

12. Does (Did) your company still (ever) has the demand to acquire

expertise from the NTP?

13. Does (Did) your company (ever) acquire telecommunication

expertise by the help of external agents?Questions on R&D resource sharing (1 = not at all, 7 = very much)

14. Does (Did) your NTP project (ever) co-publish the research

results with external research centers, university laboratories, or

companies?

15. Does (Did) your NTP project (ever) cooperate with external

research centers, university laboratories, or companies for

commercializing the new technology?

16. Does (Did) your NTP project (ever) license technological

patents mutually between the NTP participants?

17. Does (Did) your NTP project (ever) meet and discuss with

external research centers, university laboratories, or companies in

the telecom-related conference?

18. Does (Did) your NTP project (ever) reorganize a new researchagenda with external research centers, university laboratories, or

companies after the NTP cooperation?

19. Does (Did) your NTP project (ever) receive research funds

from external research centers, university laboratories, or

companies?

20. Does (Did) your NTP project (ever) share experimental data

with external research centers, university laboratories or compa-

nies?

21. Does (Did) your NTP project (ever) share research equipments

and facilities with external research centers, university laborato-

ries or companies?

Questions on perceived participation benefits (1 = not at all,

7 = very much)


8/10/2019 S08 - Hung et al, 2010

11/11

22. Is it an important purpose of participating into the NTP for

benefiting from better researcher training?


benefiting from earlier breakthrough in the critical technology?


benefiting from increasing awareness on the importance of

telecommunication R&D?


benefiting from accelerating application development of telecom-munication technology?


benefiting from subsiding research funds from public sector?


benefiting from acquiring/sharing knowledge from other partici-

pants?


benefiting from establishment of an ongoing informational

network with other participants including private and public

sectors?


benefiting from speeding the new product commercialization

process?30. Is it an important purpose of participating into the NTP for

benefiting from increasing the number of patent applications?


benefiting from obtaining a superior position in facing the

telecommunication competition?


benefiting from quicker establishing an industrial standard of new

telecommunication technology?

References

Bontis, N. (1998). Intellectualcapital:an exploratorystudy that develops measures andmodels.Management Decision, 36(2), 6376.

Bozeman, B., Dietz, J., & Gaughan, M. (2001). Scientific andtechnical human capital: analternative model for research evaluation. International Journal of TechnologyManagement, 22(7/8), 636655.

Brown, E. A. (1997). Measuring performance at the army research laboratory: theperformance evaluation construct. Journal of Technology Transfer, 22(2), 2126.

Callon, M., Laredo, P., & Mustar, P. (1997). Strategic management of research andtechnology: evaluation of programmes. Paris: Economica International.

Chiang, J. T. (1990). Management of national technology programs in a newly indus-trialized countryTaiwan. Technovation, 10(8), 531554.

Cohen, W. M., & Levinthal, D. A. (1990). Absorptive capacity: a new perspective onlearning and innovation. Administrative Science Quarterly, 35(1), 128152.

Cozzens, S. E. (1997). The knowledge pool: measurement challenges in evaluatingfundamental research programs. Evaluation and Program Planning, 20(1), 7789.

Dalpe, R. (1993). Evaluating the industrial relevance of public R&D laboratories. In B.Bozeman& J. Melkers (Eds.), Evaluating R&D impacts: methods and practice. Boston:Kluwer Academic Publishers.

Georghiou, L. (1998). Issues in the evaluation of innovation and technology policy.Evaluation, 4(1), 3751.

Kundu, S. C., & Kumar, R. (2006). Evaluating benefits of training and development: astudy of Indian and multinational companies. In Proceedings of 12th Asia Pacific

Management Conference (pp. 760768) [ISSN: 974-8257-30-4].Lanzi, D. (2007). Capabilities, human capital and education. Journal of Socio-Economics,36(3), 424435.

Lepori, B. (2006). Methodologies for the analysis of research funding and expenditure:from input to positioning indicators. Research Evaluation, 15(2), 133143.

Luwel, M. (2005). Job advertisements as an indicator for mobility of researchers:naturejobs as a case study.Research Evaluation, 14(1), 8092.

Mann, S., & Robertson, I. T. (2006). What should training evaluations evaluate? Journalof European Industrial Training, 20(9), 1420.

Mohr, J., Sengupta, S., & Slater, S. (2005). Marketing of high-technology products andinnovations(2nd ed.). Prentice-Hall.

NSC. (2007a). White paper on science and technology (20072010) . Taipei: NationalScience Council.

NSC. (2007b). 2007 Yearbook of science and technology of Taiwan. Taipei: NationalScience Council.

OECD (Organization for Economic Cooperation Development). (2002).Frascati manual:proposed standard practice for surveys on research and experimental development.Paris: OECD Publishing.

Rogers, J. D., & Bozeman, B. (2001). Knowledge value alliances: an alternative to the

R&D project focus in evaluation. Science, Technology & Human Values, 26(1), 2355.Sakakibara,M. (1997). Evaluating government-sponsored R&D consortia in Japan:who

benefits and how?Research Policy, 26(45), 447473.Sharma, S. (2005). Applied multivariate techniques. John Wiley & Sons Inc.Steinbock, D. (2002).Wireless horizon: strategy and competition in the worldwide mobile

marketplace. New York: AMA.Stewart, T. (1997). Intellectual capital: the new wealth of organizations. Doubleday

Publishing.Thomas, P., & Breitzman, A. (2006). A method for identifying hot patents and linking

them to government-funded scientific research. Research Evaluation, 15(2), 145152.

Wu, H. L. (2006). Economic analysis of developing technological human capitals bytechnology budget. Taipei: CIER.

Chia-Liang Hungis an assistant professor of department of information managementat National Chi Nan University in Taiwan. He holds a PhD from National TaiwanUniversityand now specializesin management of technology and strategy of electroniccommerce.

Jerome Chih-Lung Chou is an assistant professor of Department of InformationManagement at Hwa-Hsia Institute of Technology in Taiwan. He holds a PhD fromNational Taiwan University and now specializes in technology strategy and entre-preneurship of small business.

Hung-Wei Roanholds an MBA degree of National Chi Nan University in Taiwan. Hespecializes in designing and implementing web-based software applications.


Documents

S08 - Hung et al, 2010