2011 WIPO Evaluation Workshop, Geneva, Switzerland

“Learning from Existing

Evaluation Practices on the

Impacts and Effects of

Intellectual Property on

Development”

Geneva

6th/7th October

2011

Evaluation Section Internal Audit and Oversight Division

(IAOD)World Intellectual Property Organization (WIPO)

BREAKING the CULTURE of ‘FAKES and

COUNTERFEITS’:INFORMATION and EDUCATION PROGRAMS not CAUSING

RISE of INTELLECTUAL PROPERTY AWARENESS in the

PHILIPPINES?

Romeo B. Santos

WorkLand M&E Institute, Inc.

COUNTRY BACKGROUND

_____PHILIPPINES [PHL]____

POPULATION: 92 million

GDP per capita: 3,724

AREA: 299, 764 sq. m.

http://www.infoplease.com/atlas/country/philippines.html

Introduction

MAJOR LAWS, GLOBAL TREATIES AND

ORGANIZATIONS [PHL is MEMBER]

MAJOR IPR LAWS NAME YEAR

Republic Act 165 PATENT LAW 1947

Republic Act 166 TRADEMARK LAW 1947

Republic Act 8293INTELLECTUAL PROPERTY

CODE1998

IPR GLOBAL TREATIES /

ORGANIZATIONSCOVERAGE YEAR

BERNE CONVENTION LITERARY AND ARTISTIC 1951

PARIS CONVENTION INDUSTRIAL PROPERTY 1965

WIPO INTELLECTUAL PROPERTY 1968

ROME CONVENTION

PERFORMERS, PRODUCERS

AND PHONOGRAMS AND

BROADCASTING

ORGANIZATIONS

1984

TRIPS AGREEMENT TRADE 1995

PCT PATENTS 2001

WIPO COPYRIGHT TREATY COPYRIGHT 2002

WPPTPERFORMANCES AND

PHONOGRAMS2002

http://ph.news.yahoo.com/philippine-mayors-next-target-pirates-manila-district-111003534.html

http://ph.news.yahoo.com/anti-software-piracy-drive-nets-p4-3m-may-022218863.html

IntroductionCOUNTRY BACKGROUND

_____PHILIPPINES [PHL]____

STATE OF IPR IN THE

COUNTRY

USTR PRIORITY WATCH List until

2005

USTR WATCH List at PRESENT

Source: Business Software A lliance and the

Presidential Inter Agency Committee-

Intellectual Property Rights Executive

Summary for 1999-2001
















STATE OF IPR IN PHL

Has attained substantial

PROGRESS in the LEGAL &

INSTITUTIONAL ENVIRONMENTS

BUT in general:

Introduction

IMPROVING

DECLINING

STALLING

Software Piracy rate in the Asia-Pacific Region

1996-2003 [in percent]

SOURCE: Annual BSA Piracy Study, January 2003: First Annual BS and IDC

Introduction

Source: THE IP COALITION REPORT I: Copyright in the PHILIPPINES 2004

3 CORNERSTONES of IPR

TREATIES’

IMPLEMENTATION in the

Philippines. APPROACH

used by the Philippines in

its fight against PIRACY.

[PROGRAM

INTERVENTION

APPROACH]

The Total IP Protection Model, Philippines

Key Lessons

APPROACH TO EVALUATION

--Analyzed the PHILIPPINE GOVERNMENT IPR PROTECTION

INTERVENTION APPROACH used as basis of the

EVALUATION APPROACH

EVALUATION APPROACH INTERVENTION APPROACH

A: PUBLIC INFORMATION and EDUCATION

B: LEGAL and POLICY

C: ENFORCEMENT and ADJUDICATIOIN

3 CORNERSTONES of the PHILIPPINE IPR

TREARTIES IMPLEMENTATION

A B C

[ PROBLEM = EVALUATION CONTEXT ]

Key LessonsAPPROACH TO EVALUATION

EVALUATION APPROACH INTERVENTION APPROACH

THEORY OF CHANGE:

Recreating the LOGIC OF

CHANGE, the CAUSAL LINKS

between the PHILIPPINE IPR

PROGRAM INTERVENTION and

desired RESULTS.EFFECTS

A B C

OUTCOMES

IMPACTS

EVALUATION APPROACH = EVALUATION DESIGN

THIS REPORT

FOCUS

OF THIS

REPORT

The EVALUATION PROJECT

APPROACH TO EVALUATION

Key Lessons

METHODOLOGY

GETTING the DATA

DOCUMENTS REVIEW

SURVEY

STRUCTURED INTERVIEWS [KRPI]

CS, FGD

TRANSECTS

Key Lessons

ANALYSISTRIANGULATE

EVALUATION

CONTEXT KEY FINDINGSRECOMMENDATIONS

LESSONS LEARNED

Re-createTHEORY

OF CHANGE

DESIGN MATRIXEVAL Qs; RESEARCH

DESIGN

METHODOLOGY THEORY OF CHANGE

Key Lessons

INPUTS ACTIVITIES OUTPUTSRESULTS:

OUTCOMES

RESULTS:

IMPACTS

IPR

trainings &

educational

programs

Budgets

Personnel

Systems&

Procedures

Other

resources

Hold

education,

training and

publicity

events

Enact

legislations

Establish

institutions

Enforce laws

Operate IPR

system

People

/Sectors

trained on IPR

Advocacy &

educ.

programs held

Laws enacted

Policies

implemented

Laws enforced

P, U, D, TM,

®, © granted

Raised people’s

awareness measured

by their

understanding of the

importance of IPR

Increased IPR

responsible practices

among individuals,

sectors and communities

Increased peoples’

IPR supportive

activities measured

by reduced violations

and IPR system

applications/awards

Sustained anti -

infringement culture

among the public

Key Lessons

QUESTION* SUB-QUESTION TYPEMeasures &

IndicatorsTarget Baseline

1. To what extent

has the PHL IPR

Treaties’

Implementation

Program

[PROGRAM]

increased the

awareness on IPR

among Filipinos?

What PROGRAM

component [PIEP,

PRP, EAP]? What

audience [schools,

professionals,

SMES, others]?

Descriptive

Number of

respondents per

audience type

NA None

What extent among

people in general?Descriptive

Number of

respondentsNA None

Design

Data Sources

Sample

Data Collection Instrument

Data Analysis

One shot Survey/ Interviews/ Transects

Program records

‘Inventory’ of all IPR orgs in PHL

Record Retrieval

Frequency count

Comments Graphic

METHODOLOGY KEY EVALUATION QUESTIONS/DESIGN MATRIX

* Only Selected KEY EVALUATION QUESTIONS are presented.

QUESTION SUB-QUESTION TYPEMeasures &


2. To what extent do

PROGRAM

components

harmonize with

objectives of WIPO &

other conventions?

Do the program

components align with

WIPO & other

conventions? Descriptive/

Norm

Check of

alignment,

conformity

with WIPO,

etc.

NA None

3. What specific

strategies have

generated the highest

responses and

actions among

recipients & the public

in supporting IPR?

Are the strategies

consistent with

international

practices? Descriptive/

Norm

Check of

consistency

with WIPO,

etc.NA None

Design: Record & document review; KRPI


Key Lessons



4. How effectively have the

PHL IPR systems been

utilized and how have

these contributed to the

results?

How effectively has

the coordination

mechanism among

IPR entities

contributed to the

achievement of

results?

Explanatory

Number of

cases;

Attributes of

Instance

Yes NA

5. What are the underlying

factors that explain

barriers to knowledge and

appreciation of IPR among

the public?

Which categories of

stakeholders were

found to be most

committed to

understanding and

applying IPR

responsible

practices?

Descriptive/

Explanatory

Number of

cases;

Attributes of

Instance

Yes NA

Design: Record & document review; FGD, KRPI, CS, Transects


Key Lessons



6. Is there

evidence of

behavioural

change among

Filipinos as a

result of the PHL

IPR PROGRAM?

[This Sample Q

focuses on PIEP

component of

PROGRAM]

What is the number of

reported infringement

incidents in PIEP recipients

and the public, before and

after the program?

C&E

Number of

violations,

suits,

resolved

cases

NA None

Due to PIEP, do recipients

& the public believe that the

program has made a

difference in their attitude

toward IPR?

C&E

Number of

respondent

cases

Compared to before the

PIEP, is there evidence of

increased activities [P, U,

D, ™, © ® applications,

processing] among

recipients& the public?

C&E

Number of

cases of

applications,

awards

NA None

Design: Record & document review; FGD, KRPI, Transects


Key Lessons

QUESTION SUB-QUESTION TYPE

Measures

&

Indicators

Target Baseline

7. What are the key

lessons learned and

good practices that

can add up to the

knowledge base of

PHL IPR Program?

[ for policy change or

program design]

What are the comparative

advantages or strengths of

the PROGRAM?Descriptive/

Explanatory

Number of

cases;

Attributes

of Instance

NA None

What are the constraints and

complex instances worthy of

deeper consideration of

PROGRAM proponents?

Descriptive/

Explanatory

Number of

cases;

Attributes

of Instance

NA None

To what extent do the

PROGRAM proponents

willing to continue or scale

up the program activities?

Descriptive/

Explanatory

Number of

cases;

Attributes

of Instance

NA None

Design: Record & document review; FGD, CS, KRPI


Key Lessons

Key Lessons

Evaluation Findings and Results

The Philippines’ Program for IPR protection appears

UNDERPERFORMING.

The Information and Educational component of the IPR

thrust is likely NOT causing substantial and sustained

RISE of AWARENESS among public.

Rise in Awareness is likely affected more by

ENFORCEMENT and not by INFORMATION &

EDUCATIONAL PROGRAMS, [particularly when raids

and arrests are given wide media coverage].

AWARENESS: 3 Key

ThrustsSector

Unit

RATING

Factor

RATING

Overall

RATING

Public Information and

Education Program

(PIEP)

a. General

public

b. Industry

L

L

M

L

Policy and Regulation

Program (PRP)

a. General

public

b. Industry

L

ML-M

Enforcement and

Adjudication Program

(EAP)

a. General

public

b. Industry

M-H

M-HH

Rating System: H = High; M = Medium; L = Low

Table 1: Awareness Performance Rating of 3 IPR Protection Thrusts

EVALUATION

PerspectivesCRITERIA

Unit

RATING

Factor

RATING

Overall

RATING

Organizationa. Capacity

b. Linkage

SS

U

S

Program

a. Quality

b. Content

c. Reach

U

US

U

Implementation

a. Efficiency

b. Effectivenes

s

c. Continuity

U

U

U

U

Rating System: VS = Very Satisfactory; S = Satisfactory;

U = Unsatisfactory; US = Unsatisfactory

Table 2: Overall Performance of the Philippine IPR Protection Program

Key Lessons

It appears that the Philippine IPR

protection program is

UNDERPERFORMING.

‘Information and Educational’

component, likely, NOT causing

substantial and sustained rise in

Awareness among public.

Key Lessons

Evaluation Findings and Results

There seems no strong harmonization among the

three key components of the present IPR thrust.

There is no mechanism and structure for determining

performance in the present IPR Implementation

Program.

The strong transnational nature of commerce involved

in piracy makes it difficult to control patronage despite

raised awareness on IPR value. Lopsided price gap

between ‘genuine and fake’ influences infringement.

Key Lessons

Conclusions and Recommendations

Legal and Institutional environments are already in place but

socio-economic realities and poorly designed programs are the

likely reasons for the slow progress in establishing a strong

culture for IPR.

Strong law enforcement, coupled with sound information and

educational program, with emphasis on Media participation,

may deliver better outcomes.

Strong transnational cooperation and market adjustments

[such as pricing mechanism realignment] may support effective

implementation of IPR thrust.

There shall be a built-in M&E framework within the IPR

Programs, premised on results-oriented implementation

objectives.

Key LessonsLESSONS LEARNED

The weak empirical tradition and research culture in the

Philippines are constraints affecting availability and sound

collection of data that will feed evaluation. [NO existing

BASELINE [need to re establish], NO TARGET!!!]

IPR policies and programs designed based on traditional

project cycle management principles are weak. The use of

Results-Based principles can provide strong M&E mechanism

built within the system.

The media is a strong tool for IPR treaties implementation;

programs shall be designed with inclusion of media as a main

element of the program design.

Evaluation initiated by private entity without strong

endorsement from a government authority is prone to more

difficulties.

Evaluation ExperienceMY Evaluation experience in IP [START UP]

IP issues evaluated

STATE OF M&E

IN THE

COUNTRY

START UP

LOW

NIL

NIL

-- many agencies totally without

M&E

START UP

-- some beginning to see value of

M&E due to requirements from

partner, donor, and funding

organizations

LOW

-- a number have been doing

monitoring, traditional or hybrid

M&E

The START UP, still LOW M&E CAPACITY in the

PHILIPPINES does NOT PROVIDE much

OPPORTUNITIES for EVALUATION of IPR POLICIES in

the country. The awareness on the value of M&E is

BEGINNING TO TAKE HOLD!

It appears no comprehensive EVALUATION of IP

Treaties’ implementation, other than the few ‘studies’,

has been done.

Evaluation Experience

My Other Evaluation Experiences

Assessment of construction industries [Philippines & Guam] to

determine market environment for entry of Canadian green

technology [2010]

Meta-evaluation of M&Es done on ODA-funded urban development

projects in the Philippines, Indonesia and Sri Lanka [2009]

Design /formulation of Theory of Change & M&E framework,

EVAW Program, Afghanistan [2009]

Assessment of effects of ESD in TVET Education [2009]

Assessment of socio-urban contexts for development planning of

a fast urbanizing locality [2002]

Evaluation Experience

My OPINION on main evaluation needs

NEED to ADOPT results-oriented

implementation of IPR treaties in the

Philippines. A sound M&E framework shall be

integrated with the IPR Implementation

Program.

Government Agencies tasked to manage IP

Protection in the country shall be restructured

to incorporate an M&E body mandated to carry

out systematic evaluation functions.

26(MfDR Policy Brief, OECD, 2009)

2008 Survey on

Monitoring the Paris

Declaration to see

progress on aid

effectiveness

5% -countries with

largely developed

results-oriented

frameworks

56% -taken action

since 2005.

Adoption of MfDR in PHL is sluggish, BUT

as it moves toward its IMPLEMENTATION, the M&E

capacity of IPR RELATED INSTITUTIONS must be

built-up. This will enable sound DETERMINATION of

PERFORMANCE of IPR TREATIES

IMPLEMENTATION in the country.

Key Lessons SUGGESTION to PHL for strengthening IPR

treaties’ implementation*

PHL needs a mechanism for M&E in IPR protection. It

has to build one.

* [TYPICAL PHL AGENCY STRUCTURE FOR IP PROTECTION ]

DIRECTOR GENERAL

DEPUTY DIRECTOR GENERAL

Operation, IP Policies & International Relations

BUREAU OF PATENTS

DEPUTY DIRECTOR GENERAL

Management & Support Services

BUREAU OF TRADEMARKS

BUREAU OF LEGAL AFFAIRS

BUREAU OF DOCUMENTS & IT

DIRECTOR IV

DIRECTOR IV

DIRECTOR IV

DIRECTOR III DIRECTOR III DIRECTOR III DIRECTOR III

FINANCIAL MANAGEMENT & ADMINISTRATION

MANAGEMENT INRORMATION

SERVICE

DIRECTOR IV

DIRECTOR IV

DIRECTOR III DIRECTOR III

DIRECTOR IV

Office for OPERATIONS &

POLICIES is the ideal office to

reorganize & institutionalize an

M&E department.

Somewhere here, PHL

needs an M&E SYSTEM

in place -an M&E Office

mandated to determine

performance and provide

feedback to the

stakeholders,

administration & policy

makers.

THANK YOU

Documents

2011 WIPO Evaluation Workshop, Geneva, Switzerland