Page 1 of 37

RESEARCH METHODOLOGIES.

A report comparing two methodological stances in relation to research

within the events industry. (Version-14 - 21st December 2014).

Hugh Edwin Jones – MSc student in Event Management.

Derby University – Buxton Campus. Deadline - 27th of December 2014.

Page 2 of 37

CONTENTS.

A. An introduction to the report……………………………………………….. 3.

B. Developing a research strategy and the research question…………… 4.

C. Defining a major outdoor event or festivals together with the host

community for the purpose of this report. ……………………………… 8.

D. Putting event related research literature in context ………………… 10.

E. Aims and objectives of the research………………………………….. 12.

F. Conceptual framework for the research……………………………….. 13.

G. Considering the research process ……………………………………. 15.

H. Comparing the two research methodologies ………………............... 19.

I. Other considerations ……………………………………………………. 29.

J. Evaluation & Conclusion ………………………………………………… 30.

K. References………………………………………………………………... 33.

L. Bibliography……………………………………………………………..... 36.

Page 3 of 37

A. AN INTRODUCTION TO THE REPORT.

As Dylan Thomas states in the opening speech of his drama for voices Under Milk

Wood “to begin at the beginning” (Davies & Maud 1995 P3) and the beginning of this

report starts by giving an overview of the task that has been set.

Students on the MSc Events Management Course in Derby University have been

asked to write a report of no less than 4,000 words exploring two types of research

methodologies and to choose one over the other and give an explanation as to why

this type of methodology is more appropriate than the other for their individual field of

study.

Therefore to have an understanding of the research process at the outset is imperative

See figure A1 underneath:

Figure A 1.

(Gratton and Jones 2004 P34).

Selection of topic.

Review Literature.

Develope theoretica l / conceptual framework.

Clarification of research questions

hypotheseis.

Research Design.

Data collection.

Data Analysis.Drawing

Conclusions.

Page 4 of 37

B. DEVELOPING A RESEARCH STRATEGY AND THE RESEARCH QUESTION.

On starting the research process it is useful to consider the five W’s as stated in

(Dawson 2009 P13) which are: What is the research? Why do we want to do the

research? Who are the research participants? Where will we carry out our research?

When will we do our research?

The first two W’s are addressed in this section while the others will be addressed later

on in other sections of this report. After some discussions and further analysis, as

stated in (Dawson 2009 P14) is the first step in any research project to be undertaken.

Considering the planning stage of the research is then the next undertaking. (Dawson

2009 P4).

To put the subject of this report into context, it is important to understand first of all the

researcher’s position in relation to the research. The researchers ontology, (nature of

reality assumed by the researcher) epistemology (the relationship between the

researcher and the phenomenon theory being studied) and the methodology adopted,

(the way that knowledge and understanding are established) (Veal 2011 P30).

The author of this report is the Events Manager for Gwynedd Council in North West

Wales who is concerned about the financial austerity measures the Council are facing

over the next five years and the impact this may have on the support of events from a

public sector perspective. The author is fully aware of the financial settlement the

Council has received from Welsh Government for 2015 together with a very bleak

financial forecast for the following five years. Therefore the budgetary constraints the

Council will be under will undoubtedly have serve implication on services.

(http://www.walesonline.co.uk/news/wales-news/councils-wales-worse-more-3-

7905139) .

Events as a discipline in its own right within Gwynedd Council is not presently

embedded and only partly financed from within the core Economic Development

budget of the Council. As events may be perceived by some as a “nice to have” activity

rather than being essential, the present support offered is extremely vulnerable to

potential financial austerity considerations. Although it is fair to say that even economic

Page 5 of 37

development in itself is not a statutory obligation for the Council. Therefore the overall

position of economic development could also be argued as being compromised.

Presently any funding support offered to event organisers by the Council is made on

a three by three year project basis as stated in a report to Gwynedd Council Cabinet

May 2013 (Gwynedd Council report to full Council 2013). A robust, strong and credible

evidential argument must be put forward at this stage to counteract any potential

motion being put in front of the Council to cease all funding and support for any type

of event and to concentrate the Council’s resources on the statutory and legal

responsibilities the Council have.

It is therefore extremely important that the research to be carried out is able to show

and present quantifiable values that major outdoor events and festivals can have at

economic, social, cultural, political, physical and environmental levels as found to be

the case in (Langen and Garcia 2009 P3). It is also very important to bear in mind that

any research may also show a negative impact, or outcomes that a major event or

festivals may have on a host community as stated by (MacAllon 1994 P241-3 cited in

Hall 1992 P36). Although it is fair to assume that these will be limited against the

positive impacts that will be derived.

For this report to have credibility and to be accepted as having honest and credible

evidential substance in its findings, it’s is very important that the researchers natural

bias as an employee of Gwynedd Council and also as an individual who considers

events very much as a real driver for social, political, economic and environmental

development does not negatively affect the outcome of the research.

(www.aqr.org.uk/glossary/?term=bias).

The research and its findings will allow Gwynedd Council’s Portfolio Leader for

Economy and Regeneration to take a robust, credible and well researched argument

to cabinet for the continued support of major events and festivals in Gwynedd for the

future.

A review will be carried out of models that have been used for evaluating events

historically. There are numerous models out there that have been utilised as found by

Page 6 of 37

(Langen & Garcia 2009). Differing models have been observed amongst event

organisers in Gwynedd and wider afield across Wales. These models can be argued

to be incompatible with one another and bring to the table very different and sometimes

misleading information. They give hugely different outcomes and values because

differing methodologies, methods and variables are open to interpretation. (Vrettos

2006 cited in Langen & Garcia 2009 P4) found this to be the case through comparing

economic impact studies of major events in the UK and Spain.

The researcher will work closely with the Major Events Unit of Welsh Government to

assist in developing a one model analysis approach for events at a Gwynedd and all

Wales level.

The research question that has been agreed for this research is: “What are the values

that Festival No6 brings to the host community of Gwynedd?” (Gratton and Jones 2004

P33) notes, like many other writers on the subject, that choosing the research question

is crucial, as a question that is not well thought out can create problems later on in the

research project.

Something else to bear in mind in relation to the research question as stated by

(Veal 2006 P65) is that a research question that looks fairly simple at the outset of the

research process, can become very daunting and demanding upon further reading,

philosophical thought, debate and analysis. The situation may even become

impossible to manage within the timescale and resources available to the researcher.

On recognising the limitations put upon the research process by time and resources

available, the focus of this research will therefore concentrate on one specific event,

and one specific region the County of Gwynedd in North West Wales. However the

bigger more complex issues the question may raise will not be ignored (Veal 2006

p.65).

The event which will be the subject of the research is a fairly new event, 2015 being

its 4th year. The Festival is held annually in the Italianate village of Portmeirion in

Gwynedd. (www.festivalnumber6.com/).

Page 7 of 37

This research will not just be interested in the economic impact of the event, but will

also explore social and environmental impacts at an equal level and therefore as

(Langen and Garcia 2009 P9-10) conclude in their report, the value of such research

will be of a broader value and interest.

At this stage the subject matter lends itself very much to a case study approach in

design, although other designs will be considered as the process develops. (Gratton

and Jones 2004 P97) define case studies as an intensive study of a specific case.

This could be an organisation, a community, an audience of an event or an

individual. Case studies are designed to bring out the details from the viewpoint of

the participants by using multiple sources of data. (as cited in “Application of a Case

Study Methodology” by Winston Tellis The Qualitative Report, Volume 3, Number 3,

September,1997.

Page 8 of 37

C. DEFINING A MAJOR OUTDOOR EVENT OR FESTIVAL TOGETHER WITH

THE HOST COMMUNITY FOR THE PURPOSE OF THIS REPORT.

A research definition of what is a major outdoor event or festival and what the definition

of a community or region have to be considered and agreed in advance of any work

being undertaken.

At this stage there are many variables to take into account that may negatively affect

the outcome of the research project if not addressed at this early stage. Issues such

as an agreed definition of what constitutes a major outdoor event, the type of event i.e.

sports cultural etc. the type and size of community the event is set in, where

geographically the event is held, the time of year the event takes place, the time of day

the event takes place and the events overall offer to its audience.

What may be considered a major outdoor event or festival in North West Wales and

what may considered to be a major outdoor event or festival in a City like Liverpool,

are more than likely to be hugely different.

An event of similar size, content and staged in a primarily rural area could be

considered for a comparative study as this would draw much fairer and comparable

comparisons, although there is every possibility that the conclusions arrived at may

differ. One of the key variables in relation to Festival No6 that has to be considered is

the uniqueness of the location, being the village of Portmeirion.

For the purposes of this report and research, the definition attributed to a major event

or festival will be defined as:

“A paid for event that is held in a rural area with an audience capacity of no

more than 15,000 people where arts, music, cultural & culinary experiences are

offered”

In relation to the community or region the major outdoor event or festival is held in will

be defined as:

Page 9 of 37

“A rural area with a population of some 120,000 people covering a land mass

of some 255 km www.gwynedd.gov.uk.

Page 10 of 37

D. PUTTING EVENTS RELATED RESEARCH LITERATURE IN CONTEXT.

It is generally agreed that most commissioned event impact or evaluation research

and studies that have been carried out are primarily interested in facts and figures.

This was the conclusion that (Langen & Garcia May 2009 P9) came to in their report

in reviewing available literature in measuring the impacts of large scale cultural events

following the Liverpool City of Culture year in 2008.

In a research carried out on assessing the economic impact of the Glastonbury Festival

the researchers were able to look wider than this and were able to point out economic

impacts that were far less quantifiable such as trading opportunities for not-for-profit

organisations and the contribution of the festival to local entrepreneurial culture (Baker

and Associates 2007 P17).

Another research project that was looked at was into the value of the South African

National Arts Festival who used a very different approach. They used choice

experiments to value the utility visitors derived from various sections of the festival.

The argument put forward by them for using this type of methodology was its

usefulness in estimating benefits of the various aspects of the festival and how these

are differently valued by the audience (Snowball and Willis 2006 cited in Langen &

Garcia May 2009 P4).

A report in 2006 that looked at the Manchester Pillar Events concluded that there is a

major gap in research literature in understanding the indirect impacts of major events

upon host communities. The report further states that intangible or less easily

measured outcomes or outputs are often ignored or poorly dealt with. The authors of

the report argue for the use of focus groups to develop understanding of cultural

networks and impacts on industry and the effect on audiences etc., (Jura Consultants

2006 cited in Langen & Garcia May 2009 P4).

(Carlsen et al. 2007, cited in Langen & Garcia May 2009 P5) call upon other festival

researchers to move beyond economic impacts in order to understand the complex

and comprehensive set of benefits and possible dis-benefits associated with festivals.

They argue that the cultural, community and social benefits of major festivals have not

Page 11 of 37

been systematically studied, while they point at the limitations of focusing on more

narrow economic outcomes, such as comparability, reliability, and utility of estimates.

As an alternative, they propose an inclusive research agenda, focused on benefits

through arts, culture, community, economy, society, and stakeholders.

(Hall 1992 cited in Langen & Garcia May 2009 P9) argues that environmental, socio

cultural and political effects are probably more important than economic ones, but have

tended to be ignored because festival organisers and public sector commissioned

research require economic data. Socio-cultural impacts are less easily quantifiable,

and research concerned with socio-cultural effects may find results that are less

politically palatable, particularly if what might be regarded as negative consequences,

such as increases in crime or conflict between locals and visitors, are seen to outweigh

the perceived economic gains. To a certain extent, this observation seems to hold true

for studies of large scale cultural events as well, with economic impact studies clearly

dominating the literature.

Taking these arguments and findings into consideration, it is fair to conclude that there

is a need for more knowledge and quantifiable values in other areas of events

evaluation beyond economic evaluation.

Page 12 of 37

E. AIMS AND OBJECTIVES OF THE RESEARCH.

In this section of the report the research question is translated into the research aims

and objectives that will guide the researcher and the research.

The aim for this research can be defined as the overall purpose of the study, the aim

will need to be clearly and concisely defined. The aim is a broad statement of desired

outcomes, or the general intentions of the research, which 'paint a picture' of the

research project. The objectives define how the aim is to be accomplished, there will

also be a need for these to be highly focused and feasible.

(www.erm.ecs.soton.ac.uk/theme4/aims_and_objectives.html)

The aim and objectives are:

Aim.

To assess the impacts of Festival No6 both positively and negatively on the

host community of Gwynedd.

Objectives.

The research will seek to determine:

1. The nature extent and effect of the economic impact of Festival No6 on the

host community of Gwynedd.

2. The nature extent and influence of social and community impacts of Festival

No6 on the host economy of Gwynedd.

3. The nature, extent and effect of environmental impacts of Festival No6 on

the host economy of Gwynedd.

4. The nature extent and impact of legacies that Festival No6 may leave in the

host economy of Gwynedd.

Page 13 of 37

F. CONCEPTUAL FRAMEWORK FOR THE RESEARCH.

Following on from agreeing the aim and objectives for the research, a conceptual

framework is then developed. A conceptual framework is:

A written or visual presentation that:

Explains either graphically, or in narrative form, the main things to be studied the key

factors, concepts or variables - and the presumed relationship among them.

(Miles and Huberman, 1994, P18 cited in a presentation by Professor Roger Vaughn

of Bourmouth University 2008). The conceptual framework that has been developed

for this research is illustrated in the model underneath:

Figure G.1.

ECONOMMICS

Economics.Economic impacts

derived from the event.

Analyse and evaluate all relevent data held by event organiser.

Focus group derived from a cross section of beneficiaries to set survey or

questionnaire questions.

Sample survey or questionairs with event attendees which can be followed up.

In depth interview local businesses & other beneficiaries.

Social & Community.

Social & community values derived from the

event.

Cross sectional focus group including community & stakeholders to gain their views and develop & test questionaire

survey questions.

Postal questionaire to be sent out to members of the community & businesses

to gauge their views of the event.

Indepth interviews with a cross section of the community.

Page 14 of 37

* It is important to remember as the research process develops, this framework can

change and evolve as this stage it’s just a starting point (Veal 2006 P61).

(Veal 2006 P61). States that a conceptual framework approach seems inconsistent

with the inductive approach where theory is derived from the data rather than data

being used to test pre–existing theory. In particular it seems inconsistent with the more

apparently open ended approaches as grounded and informal approaches used in

qualitative research. Although (Miles and Hubbard 1994 P17) believe a conceptual

framework is just as vital for qualitative research as quantitative – arguably more so

A conceptual framework need not be strait jacket that ties down the researcher to the

framework as the framework can be a flexible, evolving device as research theory data

collection and analysis are often intertwined rather than being sequential (Veal 2011

P63).

Legacy.Legacies derived from the

event.

Analyse relevent event data and event related policies.

Creation of cross sectional focus groups to include community & stakeholders to gain their views.

Environmental.Enviromental values dervived from event.

Analyse relevent event data and policies.

Creation of cross sectional focus group to include community & stakeholders to

gain their views.

Observational analyses before during and after the event to draw conclusions

on evironment behaviour against objectives.

Page 15 of 37

G. CONSIDERING THE RESEARCH PROCESS.

Socialist Norbert Elias defines research as:

“The aim, as far as I can see, is the same as in all science. Put simply and cursorily,

the aim is to make known something previously unknown to human beings. It is to

advance human knowledge to make it more certain or better fitting……. the aim is

discovery” (Elias 1986: P20 quoted in Veal 2011:P4).

Research is a systematic process of discovery and advancement of human knowledge

(Gratton and Jones 2004 P32). This re-emphasises the fact that the research process

is firmly based on the principle of science and follows a systematic approach and

defined methodologies to arrive at a conclusion to the research question or hypothesis.

In the area of Social Science which is the area of interest for this report and the basis

of research to be carried out. The research will focus on people as social beings and

as members of communities, therefore it can be argued that the scientific model must

be adapted and modified and in some cases largely abandoned (Veal 2011 P5).

As the events industry deals primarily with people and their social behaviour and

attitude, this research may well fall into the realm of this adapted model of research

which is based on scientific principles, but classified as social science.

It can be said that people are less predictable than the rules of scientific and

descriptive, explanatory and evaluative research (Veal 2011 P5) Veal; further states

that the social world is constantly changing and for example, it is highly unlikely that

an exact replication of research carried out at different times or in different

geographical locations will deliver the same results as would probably be the case if

one was to stick rigidly to Elias definition of research as being a tool of science (Veal

2011 P5).

Based on the term that Elias used discovery, one can deduce that this is a process of

exploration that allows us to draw conclusions as to what actually exists out there in

the real world. However, if the process is to create more understanding and to add to

Page 16 of 37

human knowledge, it requires more than just the assimilation of information and facts.

As part of the process an explanation should be arrived at to show why things are as

they are and how they might be (Veal 2011 P5).

(Veal 2011 P8) states that there is also a third function of research which is evaluation.

That is the judging of policies, programmes and action that have been carried out and

in some cases to assist in the argument to keeping those strategies, programmes and

action in place for the future. (as will be the case for this research).

(Veal 2011 P6) states that there are three main types of research methods which are

commonly used in the leisure and tourism area. See Figure C1 underneath.

Figure C1

Descriptive research.

Finding out, describing what is.

Explanatory research.

Explaining how and why things are as they are

(and using this to predict).

Evaluative research.

Evaluation of programmes and research.

(Veal 2011 P 29).

Before we consider any methodology, we need to step back and fully appreciate the

polar positioning of both Quantitative v Qualitative approach to research.

(Creswell 2014 P3) makes a valid observation that the Quantitative and Qualitative

approaches are not as discrete as they first appear. Creswell argues that they should

not be viewed as rigid, distinct categories, polar opposites or dichotomies. According

to (Newman & Benz 1998 cited in Creswell 2014 P3) they represent different ends on

a continuum.

Page 17 of 37

Please see Figure C1.1 which shows and compares Quantitative and Qualitative

methodologies.

Figure C1.1.

Characteristics of Quantitative and Qualitative Research.

Quantitative Research.

Qualitative Research.

Uses numerical analysis to measure

social phenomena to provide facts.

Relies on non-numerical analysis to

provide understanding.

Assumes single objective social reality. Assumes social reality is a subjective

experience.

Assumes social reality is constant across

different times and settings.

Assumes social reality is continuously

constructed and related to the immediate

social context.

Uses statistical analysis to determine

casual relationships.

Objectives are description,

understanding and meaning.

Studies samples with the intention of

generalising to population.

Uses smaller samples or cases.

Research is objective and detached from

the subject under investigation.

Data is rich and subjective location of the

research is often neutral.

The setting is often contrived. Flexible approaches to data collection

Often non-traditional approaches e.g.

content analysis.

Data is collected using inanimate objects

e.g. pen paper.

The researcher is the data collection

instrument.

Associated with positive approach. Associated with the interpretive

approach.

Generally deductive. Generally inductive.

(Gratton & Jones 2004 P21).

There is also a third methodology; Mixed methodology which is becoming more

popular and gaining acceptance and credence as stated by (Brannen 2005 P4). A

Page 18 of 37

mixed methodology approach involves utilising element of both quantitative and

qualitative methodologies and related methods (Veal 2011 P34-35).

As mixed methodology is a much later addition to the armoury of researchers with an

ongoing debate amongst academics who suggest that the two forms (Quantitative

and Qualitative) of what they perceive to be the more established and recognised

forms of research, are incompatible as the rely on two differing epistemological

assumptions (Gratton and Jones 2004 P21).

However (Bryman 2008 P603) is of the opinion combining both Quantitative and

Qualitative methodologies allows for the various strengths to be capitalised on and

the weaknesses offset somewhat. Therefore it is fair to deduce that mixed

methodology resides in the middle of a continuum as stated by (Newman & Benz

1998 cited in Creswell 2014 P3) as it incorporates elements of both quantitative and

qualitative methodologies (Creswell 2014 P3).

.

Page 19 of 37

H. COMPARING THE TWO RESEARCH METHODOLOGIES.

Methodology 1 - Quantitative Methodology.

Quantitative Methodology sits in the Positivism paradigm (epistemology) and adopts

an approach of testing theories by examining relationships among variables. These

variables, in turn, can be measured, typically on instruments, so that numbered data

can be analysed using statistical procedures (Creswell 2014 P4).

A positivist positioning believes that the truth is out there in the world and it’s through

using only quantitative research methods that the truth can be discovered. The

researcher will need to be as detached as possible from the research allowing for

objectivity and reducing the researchers’ possible influence on the process.

Predominantly, this research will be evaluative in nature as it seeks to strengthen

and enable a judgement on the success or effectiveness of policy or programme that

the Council have in place (Veal 2011 P8).

It would be poignant to ask at this point, if quantitative research only deals with facts

and figures, is quantitative research then not severely compromised?

A substantial amount of data that does not naturally appear in quantitative form can

be collected in a quantitative way. This can be done by designing research

instruments aimed at specifically converting phenomena that don’t naturally exist in a

quantitative form into quantitative data which can be analysed statistically (Doing

quantitative research in education with SPPS 2010 P2).

In this way the use of quantitative research to evaluate a phenomena can be quite

flexible, although the methodology does have its weaknesses and it can be argued

that some phenomena are better studied by using a different methodology (Doing

quantitative research in Education with SPPS 2010 P2).

As quantitative research uses statistics and mathematically based methods this

could become very daunting to the researcher and influence their ontological thinking

although the introduction and development of computer software makes the work of

analysing quantitative data quick and relatively easy these days.

In this research methodology, the constant variable will be the event and its impact,

the independent variables are the elements like people, their opinions, weather,

Page 20 of 37

expenditure values, community involvement engagement, business opportunities

offered etc.

Reliability and Validity.

How do we know that what is collected is reliable and has reliability, validity and is

trustworthy? Does the research truly reflect the phenomenon which is being looked

at in the research? Reliability refers to a measure of a concept while validity refers to

the issue of whether and indicator that is devised to gauge a concept really and

truthfully measures that concept (Bryman 2008 P149 &150). If for example we were

to raise the question of enjoyment at an event, based on the Lickert scale that was

carried out, we would be questioning the validity of that particular value against the

concept of enjoyment. There are too many types of validity to go into in this paper but

as (Bryman 2008 P139) puts it, the differing types of validity reflect the different ways

of gauging validity of a measurement of a concept.

In relation to reliability, we have to be aware when the research is being carried out

that the researcher should be able to administer that research, again with little

variation in the results over time. There will be a requirement to analyse whether

individuals scores or any one indicator / question have a similarity to the scores

within another indicator / question. Finally, the researcher will need to be aware of

consistency in the approach to the research matters like how to categorise media

items, or when answering open ended questions and how these are categories, or in

structured observation when there are many observers, or on how to classify subject

behaviours (Bryman 2008 P149-152).

Methodology 2 - Mixed Methodology.

Mixed methodology sits in the pragmatic paradigm which aims to capture data from

within the positive and interpretive paradigms = pragmatic. This methodology involves

integrating the two forms of data and using distinct designs. The core assumption of

this inquiry is that the combination of qualitative and quantitative approaches provides

a more complete understanding of a research problem than either approach alone

(Creswell 2014 P4).

Page 21 of 37

One of its strengths is seen by many researchers as offering a range of methods to

arrive at a conclusion but (Brannan 2005 P6) states that this should not take the place

of creative thinking.

Mixed Methodology creates the opportunity to learn new research skills which is to be

welcomed and is particularly facilitative of cross-disciplinary collaboration, but should

not undermine disciplines and the importance of theory (Brannan 2005 P6).

A mixed methodology approach will allow for a fuller more comprehensive evaluation

of the research question within the overall method. It will allow for easier corroboration,

validating and triangulation of data. This will allow for checks and result comparisons

to be made against one form of data collected in a specific way, against another for

the purpose of validity, reliability and truth, in an attempt to arrive at a more rounded

and complete answer to the research question. This will also allow the researcher to

have a better understanding of a social phenomenon from different vantage points

(Brannen 2005 P12).

It’s suggested by (Gratton and Jones 2004 P28) that one should not consider

qualitative or quantitative approach to be better than the other and to consider the

methodology or methodologies that best answers the research question. As supported

by (Bryman cited in Brannen 2005 P9).

Mixed methodology is relatively new in origin as a distinct research approach, but it

seems that mixed method strategies are being increasingly utilized in research

methodologies. This hypothesis is further strengthened by the staging of numerous

National and International seminars on mixed methodologies, together with increasing

research posts being funded at UK and International Universities in this discipline

(Brannen 2005 P4).

A mixed methodology process collects qualitative and quantitative data and according

to (Creswell 2014 P4) it then becomes a process of integrating the two forms of data

and using distinct designs that may involve philosophical assumptions and theoretical

frameworks.

(Bryman cited in Brannen 2005 P9) states that we need to be careful that we do not

lose sight of the concept that research questions should determine the research

Page 22 of 37

methods. This does seem to be the stance of the majority of text available that

addresses methodological issues in research (Brannen 2014 P9).

Another reason put forward by (Brannen 2014 P4) for the popularity of mixed

methods research is related to the fact that it is more prevalent and reflected upon

today.

Reliability and Validity.

(Bryman 2008 P376) is of the belief that measurement is not a major pre-occupation

among qualitative researchers and therefore the issue of validity would seem to have

very little bearing on such studies. However, certain methods have been developed

such as external reliability which addresses the replication of research to arrive at a

result, although this is recognised to be very difficult in qualitative research

Internal reliability, where the researcher and his team in an observational approach

to the research, agree about what they hear and see and finally external validity

considers how the research findings can be generalised across social settings.

Internal validity, whether there is a good match between the researchers

observations and the theoretical ideas that are developed.

External Validity, refers to the degree to which findings can be generalized across

the social settings. Qualitative researchers have tended to employ the terms reliability

and validity in very similar ways to quantitative researchers when seeking to develop

criteria for assessing research (Bryman 2008 P337).

The research design for both methodologies will be of a case study in nature which

entails intensive analysis of a single case, being Festival No6 and its impact on the

community of Gwynedd (Bryman 2008 P52).

Both Methodologies to be considered are set out side by side in table Figure G1

underneath so that comparisons can be more easily made.

Page 23 of 37

Figure G1.

Quantitative Research Methodology.

Mixed Method Research Methodology.

Focus group.

A sample cross sectional focus group

derived from stakeholders, community and

event beneficiaries will be brought together

to set and test the questions for the survey

for attendees, community, event specific

traders and local businesses. This will assist

in gaining correctness and validity and the

elimination of bias.

Data will be gathered in a numerical way

using the Likert scale (Bryman 2008 P146)

1-5 scale, within a survey, using a

computerised data analysis programme for

evaluation.

Focus group.

A sample cross sectional focus group

derived from stakeholders, community

and event beneficiaries will be brought

together to set questionnaire questions

for attendees, community, event specific

traders and local businesses. This will

assist in gaining correctness and validity

and the elimination of bias.

Data will be gathered in a numerical

and narrative way through the

questionnaires using a computerised

data analysis programme for evaluation.

The cross sectional focus group will be

brought back following the festival into a

discussion group to gauge their overall

views of the Festival. This information

will be collected and analysed in a

narrative form and compared against

the other data collected for correctness,

correlation validity and bias.

Economic impacts.

Secondary data analyse and evaluation;

of relevant economic data held by the event

organiser. This to include expenditure

patterns, ticket sales with postcode analysis.

An evaluation of the events and media

Economic impacts.

Secondary data analyse and

evaluation; of relevant economic data

held by the event organiser. This to

include expenditure patterns, ticket sales

with postcode analysis. An evaluation of

Page 24 of 37

coverage which will be converted into

numerical values for evaluation. This will be

carried out through desk research of

secondary data.

Event traders; to be questioned through an

online survey before and after the event

allowing for an understanding of their

expenditure and supply chains etc. Evaluate

their aspirations against the reality. This

would be collected and evaluated

numerically in a quantitative manner.

Local business; would be carefully

selected and a postal and internet business

survey would be carried out with questions

focusing on the festival’s impact on their

staffing and turnover. This would be

collected and evaluated numerically and

analysed in a quantitative manner.

An economic impact analysis; will be

carried out at the end of the data collection

and analysis of each section above.

Economic benefit will be considered in three

areas:

Direct impacts: expenditure and income

generated at the festival together with the

promoters expenditure to stage the event.

Indirect impacts: effect on businesses

receiving direct expenditure through

increased activity in the supply chain to

meet requirements

Induced impacts: Increase in expenditure

as a result of increased wages expenditure

directly or indirectly associated to the event.

the events and media coverage which will

be converted into numerical values for

evaluation. This will be carried out

through desk research of secondary data.

Carry out an in depth interview; with

the Event organiser to draw his / her

opinion of the event in a qualitative

manner drawing out opinions, thoughts,

desires etc.

Event traders; questioned through an

online questionnaire before and after the

event allowing for an understanding of

their business model, expenditure and

supply chains etc. Evaluate their

aspirations against the reality. This would

be collected and evaluated numerically

and narratively in both quantitative and

qualitative manner.

Local business; would be carefully

selected and a postal and internet

business survey would be sent to them

with questions focusing on trading

patterns and the festival’s impact on their

staffing and turnover. This would be

collected and evaluated numerically and

narratively in both quantitative and

qualitative manner.

Carry out an in depth interview with a

sample of event specific traders and local

businesses to gauge their opinions of the

event and their trading opportunities or

not as the case may be.

Page 25 of 37

Generating and stimulating further economic

activity. This will be carried out in a

quantitative way

There are many recognised methods of

calculating economic value of events in a

primarily quantitative ways that are

reasonably costs effective to execute with

fairly acceptable results. Please see the

model favoured by the Major Events Unit of

Welsh Government and recommended to

event organisers across Wales, which will

be considered for this research.

www.eventimpacts.com/economic/intermedi

ate/economic-calculator/

(Baker Associates Economic impact of

Glastonbury Festivals 2007 P7).

An economic impact; analysis will be

carried out at the end of the data

collection of each section mentioned

above. Economic benefit will be

considered in three areas

Direct impacts: expenditure and income

generated at the festival together with the

promoters expenditure to stage the event.

Indirect impacts: effect on businesses

receiving direct expenditure through

increased activity in the supply chain to

meet requirements.

Induced impacts: increase in

expenditure as a result of increased

wages, expenditure directly or indirectly

associated to the event. Generating and

stimulating further economic activity.

This will be carried out in a quantitative

way but supported and corroborated with

narrative findings. Please see the model

favoured by the Major Events Unit of

Welsh Government and recommended to

event organisers across Wales, which will

be considered for this research.

www.eventimpacts.com/economic/interm

ediate/economic-calculator/

(Baker Associates Economic impact of

Glastonbury Festivals 2007 P7).

Attendees and community *(see sampling

underneath).

Face to face survey with a cross sectional

sample of the audience carried out over the

weekend. The survey will evaluate event

Attendees and community *(see

sampling underneath).

Face to face questionnaire with a cross

sectional sample of the audience carried

out over the weekend. The questionnaire

Page 26 of 37

expenditure accommodation costs, travel

costs and daily spend costs. It could also

evaluate event satisfaction, legacy, gender,

age, enjoyment value, value for money and

many other hosts of potential questions.

However this research is primarily interested

in economic evaluation in relation to the

audience.

The survey will be collected in Likert scale

(Bryman 2008 P146) 1-5 scale which will be

analysed using a computerised quantitative

approach.

A survey will to be sent out to a sample of

the community. Both surveys will use the

Likert scale (Bryman 2008 P146) 1-5 scale

for data collection. Evaluation will follow a

quantitative computerised process.

will evaluate event expenditure,

accommodation costs, travel costs and

daily spend costs. It could also evaluate

event satisfaction, legacy, gender, age,

enjoyment value, value for money and

many other hosts of potential questions.

However this research is primarily

interested in economic evaluation in

relation to the audience.

The questionnaire will be a mixture of

both numerical values and an element of

narrative which will be analysed using

both quantitative and qualitative

computerised processes

A questionnaire will to be sent out to a

sample of the community to gauge their

views on the event and the legacies

created by the event

Both questionnaires in this section will

use the Lickert scale for data collection

evaluation and will follow a quantitative

process of evaluation. Some narrative

may be introduced.

With the discussion group information

being collected in a qualitative manner

this can be used for correlation and

validity of the other data collected.

Page 27 of 37

Environmental Impacts.

Postal survey to be sent out to members of

the community, traders and businesses to

gauge opinion on the green credentials of

the event.

Event policies and statements, especially in

relation to environmental and the Welsh

Language, will be studied. Observational

study to be carried out to evaluate the event,

organisers and audience behaviour against

the policies and actions set out by the event

organiser in their environmental policy (if

they have one).

Both sets of data will be collected in

numerical manner to allow for statistical

analysis through a computerised system.

Environmental Impacts.

Postal questionnaires will be sent out to

members of the community, traders and

businesses to gauge opinion on the

green credentials of the event.

This will be followed up by an in-depth

interview with a sample of the

community traders and businesses to

corroborate and validate the data

received from the questionnaire.

Event policies and statements, especially

in relation to environmental and the

Welsh Language, will be studied.

Observational study to be carried out to

evaluate the event organisers and

audience behaviour against the policies

and actions set out by the event organiser

in their environmental policy, (if they have

one).

Both sets of data will involve

quantitative and qualitative data which

will be analysed using both methods of

analysis and evaluation through a

computerised system.

Stakeholders

A survey will be sent to each stakeholder to

evaluate in numerical terms what their views

of the event are & were.

This data will be analysed in a computerised

quantitative manner.

Stakeholders

In depth Interviews will be held with each

stakeholder to obtain their views on the

event in a qualitative manner which then

could be corroborated for truthfulness

with a survey if this was to be carried out.

Page 28 of 37

Reporting findings.

These will be reported under the following

headings:

Economic impacts.

Legacy, social and community

impacts.

Environmental impacts.

Overall conclusion.

Reporting findings.

These will be reported under the

following headings:

Economic impacts.

Legacy, social and community

impacts.

Environmental impacts.

Overall conclusion.

Page 29 of 37

I OTHER CONSIDERATIONS.

*Sampling.

With sampling there is always a possibility that a particular sample chosen might not

be truly representative, even though selected in an unbiased way. But if the size of

the sample was large enough the significant difference between study groups stand

out better. It is important to bear in mind however, that the researcher will always

want a small sample group to be practical. Therefore deciding on the sample size is

not an easy task (Journal of Tropical Paediatrics Oxford University Press P44).

The research would work towards drawing a representative sample from the overall

population to be studied. This representative sample would then be analysed and

generalised to the overall population (audience and community) against the sample

that was drawn. This would obviously save time and money. If the sample is biased

(not felt to be representative) less validity can be generalised from the sample to the

population (Journal of Tropical Paediatrics Oxford University Press P40). Great care

will have to be taken in how the sampling is arrived at and how the data is collected

and administered.

Ethics.

The fundamental elements of ethics is “to always do good and never do any harm”.

The researcher will need to protect their research participants; develop a trust with

them; promote the integrity of the research; guard against misconduct and

impropriety that might reflect on their organisation or institution; and cope with new,

challenging problems (Israel & hay 2006 cited in Creswell 2014). This research will

be carried out following the code of conduct and ethics of Derby University and

Gwynedd Council at all times.

Page 30 of 37

J. EVALUATION AND CONCLUSIONS.

On assessing both the advantages and limitations of both methodologies the

comparative chart below (Figure J.1.) addresses six advantages and six

disadvantages of both methodologies under consideration.

The chart in the opinion of the author clearly defines the advantages of using a

mixed methodology in comparison to sticking with a fairly rigid methodology of

quantitative evaluation. Using a mixed methodology approach will allow the

researcher to address the question set in a more comprehensive manner. It will

also allow more scope to compare, corroborate and triangulate data. Resources

and time must clearly be taken into account and evaluated when following this

methodology.

Therefore based on the evaluation of the methodologies looked at and as shown in

(Figure J.1.) below the author of this report favours a mixed method approach to

answer the research question that has been set.

Figure J.1.

Advantages of Quantitative

Research.

Dis-advantages of Quantitative

Research.

1 Through this methodology the

researcher collects and analyses

their data in a methodical scientific

way.

This Methodology is unable convey

the richness and depth of detail as

other methodologies.

2 The researcher can be more

accurate and objective in their

conclusions as a variable can be

studied in detail.

Unable to answer or decipher the

question of meaning.

3 Through experiments and its ability

to measure data using data and

mathematical calculations allowing

the researcher to test a hypotheses.

Not able to look at how individuals

react or respond differently to

situations and each other.

Page 31 of 37

4 This type of research normally

involves fewer variables.

Not able to address or answer a

question of how people may be

affected and how.

5 This research allows for greater

reliability and validity by following

agreed procedures.

The tendency is to sample a large

amount of the population this will

need evaluation and analyses to

achieve more statistically accurate

results.

6 The researcher can at a later time

following the same research

procedures replicate the research to

test results or allows for other

research to be carried out.

Analysing relationships between

variables tends to create

formulated and static view of social

life. This can also be viewed as

being independent of people’s

lives.

Advantages of Mixed

Methodology Research.

Dis-advantages of Mixed

Methodology Research.

1 It reduces weaknesses in the

research process, allowing for the

drawing of the strong points of both

quantitative or qualitative research

methodology.

The argument is still ongoing

within the research fraternity that

Mixed Methodology is seen as a

poor quality research process,

masquerading as mixed methods,

and argued to be devaluating

basic assumptions of both

methodologies.

2 Allows comparison of one date set

collected in one methodology against

another for the purpose of correlation,

triangulation and validity.

The researcher within Mixed

Methodology will need to be aware

and clear in recognising the

dominant approach otherwise

effectiveness could be lost.

3 Mixed methodology allows for the

consideration of numerous designs

that could be utilised.

The researcher will need to be fully

conversant with both Quantitative

Page 32 of 37

and Qualitative research

methodologies.

4 As qualitative and quantitative data

may be collected at the same; there is

substantial opportunity for time and

cost savings.

Mixing the methodologies could

become impracticable, requiring

substantial amount of resources at

times.

5. It is far less time consuming to collect

and analyse data these days, owing to

the advancement of technology and

computers.

The researcher may have a

preference for either a quantitative

or qualitative research, bias could

then develop.

6. Both quantitative and qualitative

data are merged in order to provide

a comprehensive analysis of the

research problem (Creswell 2014

p15).

Thought by some to involve the

Positive and interpretive

paradigms which are total

opposites and therefore

incompatible to address the

research question. (Gratton and

Jones 2004 P21).

*Footnote.

This report is written with the assumption that the processes set out in this report

will not face no problems or issues as it progresses, this however is rarely the case

with a research project.

Page 33 of 37

K References

Davies & Maud1995 “Under Milk Wood the definitive edition” Publisher:

Orion Publishing Group.

Gratton and Jones 2004 “Research methods for sports studies”

Publishers: Routledge – Taylor and Francis Group.

Dawson 2009 “Research Methods” Publishers: How to books Ltd.

Veal 2011 “Research Methods for Research and Tourism” Publishers:

Pearson Education Ltd.

www.walesonline.co.uk/news/wales-news/councils-wales-worse-more-3-

7905139

Gwynedd Council Report to full Council 5th of December 2013.

https://www.gwynedd.gov.uk/cy/Cyngor/Cynghorwyr-a-

phwyllgorau/Pwyllgorau,-cofnodion-ac-agendau/Pwyllgorau,-cofnodion-ac-

agendau.aspx?pwyllgor=/2013-14/Cyngor Llawn_Full Council/2013-12-05

Langen and Garcia “Measuring the Impacts of Large Scale Cultural Events : A

literature review” Impacts 2008 University of Liverpool and John Moores

University commissioned by Liverpool City Council.

MacAllon cited in Colin Michael Hall 1992 “Hallmark Tourist Events”

Publishers: Belhaven Press.

www.aqr.org.uk/glossary/?term=bias

Vrettos 2006 cited in Langen and Garcia “Measuring the Impacts of Large

Scale Cultural Events : A literature review” Impacts 2008 University of

Liverpool and John Moores University commissioned by Liverpool City

Council.

Veal 2006 “Research Methods for Leisure and Tourism” Publishers: Pearsons

Educational Ltd.

www.festivalnumber6.com

“Application of a Case Study Methodology” by Winston Tellis The Qualitative

Report, Volume 3, Number 3, September,1997 www.nova.edu/ssss/QR/QR3-

3/tellis2.html

Page 34 of 37

www.gwynedd.gov.uk

Baker Associates 2007 cited Langen and Garcia “Measuring the Impacts of

Large Scale Cultural Events : A literature review” Impacts 2008 University of

Liverpool and John Moores University commissioned by Liverpool City

Council.

Snowball & Willis 2006 cited in Langen and Garcia “Measuring the Impacts of

Large Scale Cultural Events : A literature review” Impacts 2008 University of

Liverpool and John Moores University commissioned by Liverpool City

Council.

Jura Consultants 2006 cited in Langen and Garcia “Measuring the Impacts of

Large Scale Cultural Events : A literature review” Impacts 2008 University of

Liverpool and John Moores University commissioned by Liverpool City

Council.

Carlsen et al 2007 cited in Langen and Garcia “Measuring the Impacts of

Large Scale Cultural Events : A literature review” Impacts 2008 University of

Liverpool and John Moores University commissioned by Liverpool City

Council.

Hall 1992 cited in Langen and Garcia “Measuring the Impacts of Large Scale

Cultural Events : A literature review” Impacts 2008 University of Liverpool and

John Moores University commissioned by Liverpool City Council.

www.erm.ecs.soton.ac.uk/theme4/aims_and_objectives.html

Miles and Huberman, 1994, P18 cited in a presentation by Professor Roger

Vaughn of Bourmouth University 2008.

Miles and Hubbard 1994 P17 cited in lecture slides.

Elias 1986 P20 cited in Veal 2011 “Research Methods for Research and

Tourism” Publishers: Pearson Education Ltd.

Creswell 2014 “Research Design” Publisher: SAGE Publications Ltd.

Newman & Benz 1998 cited in Creswell 2014 “Research Design” Publisher:

Sag Publications Ltd.

Creswell and Clark 2007 cited in lecture slides.

Page 35 of 37

“Doing Quantitative Research in Education with SPPS” 2010.

http://www.oxfordjournals.org/our_journals/tropej/online/ce.html

www.eventimpacts.com/economic/intermediate/economic-calculator/

Alan Bryman 2008 “Social Research Methods” Publishers: Oxford University

Press.

Brannen 2005 “Mixed Methods Research: A discussion paper” ESRC National

centre for research methods.

Baker and Associates 2007 - Glastonbury Festivals 2007 Economic Impact

Assessment.

“Journal of tropical Paediatrics” Publisher: Oxford University Press.

Israel & Hay 2006 cited in Creswell 2014 “Research Design” Publisher: SAGE

Publications Ltd.

Page 36 of 37

L. Bibliography

Davies & Maud1995 “Under Milk Wood the definitive edition” Publisher:

Orion Publishing Group.

Chris Gratton and Ian Jones 2004 “Research Methods for Sport Studies”

Published by Routlidge.

Dawson 2009 “Research Methods” Published by How to books Ltd.

Veal (2011) “Research Methods for Research and Tourism”

Published by Pearson Education Ltd.

www.walesonline.co.uk/news/wales-news/councils-wales-worse-more-3-

7905139

Gwynedd Council Report to Cabinet May 2013.

Floris Langen University of Glasgow and Beatriz Garcia (Impacts 08) 2009

measuring the impacts of large scale cultural events: A literature review

Published Impacts 08 European Capital of Culture Research Programme.

Colin Michael Hall 1992 Hallmark Tourist Events – Impacts, Management &

Planning Published by John Wiley & Sons.

Veal 2006 Veal 2006 “Research Methods for Leisure and Tourism” Publishers:

Pearsons Educational Ltd.

John W Creswell 2014 Research Design – Qualitative Quantitative & Mixed

Method Approaches Published by Sage Publications 2014.

Anthony J Onwuegbuzie University of South Florida & R. Burk Johnson.

University of South Alabama 2006 “The validity issues in Mixed Research”

Mid South Educational Research Association.

Julia Brannen 2005 “Mixed Methods Research – A discussion paper ESRC

National Centre for Research Methods.

Journal of Tropical Paediatrics – Oxford Journals. www.oxfordjournals.org/our_journals/tropej/online/ce.html “Application of a Case Study Methodology” by Winston Tellis The Qualitative Report, Volume 3, Number 3, September,1997. www.nova.edu/ssss/QR/QR3-3/tellis2.html Baker and Associates 2007 - Glastonbury Festivals 2007 Economic Impact

Assessment.

Page 37 of 37

Colin Michael Hall 1992 “Hallmark Tourist Events” Publishers: Belhaven

Press.

www.erm.ecs.soton.ac.uk/theme4/aims_and_objectives.html

Miles and Huberman, 1994, P18 cited in a presentation by Professor Roger

Vaughn of Bourmouth University 2008.

www.eventimpacts.com/economic/intermediate/economic-calculator/

Rogers “Events are Great” for Business Visits & Event Partnership.

Maughan& Bianchini 2004 “The Economic and Social Impacts of Cultural

Festival in the East Midland of England” - De Montford University Leicester.

Wood and Pratt 2006 “Qualitative Research” Faculty of Education University

of Plymouth.

Onwuegbuzie & Johnson 2006 “The validity issues in mixed research”

University of South Alabama.

“Introduction to qualitative research”. http://www.sagepub.com/upm-data/36869_muijs.pdf