Student Staff Selection Eval

RUNNING HEAD: Student Staff Selection 1

Loyola University Chicago’s

Department of Residence Life

Student Staff Selection: Evaluation Plan

Tyeishia Banks and Crystal Norwood

STUDENT STAFF SELECTION PROCESS 2

Table of Contents

Introduction ........................................................................................................... 3 Description of Loyola University Chicago ..................................................... 3 Description of the Resident Life Program at Loyola ..................................... 4 Description of the Student Staff Selection at Loyola ...................................... 5 Program Description ....................................................................................... 6 Student Staff Selection Goals and Mission ..................................................... 6 Stakeholder Analysis ...................................................................................... 9 Literature Review ............................................................................................ 10 Logic Model Description ...................................................................................... 11 External Factors .............................................................................................. 14 Evaluation Design ................................................................................................. 17 Purpose ............................................................................................................ 15 Goal ................................................................................................................. 15 Quantitative Approach ......................................................................................... 17 Survey Population .......................................................................................... 17 Survey Design ................................................................................................. 17 Implementation ............................................................................................... 17 Pilot Testing ................................................................................................... 19 Statistical Analysis .......................................................................................... 20 Qualitative Approach ........................................................................................... 21 Focus Group Plan ........................................................................................... 22 Focus Group Implementation ........................................................................ 25 Focus Group Analysis .................................................................................... 26 Focus Group Limitations ............................................................................... 27 Final Report ................................................................................................... 28 References .............................................................................................................. 30 Appendices ............................................................................................................. A-L Appendix A: Logic Model ...................................................................................... 31 Appendix B: Survey Matrix ................................................................................... 33 Appendix C: Survey ................................................................................................ 38 Appendix D: Focus Group Protocol ....................................................................... 43 Appendix E: Focus Group Note Sheet .................................................................... 46 Appendix F: Informed Consent .............................................................................. 49 Appendix G: A Prior Construct Map ..................................................................... 51 Appendix H: Student Staff Selection Timeline ...................................................... 52 Appendix I: Budget ............................................................................................... 53 Appendix J: Survey Communication ...................................................................... 54 Appendix K: Focus Group Communication ........................................................... 56 Appendix L: Final Presentation .............................................................................. 59


Introduction

As part of the Residence Life community that strives to serve diverse identities, several

student populations are absent and overlooked. This student population includes students of color.

Diversifying the student applicant pool is crucial to the department because they recognize the

essential role that diversity plays in educational excellence and developing communities. The

department of Residence Life is dedicated to recruiting, hiring, selecting, promoting, and

retaining a competent and diverse staff that reflects the cultural and personal diversity of Loyola

and greater Chicago community. We are concerned about the issue and are exploring it in the

evaluation process due to the lack of applicants of color applying for positions. The problems

surrounding the Selection Process focus on unsuccessful recruitment strategies. The recruitment

process and strategies have shown to recruit great student staff leaders, but show a small amount

of diversity in the selection pool.

We recognize the issues with recruiting and retaining specific student populations, but

have not found sufficient ways to use this information for the student staff selection process. This

process recruits students for the resident assistants (RA) and learning community assistants

(LCA). Discussions within the department are on stigmas, areas of discomfort, and assumptions

when making the final placement decision. Another stigma focuses on the impact of a small

professional staff identifying as people of color, and the effect they have on students of color

during the selection process. An area of discomfort that is most apparent during the placement

process when placing students of color in the LUC residential communities. As a department

they gather valuable information pertaining to the student staff population but are not successful

in implementing this results for the improvement of the selection process. The end results are the


completion of the selection process, along with placement and tokenizing the limited amount of

student of color as the diversity in each residence hall.

Loyola University Chicago

Loyola University of Chicago is a four-year, coeducational, private research university

located in Chicago, Illinois. Loyola is a major piece of the city of Chicago and it is comprised of

four campuses in the Chicago area, and three international campus centers (Rome, Beijing, and

Vietnam). It was established in 1870 and is affiliated with the Roman Catholic Church, and is

the largest Jesuit University in the country (Loyola University Chicago, 2013). The institutions

mission states “We are Chicago's Jesuit Catholic University- a diverse community seeking God

in all things and working to expand knowledge in the service of humanity through learning,

justice and faith”(Loyola University Chicago, 2013). Loyola’s mission is to the whole person

and supporting the lifelong professional development that is demonstrated through all

departments on campus.

Loyola University Chicago students, staff, and faculty appear to benefit from Loyola’s

Jesuit education and values. Loyola University Chicago Jesuit values allude to seeking

knowledge in the service of humanity; preparing to live in a shared, global community; and

building on the desire to always do and become more. Loyola University Chicago’s Jesuit

tradition distinguishes Loyola University Chicago from other colleges and universities. Loyola

values the view of pursuit of knowledge, the embracing of faiths, and the promotion of justice as

intrinsically related.

Department of Residence Life

Loyola's Department of Residence Life manages all residence halls for the on-campus

student population of 4500 plus students. The department oversees and operates a total of twenty


residence halls including apartments; nineteen located in the Rogers Park and Edgewater

neighborhoods surrounding the Lake Shore Campus and one at the Water Tower Campus. The

vision of the department is to foster development and to support the vision, which states “Our

residential communities prepare people to lead extraordinary lives by integrating learning,

spiritual growth and development. We contribute to students’ transformative education by

offering student centered programs, services and environments that foster student involvement,

responsibility and leadership” (Loyola University Chicago, 2013).

Loyola University Chicago, Department of Residence life serves a multi- dimensional

population, which includes: learning communities, first year communities, upper-class

communities, transfer students, and graduate students. At Loyola University Chicago the

mission aligns and coincide with the departments vision. One of the department’s core values is

diversity. It states, “We value civility, respect, appreciation, and an in-depth understanding of

diversity including the ability to put knowledge into meaningful action” (Loyola University

Chicago, 2013). The mission of the Residence Life Department focuses on “enhancing the

campus experience by creating transformative environments by providing safe, secure residence

halls and inclusive communities where students, staff, and faculty integrate key academic, social,

spiritual, and recreational experiences. The overall commitment is to ignite the passions and

social responsibility of every individual in partnership with our residents” (Loyola University

Chicago, 2013).

The goal of the Department of Residence Life is to form a cohesive 100+ Student Staff

for the following academic year. The department works to create a diverse and mission driven

identity each year. Positions offered by the department include but are not limited to Resident

Assistants, Furniture Crew, Student Office Assistants, Desk Managers and Receptionist. These


positions are highly valued because each serves as unique contributor to the student experience.

Each position serves as a peer to peer resource and assist in the development of the community at

Loyola University Chicago.

Student Staff Selection

The Student Staff Selection process is a leadership opportunity offered to students who

are looking for opportunities of meaningfully contribution to the residential community at

Loyola University Chicago. Loyola University Chicago (LUC) Department of Residence Life

initiates this opportunity. The Student Staff Selection process intends to recruit an estimated 100

students across different disciplines, identities, and class standings. Student Staff Selection

engages and allows students to deepen their own development personally or professionally

whether or not chosen for the position.

As mentioned previously the mission of the Department of Residence Life supports this

process of student engagement in during student staff selection. Part of the department mission is

that they provide integrated learning, which supports the group process students go through

during selection. Another part of the mission is that the selection process supports student

development. In the process the department provides opportunities of professional development

through one on one interview process. The importance of the student leader positions is

reinforced through Group Process. Group Process is a two-day, in-depth process that captures

group interactions. During the group process the Department of Residence Life observes each

candidate's knowledge and understanding of a variety of competences, communication and

leadership skills, individual and group manners, and participation as they interact as a group to

address each prompt. Through this process students understand that through training and

accessible resources that can have an impact on students.


Through these positions Resident Assistants become agents for change and allies for their

communities and other student populations. Through this process the Department of Residence

Life also hope students understand the department’s mission and values. As a department they

aim to hire strong leaders they can ultimately be a resource for different identities such as sexual

orientation, socioeconomic status, first generation, religion but not limited to racial and ethnic

identities.

The Student Staff Selection process has played a dynamic role in developing the student

leadership community, and has created opportunities for leadership among the growing student

populations. With the selections process serving as a recruitment tool for the Department of

Residence Life’s evolving and changing culture, the selection process is forever changing. As the

department makes changes to better assist the needs of the current student population, the

selection process will need to continue make the similar changes.

Currently the Department of Residence Life lacks the archive of the history of their

selection process prior to 2010. Currently we have a new Director but not new to the institution

or department. Prior to him accepting the Director position, the former director and professionals

within the department started to implement structure with the selection process. Since then there

has been no archiving of materials to evaluate and improve the program. The student Staff

Selection has operated without official written goals, objectives, or learning outcomes stated for

the overall process (Katie Rutkowski, personal communication, October 2013). The department

has operated under the impression that staff understands the goals and purpose of Student Staff

Selection. The Department of Residence Life every Spring Semester implements a selection

process and has operated each year without documenting goals, objectives and learning

outcomes. In speaking with the Assistant Director, the Student Staff Selection has changed over


the years, whether adding positions, revamping group process, or even the marketing strategies

(Katie Rutkowski, personal communication, October 2013). As a result, challenges and success

have developed.

Student Staff Selection typically starts in February and last until late March ending with

the filling of positions. In order to prepare for student staff selection, the department specifically

created a Selection Committee who is comprised of Graduate Assistants within the department

and Professional Staff members (Resident Directors, Assistant Directors and Area Coordinators).

The committee is tasked with creating the timeline for student staff selection, overall group

process, interview breakouts, marketing, hiring, and assessing the overall process. During the

recruitment process the Department of Residence Life focuses on cost efficient marketing, and

forms of outreach that encourages other partners to get involved with the selection process.

Previous year’s advertisements that were used during the recruitment included forms of

tabling with current student staff, items such as coffee sleeves based on the culture of the campus,

informational sessions, and informational flyer. The committee is created by mid-September and

meets and average of three times a month to prepare for the spring. Being that the student staff

selection process committee’s primary focus it a time to prepare for the high traffic period of the

year. As members of the committee in 2012-2013, we (Crystal and Tyeishia) have been able to

develop some general themes that we believe can become learning outcomes for Student Staff

Selection.

The following outcomes are listed below:

● Students able to better articulate the department’s values and mission.

● Students will be able to experience a professional interview process to deepen their skills

and being able to articulate their experience.


● Students will be able to deepen their understanding of the position applied for and

become knowledgeable further in regards to the process.

● Returning student staff will be able to better articulate how they have grown throughout

there time in their position as well as how they have changed.

Importance: Stakeholders

Stakeholders are important in order to maintain a strong collaboration and united front as

a part of the Division of Student Development. They are the agents that will implement the

changes that any office goes through and will be able to provide their insight on best practices.

Wholey, Hatry, and Newcomer (2010) define stakeholders as “individuals, groups, or

organizations that can effect or are affected by an evaluation process or its findings (p.31).

Direct stakeholders are those individuals that have a direct or immediate stake in the program

while indirect stakeholders are those individuals who might indirectly benefit from the success or

improvement of the program.

Direct & Indirect Stakeholders

For the means of this evaluation the direct stakeholders include the Residence Life staff,

graduate assistants and participants involved in the process. The first’s sets of stakeholders are

the students, which estimate about two hundred. These are students who have been selected to

move on in the selection process, upon submitting their application. The students who are hired

are the front line of communication and strive to create a safe and secure atmosphere for the

residents within the residential communities. The students that are hired serve as an academic

support, policy enforcers, community builders, program implementers and role models for their

community. Another pair of stakeholders are the individuals that oversee activities and


interviews during the selection process. These individuals are Residence Life staff that will

eventually oversee and work of the student leaders throughout the year.

Each of these individuals can be seen as direct stakeholders because of their involvement.

Without them the selection process could not be implemented. Primary intended users are

stakeholders within our evaluation project. Primary intended users are “specific stakeholders

selected to work with the evaluator throughout the evaluation to focus the evaluation, participate

in making design and method decisions, and interpret the results to assure that the evaluation is

useful, meaningful, relevant, and credible” (Wholey et al., 2010, p.31). The primary user

identified in this evaluation approach is Katie Rutkowski who is a full time professional in the

department committed to this selection process.

Other groups affected by the evaluation approach are indirect stakeholders. Within this

evaluation approach the indirect stakeholders are the residents that make up the on-campus

residential population. This comes into effect because the department needs to hire a diverse

student staff to meet the needs of our students, department and institution. The residents and the

surrounding Loyola community benefit because it’s an opportunity for students to seek out a

mentor.

The investment that stakeholders have in an office or particular programs also allows for

them to influence other stakeholders and share their thoughts and experiences to provide new

suggestions (Wholey, Hatry, & Newcomer, 2010). As previously mentioned there are several

stakeholders throughout the Student Staff Selection Process, and to ensure each student staff

learns the information intended for them the stakeholders must be present.

Literature Review


The conceptual framework for the Student Staff Selection process has originated in

several articles and documents. Alimo et al (2002) stated that students believe that there most

significant interactions with those racially and/ or ethnically different from themselves were in

residence hall. This supports our evaluation concern of seeking a diverse student staff population

to foster and support these future conversations. Other articles used were Student Perceptions of

Campus Cultural Climate by Race. This article suggest that by understanding of the particular

experiences and perceptions of African American, Asian American, Latino/a, and White college

students may also influence the development of culturally relevant and effective interventions

(Ancis, Sedlacek, Mohr, 2002).

The Cultural Attitudes and Climate Questionnaire (CACQ), discussed in Ancis, Sedlacek,

and Mohr (2002) measures students’ perceptions and experiences of the university racial and

ethnic climate and advocates for the Department of Residence Life mission and values. As the

Student Staff Selection process continues the department will need to continue to benchmark

against other insinuations and develop its philosophy to dive their belief of diversity and

community in the department.

Logic Model Description

The structure of the Student Staff Selection process at LUC is described in the logic

model, found in Appendix A. A logic model starts with long-term visions of how participants

will be better off because of a program (Fitzpatrick, 2004, p.70). The model transitions from

resources, to inputs, to outcomes as illustrated in the logic model.

THIS IS WHERE I STOPPED Inputs are described as the resources the Department of

Residence Life invests in the Student Staff Selection. Fitzpatrick (2004) identifies inputs as

annual budgets, staffing facilities, and materials, needed to run a program. The most critical


individuals to this process are the department’s selection committee who has coordinated all

aspects of the Student Staff Selection process. In addition there are outside staffs that are not

members on the department committee, which include Assistant Resident Directors (ARD),

Resident Directors (RD), Area Coordinators (AC), Associate Directors (AD) and the Director

who plays an indirect role. These members take in consideration other inputs such as recruitment

strategies, funds allocated for resources (printing, room reservation, food), and student staff

necessities. Outputs describes what takes place within the Student Staff Selection process and

who participates in the selection process. Fitzpatrick (2004) suggest output as including the

number of participants each week by demographic category, the number of class meetings, and

even hours of direct service to each participant (p.79). Some outputs are team builders,

icebreakers, selection social, group process, and interviews.

Outcomes describe the impact of the program in the LUC community and department,

with both short and long term goals. To achieve these goals certain assumptions must be made of

the Student Staff Selection process and the participants. Additionally there are factors that should

be considered when implementing this process each year. However the following sections below

provide a more detailed understanding of these sections of the logic model (Appendix A).

Outcomes

The outcome structure shows the benefits of the participants and the organization

involved. These short-term goals are intended to lead to medium goals. Some short-term goals

are giving students the ability to learn and explore the positions with deeper understanding pass

the basic compensation provided. Secondly, to be able to understand the selection process and

developmental structures put in place such as the interview. The interview structure use by the

selection committees mimics a professional interview process. Moving from short-term goals to


medium goals within the logic model (Appendix A) one goal is to assess where students are at in

supporting the departments mission and values. Additionally, another goal is create comfort

within other students and staff. The medium goals are ultimately within the process to lead to a

long-term goal.

The ultimate long-term goal is to hire a cohesive student staff that can articulate their

values and what it means to live out the department’s values and ideals. Another long-term goal

is to hire a student staff that is reflective of the student population at LUC. An important long-

term goal is that students will be equipped with knowledge to approach any situation and the

skills and techniques to build on their own development and articulate these capabilities on a

resume.

Assumptions

As we approach this evaluation project we see importance in recognizing that certain

assumptions will be made. These assumptions whether they are intentional or unintentional may

influence the outcomes. We acknowledge that we bring our biased perspective when

approaching the project as previous members of the selection committee and current staff of the

department. Our logic model includes assumptions that influence our evaluation. First, we

assume that students will be able to benefit from the group process session. Although every

student anticipates to be hired, we assume if a student does not receive a position that they will

reapply next year or apply for a different staff position within the department. We also assume

that students will be able to learn something about themselves as the go through the process and

about the department. Lastly, we assume that through the group process students will realize if

the position is what they are looking for and by participating in the process they will be able to

attain and reproduce information in different situation as they encounter them.


External Factors

There are few contextual events or circumstances that could affect the program in ways that may

distort the evaluation. There are many factors that intersect and some that are out of the

department’s control. One of the largest factors reflects the preparation of students during the

group process portion of interviews, which may present a challenge. Another external factor

could be the amount of informational session being offered, which leads to fatigue from students

and staff members. A third circumstance that could have a large effect on the evaluation process

is the interviewer bias as well as questions that pertain to a specific group of students.

Additionally, the lack of diversity in sessions and activities can contribute as external

factors. The recruitment dramatically affects the selection process and the direction in which

selection will unfold. Lastly, follow up is a very important factor. If there is no follow up it can

contribute negatively to the process. These factors are able have a short term or long term effect

on the process when identifying and defining what the program should and can be.

Evaluation Approach (Combination)

The evaluation approach that we will use for the Selection Process is formative. We

chose to use this method because the area we would like to focus on is methods to improve the

delivery of the selection process. An evaluation is considered to be formative if the primary

focus is to provide information for program improvement (Fitzpatrick, 2004, p.16). Formative

evaluation is beneficial to the selection process because of its goals to provide information for

program improvement through the lens of the students at LUC. As previously stated the

problems surrounding the selection process focus on unsuccessful recruitment strategies. The

problem with the process is that it does not allow the department to reach the overlooked student

populations. This helps the Department of Residence Life see success with recruitment but also


the needs for improvement with the recruitment, marketing, and outreach plan. Without reaching

each student population the results of the selection process lacks a diverse student staff.

For the formative evaluation we are taking a combination approach of needs assessment

and process evaluation. The needs assessment will determine if there is a need or problem with

the delivery of the Selection Process. Fitzpatrick (2004) suggest that needs approach questions

are concerned with addressing whether a problem or need exist. After determining if a problem

or needs exist the next step is making recommendations for ways to reduce the problem

(Fitzpatrick, 2004, p.21). The use of the needs approach will be determined by the quantitative

data collected by the Department of Residence Life. The quantitative date will reflect the

applicant pool, accepted staff, and attendees of information sessions.

The process approach will focus more on the delivery of the selection process. The

process evaluation will allow us to determine if the outlined outcomes of the selection process

are being met (Fitzpatrick, 2004, p.21). Process evaluation describes how the programs is being

delivered with the following area as a focus:

• Nature of deliver

• Successes and problems encountered

• Quality of delivery

• Delivery environment

• Printed material One area that will be assessed in this evaluation is the marketing, which includes

informational sessions, which are mandatory for students interested in applying for any student

staff position. We will also need to determine if the timing of the informational sessions and the

marketing locations create issues with the selection process. These approaches are appropriate

because they will allow students to disclose areas that may need to be reevaluated and improved


upon. We think the best way to evaluate the selection process is to attack the problem at the

source, and the source in this situation is the delivery of the recruitment process and the pol of

students being reached by recruitment efforts.

With this approach there are strengths and weaknesses. We believe some areas of

strength will be the use of information gathered from the students who attend the information

sessions, participates in the interview process, and student who accept the position. This may

also serve as a weakness because the information gathered may not help us in answering our

evaluation question nor may it be of use. Another issue is the information gathered each year has

never been analyzed for these of assessment implementation (Rutkowski, K, 2013). This may be

a weakness to the approached based in the timing allotted for the selection process. If the

information gathered is not consistent across all student groups this can be a weakness of the

process. Some information that will help in the evaluation process that was not previously

collected is demographic, gender, first generations college student. This forces us to relay on

information students have decided to disclose.

Some weakness of this approach focuses on the gathering of information from students

participating in the Selection Process. If students decide not to provide personal contact

information or if they do and the information change this presents a challenge with the

evaluation of delivery. This lessens a pool of students that are represented in low numbers. If

current student staffs are being questioned on topics of recruitment, the information gathered

may be skewed because this student population “fits” the outreach process. This may present

biases when evaluating this selection process. The last weakness reflects formative approach, and

evaluating the process from the student perspective. If the student feels there is no an issue of

diversity in their staff this could potentially change the approach from formative to summative.


Survey Population

This survey will be administered as a testing instrument following the recruitment

process to students meeting the student criteria for the survey (Appendix C). The administrator

of this survey will be Tyeishia Banks and Crystal Norwood, who are evaluating the Residence

Life selection process in terms of marketing, interviews, and group process. The type of

sampling that will be used is a criterion sampling. In criterion sampling, we will utilize a sample

with criteria identified prior to a treatment or program. This approach seems to be an effective

way to receive information regarding marketing, and diversity. With this approach 114 invite will

be sent to current student staff members asking them to participate in the survey and focus

groups. In addition this specific sampling it will allows us to sample the student population

fitting the criteria to capture more data for a more effective program evaluation.

Survey Design

This research will utilize a cross-sectional design as we are attempting to determine the

effectiveness of marketing. The foundation of the group process will give us the ability to select

and hire students that are diverse but also serves as a representation of the staff population. Due

to returning staff in the positions, we have the ability to assess the perceived knowledge gained

from student staff’s selection. This evaluation will utilize a cross sectional design as we are

attempting to determine the effectiveness of the selection process. This will reflect of the

marketing strategies, diversity amongst candidates and overall structure in the group process

over a specific point in time; specifically directly after the staff is hired (Creswell, 2009). A

cross-sectional design will assist us as we work to determine that the student staff selection

process is reaching its intended outcomes, as well as creating a diverse candidate pool in relation

to the student population at LUC. Cross- sectional design is used to examine trends and patterns


across a single point of date collection (Schuh, 2009). By using this method research concludes

that we will be able to pinpoint any abnormal results of the selection process and identity areas

of improvement.

This design will assist us in our evaluation regardless of the student staff status of

returner or a new staff member. We want to avoid survey fatigue by doing one long survey

verses several short surveys for the students (Schuh, 2009). There will be no comparison group

because all of the students will have gone through the same student staff selection process. The

survey will be structured to ask questions regarding marketing, overall satisfaction, diversity and

demographics. Questions regarding marketing in relation to Resident Assistants will reflect their

experience and marketing effectiveness. Is it the locations of marketing or is it the type (tabling,

emails, etc.) of marketing. Specifically, the survey will contain 29 questions covering these

specific target areas: marketing, diversity, overall satisfaction and demographics (Appendix C).

Implementation

Since we have formulated the questions, we serve as the main contacts for the

distribution of the survey. In the future years, the chair or the co- chairs of the Departments

selection committee, a committee that plans and implements all departments’ staff hiring, would

oversee the survey distribution and evaluation process. Qualtrics, research software will be used

to implement the survey and the committee chair would provide the company with the necessary

information to effectively distribute the survey.

The survey would be distributed within a week after the selection process with the

expectation that the Resident Assistant population will complete the survey. To ensure that we

are receiving fresh perspectives from the student populations the expectation is for the survey to

be completed within two weeks of receiving the email (Appendix K). There will be an incentive


of $50.00 LUC bookstore gift cards for students as the survey is distributed, in efforts to gain

more participants in the survey. Upon completing the survey student will be place in a raffle for

the drawing of five gift cards. We (Tyeishia and Crystal) will remind and encourage students to

complete the survey in regular intervals. These intervals will occur once a week for three weeks

in a series of emails (Appendix K). In addition Qualtrics will notify us of who has not completed

the survey after the first week to help in the reminder process for the survey.

Survey Instrument

The survey has twenty –nine questions using radio buttons for participants to fill in

Likert, interval scales, and nominal scales. This survey should take approximately 15 minutes to

complete. The survey include approximately 6 student demographics questions and topic areas

based on the 4 programs outcomes of the Student Staff Selection process: marketing, overall

satisfaction, diversity and group process (See Appendix A).

Pilot Testing

A pilot test of the survey will occur before the survey is in its final form. According to

Schuh (2009), pilot testing can help uncover flaws or possible biases in the instrument. In

attempt to get the most useful perspectives as possible, current Loyola RA’s that are returners

will be asked to participate in pilot test of the survey. These students will serve as the group to

recognize any significant missing components of the selection process experience, as well as the

providers of feedback on area to address in the survey. These students would be the ideal

candidates to test if the various questions of the survey flow naturally or need improvement

(Wholey et al., 2010). Pilot testing is necessary to test for clarity and eliminate any errors in the

questions.


The procedure for pilot testing the returners (RA) would go through two sessions: survey

and feedback. In the first session, the board members would look over and take the survey. The

second session would allow for the participants to discuss issues with the survey, allowing

researchers to assess feedback for the consideration of change, clarify any miscommunications,

and improve the survey. It will be beneficial because they have been through at least a year of

the process.

Statistical Analysis

Several statistical analyses will help explore trends in data. Measures of central

tendencies will be helpful as we calculate the mean, median, and mode for all continuous

variables (Wholey et al., 2010). These variables are measured by interval scale on the Likert-

type scale, which helps identify general patterns. The data will be looked at in relation to

demographics including year in school, gender, race, and involvement within the department of

residence life.

As previously mentioned we will use interval scale as a means of measurement. The

scale we are using is a Likert-type scales that will provide a variety of response possibilities to

the questions ask in the survey. During the evaluation we will use inferential statistical analyses.

In the survey we will examine gender, race, ethnicity and the effects of our marketing on

the current student population at LUC. For this approach we have chosen not to compare results

across a treatment and control group because we are using a criterion sampling. We chose

criterion sampling because we want to review and study all cases that meet some predetermined

criteria of importance as it reflects the student populations as we analyze the results. We have

decided to use ANOVA as our approach to compare several different variables. With this

approach we are looking to find how marketing is perceived from college students and how it


affects the different populations at LUC. The information we are gathering will help understand

how different factors such as class standing and residential live effects our study.

SPSS will serve as a tool to run analysis in descriptive and inferential ways. ANOVA

test will be used to show differences in outcomes across different categorical variables. ANOVA

will assist in evaluating the development of the student staff selection process in relation to three

or more groups. By using independent demographic variables such as race, gender, and class

standing we can assess how students perceive marketing from the department and the intended

effectiveness of its use. The test will determine if differences in participant’s race, gender, or

identities have an impact on their understanding of the department’s marketing strategies.

After the data is collected, Qualtrics will provide the results in bar graph form, depicting

the percentage of each answer gathered from the participants. Our final report will include these

graphs that contain the results of each answer presented in the survey. Utilizing this information

a comparative line graph will be included depicting the levels of competence in students of color.

This will also be used for the white students to determine if there is a significant difference in

these levels as well.

This survey will determine if time, resources, and staff are being utilized

effectively for student staff selection. With these results, we will be able to improve future

selections processes to ensure the information intended is delivered and easily understood

by the students, along with students attending the information session

Qualitative Approach

After having the selected students complete the evaluation survey for the Department of

Residence Life, on marketing regarding strategies and its importance to further engage them.

Qualitative research derives its strength from gaining insight and understanding by studying a


purposeful sample (Patton, 2002). This engagement will further discussion to ensure that we ask

for and receive as much information we can about student staff selection. The qualitative

approach of the evaluation will include focus groups. We are using qualitative research, “the idea

behind qualitative research is to purposefully select participants or sites that will best help the

research to understand the problem and research question” (Creswell, 2009, pg.178).

In improving both the process and the outcomes of the student staff selection group

process, the focus groups will ask questions to address both aspects. The answers provided from

the student will be linked to the process through specific questions about satisfaction of the

process, structure of the selection process, and the learning outcomes and goals set by the student

staff selection committee and the Department of Residence Life.

Wholey, Hatry, and Newcomer (2010) wrote that focus groups work well for evaluating

programs because they give descriptive answers and explanations as to the reasons behind the

results of the quantitative aspect. Focus groups allow us to collect data with questions posed in

interactive group settings that will encourage participants to speak honestly and freely with other

group members (Kress & Shoffner, 2007). This form of data collection will be implemented so

that we can get a better understanding of the learned outcomes of the Student Staff Selection

process.

While the survey focuses on the marketing efforts of the Department of Residence Life

and evaluates the effectiveness of selection marketing, the focus groups will be used to evaluate

the outcomes achieved from the selection process. It allows for reflection of students regarding

their experience of new and returning resident assistant who completed the process. The

evaluations are administered to students, because of the information needed to deepen our

understanding of the student staff selection process. Intended participants will be contacted via


email. The email will serve as the invite to the focus group (Appendix K). Anticipated dates for

delivery of emails will be tentative to the selection process. A series of informational emails will

be sent to the focus group invitees (Appendix K-L) explaining the purpose of the focus group.

This email will explain why they were selected to participate in the data selections process, and a

structure for the focus group. This structure will include duration of the groups the number of

expected participants, moderators, evaluators and form of transcription being used.

Focus Group Participants

Creswell (2009) wrote about purposefully selecting individuals who will best help the

process of the evaluation. The survey will focus on all the RAs experience in the selection

process. Wholey et al., (2010) wrote that focus groups participation all have something in

common; Upcraft and Schuh (1996) labeled this as utilizing “homogenous samples” (p.56).

While all the RAs been in the RA role or went through at least the process they will best able to

inform us about the information throughout the selection process what needs to be improved.

Using the focus groups to determine why the students were not satisfied will help improve the

Student Staff Selection for the future. This would give us a pool of approximately 100 students

who can be invited to be a part of the focus group but we are only selecting 30 students in total.

The cap for the focus groups will be 10 participants per group, with a goal of 3 focus groups. We

will split the group evenly based on availability into three groups with no more than 10 members

per group with a total of 60 minutes allotted for each focus group. Our goal is to get 30 students

to volunteer, giving us 100 percent volunteer rate. Participants will be contacted via email as

previously mentioned with a formal focus group invitation, as noted in Appendix K. Initially,

survey respondents who expressed interest in participating in focus groups will be contacted to

confirm their participation, approximately 3 weeks after participating in the survey. A second


invitation for participates will be sent a week later. Three days following, an email will be sent to

participants who responded about participating, to confirm the date, time, and location of the

focus group sessions (Appendix K).

Focus Group Protocol

The focus group protocol instrument will be used to collect information about students’

experience during the selection process (Appendix D). The protocol contains the purpose of the

focus group and outlines the structure of the focus group. The focus groups will be conducted on

campus for participants’ convenience and will take place in the Regis Hall Conference room.

The note- takers who will tape record and take notes during the focus groups to accurately

capture everyone’s responses. (Appendix D). Participants will be notified that their identities will

remain confidential and only the necessary Residence Life staff will use and have access to the

information collected in the focus group.

The structure of the focus group will include introducing of participants by stating their

name, class standing, major, and which staff they are on. The questions asked during the focus

group focus on the student’s experience during the selection process, including meaningful

moments, learned outcomes, challenges, things they liked and disliked, and changes they would

like to see for the process. Students are also asked to reflect on the pool of candidates and their

thoughts on the pool representing the LUC student community. As previously mentioned the

focus group will roughly last sixty minutes. The department will use internal departmental

members to serve as moderators. This person will have experience working with the student

population, but that does not serve as a direct supervisor.


Focus Group Implementation

The focus groups will be implemented during the spring semester; approximately 3

weeks after participants have accepted their positions as Resident Assistants. The 3-week time

frame will allow the Department of Residence Life to hire all candidates’ need for the positions,

and to correspond with the candidates. This will give students the time to reflect on the

interview process, without forgetting their experience, and emotions that came up for them

during the selection group process. We will keep stakeholders well involved because, it is

important to provide experiences to the administrators that will be applying the data collected to

the selection process. All of the focus groups will be held on a day and time that is determined to

work for all members of the focus group to ensure consistency.

The focus groups will occur in the Regis Conference room, a room with a long oval table

where the participants and the moderator can all be on the same level and maintain eye contact

with each other. A graduate student will be the assistant moderator and two graduate students

will be taking notes of the focus group’s answers on a notes form designed for this focus group

(Appendix E). We will use incentives listed by Wholey et al. (2010): food, in gathering

participants, we will explain to the RAs that their opinions and feedback is valued greatly,

hopefully making them want to provide feedback. A second incentive, lunch from Jimmy Jones

will be provided for the participants.

In addition to being the incentives, as “the focus group environment should be

comfortable” (Wholey et al., 2010, p. 381), it is our hope that sitting around and eating lunch

together will make the process feel like a casual conversation, leading more students to openly

share their opinions with us. The session will be recorded, along with the two graduate assistant

note takers in order to ensure that no information is missed, as “it is important to remember


everything that is said in a focus group” (Wholey et al., 2010, p. 394). At the end of each

question set, the moderator and note takers will work together to ensure that the focus group’s

answers were understood correctly before moving to the next section and also assist with the

analysis of the data.

Focus Group Analysis

The focus group analysis will occur through transcriptions, and coding schemes.

Creswell’s (2007) chapter suggests, “the process of data analysis involves making sense out of

text and image data” (p. 183). For the focus groups we will transcribe and code for the analysis.

Coding is described as organizing material such as text and making meaning of the information

(as cited in Creswell, 2009). The department’s moderators will need to listen to the audio

recordings, and also analyze the notes taken by the note takers (Appendix E). We will utilize two

writing strategies Creswall (2009) mentioned: direct quotations from participants as well as

interpretations of the data (pp.193-194). In addition, a member of the department will be asked

to transcribe the data collected from the recorders to ensure complete transcription of the

sessions. Following this process the notes will need to be transcribed. For a more accurate coding

the moderators and note takers will need to cross analysis the first focus group. For the second

and third focus groups transcription will be done independently after those analyzing the data

first to establish some level of inter-rater reliability to ensure coding is being done consistently.

For the coding rubric, constructs will be based on prior notions of what we believe to be

important area to focus on in the analysis process. The constructs are: Preparation for Position

(PREP); Diversity (DER); Opportunities of Improvement (IMPV); Climate of Diversity (CLIM);

Aspects of Marketing (MARK) (Appendix G).

. These are constructs that have been determined before the focus group but we anticipate


other themes and constructs will be added. Previous to the focus groups the evaluators will

receive the coding rubric for their review (Appendix G). We will use the cutting and sorting

technique because it allow use to develop more natural codes that emerge from the focus groups

(Creswell, 2009, p. 84). Our goal is to have validity in our evaluation by using the triangulated

method of a quantitative data source and a survey. This will also help in the process of justifying

the themes.

Limitations

A few limitations exist in the qualitative evaluation design that will impact the results of

the evaluation. Due to committee structure in the department and moderators it is possible that

their bias or understanding or understanding of the selection process and the intended outcomes

may cause a certain level of bias as they facilitate the focus groups. Since they are familiar with

the intended learning outcomes for the participants, their presence during the focus group may

hinder participants from being honest about their experience.

The dynamic between staff members and participants may shape the types of data that is

collected in the focus groups. Since there will be three different focus groups, it is likely that the

direction of the conversations may alter based on the participants present in each session. It is

important for each moderator to maintain a level of consistency as well as a neutral stance. To

counter these limitations we will provide guidelines and tips for maintaining unbiased behavior

and maintaining the conversation direction. Since the department has been in transition for the

past two years with new staff members and directors, much of the rich history and knowledge of

the selection process has not been archived.

While we have staff members who have been in the department during its transition they

may know it still lacks the data, including the perspective of the participant who did not receive a


position. This evaluation plan begs the question; are we serving our student staff based on the

needs of the residential community or what external factors and consideration are we taking into

account?

LOGISTICAL INFORMATION

Timeline A timeline (Appendix H) is attached to serve as a guideline for future implication of the

evaluation plan. The timing of the evaluation plan is crucial to the success of the evaluation

instrument and accuracy of the data collected. Ideally with the student staff selection survey

instrument and focus group all should be done within 30 days of the completion of Student Staff

selection, but it is not possible due to Spring Break, preparation for Hall Closing, Room

Selection, and other aspects associated with the ending of the academic year. Since time is

crucial and of the essence, the most feasible timeline suggest that survey would be completed

within three weeks of the end of student staff selection, and focus groups will begin within two

weeks of the close of the evaluation survey. By the late May early June a rough draft of the

analysis and initial report will be used to start brainstorming for the following student staff

selection which occurs in February. Only a rough draft would be scheduled for this time because

the department has other processes coinciding with the evaluation process that must take priority,

including summer housing, capital projects, and hall opening initiatives. A final report will be

completed by Mid-November when evaluators have more time to analyze the data. The final

report will be presented and used to help inform the following year’s student staff selection for

which the planning stages begin in late November.

Budget

The Department of Residence Life will not need to allocate a significant amount of

money to the evaluation due to its access to the SPSS software, a contract with Qualtrics, and


printing services. By using the Residence Life employees as the moderators and note takers, as

well as members of the Student Staff Selection Committee as transcribers other cost are diverted

as well. The majority of the $187.42 is allocated for the focus group food as displaying in the

budget (Appendix I). Other cost to the department are minimal, needing only various office

supplies to help conduct the focus groups.

Results Presentation

Qualitative data will be presented in several ways so that data can be shared with the

stakeholders. This will assist in making sure the codes are outlined clearly for the stakeholders.

Qualitative data will be presented in several ways so that data can be shared with stakeholders.

Survey findings and data from focus groups will be consolidated and put together for the

Department Selection committee. Reports will be compiled that include tables of descriptive

statistics, as well as direct quotation from focus group participants. Bar graphs can also be used

to show the major themes, for example the frequency that each theme emerged in the focus

groups.

Next Steps

In a time of significant transition, an evaluation can give important insights on whether

the programs goals are being met and what factors of the program may be contributing to the

unmet goals. The next steps of this evaluation approach is benchmarking what similar schools

are doing, and looking into other aspects that could improve group process, marketing, and

overall goals. Due to the lack of learning outcomes and consistency of archive information, there

is no way to compare one year’s process to another. An effect in implementing this plan will

adjust aspects of the selection process to maximize efficiency and effectiveness as well making

sure the department is hiring in relation to the student population.


References

Creswell, J.W. (2009). Research design: Qualitative, quantitative, and mixed method approaches

(3rd Edition). Thousand Oaks, CA: Sage Publications.

Fitzpatrick, J., Sanders, J., & Worthen, B. (2003). Program evaluation: Alternative approaches

and practical guidelines (3rd Ed.) New York: Longman.

Fuller, W.A. (1976). Introduction to statistical inference time series .New York, John Wiley

Katie Rutkowski (personal communication, September 20,2013)

Kress, V. E. & Shoffner, M. F. (2007). Focus groups: A practical and applied research approach

for counselors. Journal of Counseling and Development, 85, 189-195.

Loyola University Chicago. (2013). Department of Residence Life. Retrieved from

http://www.luc.edu/reslife/

Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks,

CA: Sage Publications.

Residence Life. (n.d.) Retrieved on September 26th, 2013 from

http://www.luc.edu/reslife/about/mission/

Schuh, J.H. & Associates (2009). Assessment methods for student affairs. San Francisco,

CA:Jossey-Bass

Upcraft, M.L. & Schuh, J.H. (1996) Assessment in Student Affairs: A Guide for Practitioners.San

Francisco: Jossey-Bass

Wholey, J.S., Hatry, H.P., & Newcomer, K.E. (Eds.) (2010). Handbook of practical program

evaluation (Third Edition). Jossey-Bass: San Francisco.


Appendix A


Appendix B






Department of Residence Life Student Staff Selection Survey

Appendix C

This survey will be used to improve the selection process. In years past, the Department of Residence Life has used the feedback to further enhance and improve the Student Staff Selection Process. Your feedback is important to us and will remain confidential.

Marketing: These questions will gather information regarding your experience with the Student Selection Process with the Department of Residence Life marketing

1. Select the following position(s) that are a part of the Student Staff Selection Process? □ Resident Assistant (RA) □ Learning Community Assistant (LCA) □ Both(RA/LCA)

2. Have you ever seen marketing (tabling, flyers, emails, coffee sleeves) for the Student Staff

Selection? □ Yes, I have seen marketing □ No, I have not seen marketing

3. Are you familiar with the slogan "Make Loyola Home"

□ Yes □ No

4. Choose the options that best represent this logo?

□ Student Staff Selection □ Department of Residence Life □ Both


5. Have you seen this logo? □ Yes □ No

6. If yes, where have you seen this logo?

7. Please rate the effectiveness (successful in producing a desired result) of each form of

marketing. Please rate on a scale of 1 to 4 where: 1=not effective; 2=somewhat effective; 3=moderately effective; 4=very effective

A. Flyer 1 2 3 4

B. Tabling 1 2 3 4

C. Informational Sessions 1 2 3 4

D. Emails 1 2 3 4

8. On average how many flyer do you walk pass a day?

□ 0-20 flyers a day □ 21-40 flyers a day □ 41+ flyers a day

9. Rate the likelihood of you doing the following by rearranging them in the order from 1-3. (1

being more likely 3 being less likely) □ Stopping to read a flyer □ Stopping to get a free item and a flyer □ Asking about the content on the flyer after reading

10. If the department is using flyers to market, what are the best locations for marketing? (Check

all that apply) □ Residence Halls □ Dining Facilities □ Academic spaces □ Fitness areas □ Information Commons □ Student Union □ University recognized venues (Pete's, Felicia’s) □ Off campus venues (i.e. CVS, Five Guys) □ Other ____________________

11. Do you believe Selection Process flyers are overlooked?

□ Yes □ No


12. If yes, please provide 2 reasons why flyers overlooked?

13. If a department is hoping to market a position or a program what are some ways you believe is

effective for the community? Overall Satisfaction These questions will seek to understand your overall satisfaction of the Student Staff Selection process. 14. Did you partake in the entire selection process?

□ Yes □ No (If students select No, they will be redirected, and will have completed the survey)

15. Please select your overall level of satisfaction with the Student Staff Selection process

□ Very Satisfied □Satisfied □Neutral □Unsatisfied □ Very Unsatisfied

16. Please select your overall level of satisfaction with the length of Student Staff Selection □ Very Satisfied □Satisfied □Neutral □Unsatisfied □ Very Unsatisfied

17. Please indicate your level of agreement with the following statements about the selection process: Strongly Agree Agree Neutral Disagree Strongly Agree

A. The group process was a good use of my time

□

□

□

□

□

B. The group interview allowed me to express myself fully and the questions met my needs

□

□

□

□

□

Diversity These questions will indicate your level of agreement to the following statement.

18. I felt comfortable and well informed by the senior staff member when I arrived for each

interview


□ Strongly Agree □Agree □Neutral □Disagree □ Strongly Disagree

19. I felt that the interview questions gave insight to how I would be supported as a staff members □ Strongly Agree □Agree □Neutral □Disagree □ Strongly Disagree

20. I felt the questions asked by senior staff members in the interview and the group process allowed for a great understanding of the department value “DIVERSITY”

□ Strongly Agree □Agree □Neutral □Disagree □ Strongly Disagree

21. The Student Staff Selection process allowed me to express my identities and myself □ Strongly Agree □Agree □Neutral □Disagree □ Strongly Disagree

22. I felt supported when I received my letter regarding the final selection decision □ Strongly Agree □Agree □Neutral □Disagree □ Strongly Disagree

Demographic: The last set question that seeks to obtain the demographic of the student staff participants. 23. In which area do you currently serve as a Student Staff Member? (This information will remain

anonymous) □ North-side (Bellarmine, Campion, Mertz, Fordham, Santa Clara) □ West-side (Regis, Simpson, Winthrop) □ East-side (San Francisco, de Nobili, Kenmore) □ Off-campus (commuter)

24. Select your academic standing

□ Sophomore □ Junior □ Senior □ Transfer □ Graduate Student

25. Select from the following that best describes your status

□ I am a first year staff member □ I am a second year staff member □ I am a third year staff member

26. What is your gender? (How do you identify)

□ Female


□ Male □ Other

27. What is your Race/Ethnicity?

□ Asian /Pacific Islander □ Black/African American □ Latino/Hispanic □ White/Caucasian □ Other

28. Based on your knowledge of student staff positions at Loyola University Chicago, which positions are available with the department of Residence Life? (Check all that Apply)

□ Resident Assistants □ Furniture Crew □ Student Office Assistant □ Information Commons □ Desk worker □ Damen Desk Worker □ Learning Community Assistant □ Desk Receptionist/ Mangers □ Other (Please Specify) ____________________

29. Would you like to participate in focus group to share your ideas and thoughts regarding the

Student Staff Selection? □ Yes- If so please include email: @luc.edu □ No

Thank you for participating in the Student Staff Selection Process Survey! The Department of Residence Life values your feedback and will continue to improve the Student Staff Selection process for you and the department. If you have any questions, feel free to contact Crystal Norwood at [email protected] or Tyeishia Banks at [email protected]

Appendix D

Focus Group- Department of Residence Life Focus Group Purpose:

To evaluate the Student Staff selection process and the departments marketing strategies for


the process Focus Group Topics: 1. To explore the satisfaction of Student Staff

Selection marketing strategies 2. Explain in depth the ways in which the Selection process has prepared them for each component

a) Group process readiness b) Informational sessions c)

3. Identify ways in which areas of the Selection process can be improved

a) Group process interview b) Informational sessions prep c) Marketing Strategies d) Diversity in activities

Focus Group Script: Thank you all for coming today. My name is _____________and this is ________________and ______________. We are grateful for your willingness to provide feedback about the Department of Residence Life Selection process by participating in this meeting. Please feel free to grab some lunch if you haven’t done so already. We will be spending the next 60 minutes talking about your experience with the Student Staff Selection process. There is no right or wrong answers, and we’d like to hear from everyone if possible. What is shared today will be recorded by _________ and _________and a tape recorder so we can be sure fro program evaluations to help us make improvements for future participants. We will be sharing this information with the Selection committee within the Department but not with external administrators. Please look at the consent form and sign it if you agree to the terms. Are there any questions before we begin? The evaluators have three goals 1. To explore the satisfaction of Student Staff Selection marketing strategies; 2. Explore ways in which the Selection process has prepared them for each component; 3. Identify was in which areas of the Selection process can be improved. We are going to focus on these goals in our discussion today. Let’s begin by introducing ourselves. Let’s go around the table- please say your name, major, and your year in school at Loyola University Chicago. Let’s begin Introductory Question (5 Minutes)

• Let’s begin by having you all introduce yourselves. Please state your name, class standing, major, and which staff you are on. Moderator: Thank you all very much and it is nice having you here today, so let’s start with your overall impression of the marketing strategies Student Staff Selection.

10 minutes Goal 1: Marketing 1. How do you think marketing effects the process?

Probe: What is your overall satisfaction with marketing for student staff selection? Do you have for improving the marketing strategies for the process? Probe: Who responsibility do you think it is to market these position? And who should be included in this plan?


Probe: What are your thoughts regarding the marketing timeline? (Starting 8 weeks in advance)

Moderator: So what I am hearing is that many of you believe marketing should look like (list them) based on what is appealing to the students. In addition, your overall satisfaction with the marketing strategies were (summarize satisfaction) and moving forward you suggest this (list them). Moving on from what went well or what did not, we now want to focus on how the application process met your needs and prepared you. 10 minutes: Goal 2: Application Process

2. What suggestions do you have for improving the application? 3. What suggestions do you have for improving the interview process?

Probe: How informed were you after attending thee informational sessions? . Probe: Do you think the informational sessions were effective in preparing you for the selection process?

Moderator: Thank you for identifying ways for improvement if any within the application process. I also heard many people say they were more comfortable with certain aspects and seen effectiveness in certain areas (list them), while others felt differently for the following areas (list them), now we will focus on aspects and ideas that will improve Application Process. 10 minutes Goal 3: Group Process

4. How supported did you feel by the graduate assistants, professionals, and returning staff members? Probe: How was your interaction in the activities in the group process? How would you describe the environment and comfort?

Moderator: In terms of improvements, you all had the following suggestions about time [list them] and ideas for how the information should be presented, such as [list]. Finally you all head helpful tips to help improve Group Process overall with [list suggestions]. Today we were able to gather your feelings on overall satisfaction of Group Process, your preparation, and ideas for improvement, we know that there may be some lingering topics you wanted to discuss, so our final questions seeks to wrap up any last statements. 10 minutes Goal 4: Diversity

5. Do you feel there is a decrease or increase in diversity within the applicant pool? Probe: How did you feel in the interview setting? Did you feel informed by the panel? Probe: How comfortable were you with the questions being asked by the interviewer?

And why? 6. How do you see the Department of Residence life meeting the needs of these specific

populations? White/ Black/Hispanic/Pacific Islander? Probe: What do you see as challenges for staff members that identify with these identities?

Moderator: Thank you for discussing areas pertaining to one of the department’s values in regards to diversity. We heard many responses in regards to needs and comfort. We will now close with a final wrap and we appreciate you openly sharing how you feel in terms of diversity within our selection process.


5 minutes: Wrapping up Overall: 7. Is there anything else related to the Student Staff Selection process that we did not cover that you

would like to share? Thank you so much for sharing your thoughts with us today, we appreciate it! We will be compiling the results of this focus group and sending it out to everyone here today to get your feedback as to whether our summary has captured your comments accurately.

Appendix E

Note-‐ Taking Sheets for Focus Groups Name:


Date: Note-‐ Taker: While taking notes, please try to capture affect via behaviors that capture the affect

for example: tone, pitch, facial expressions and or gestures, etc. Participants Name Codes and Identifying features if needed

Questions/Prompt Notes Intro:

• Let’s begin by having you all introduce yourselves. Please state your name, class standing, major, and which staff you are on.

Goal 1: Marketing

8. How do you think marketing effects the process? Probe: What is your overall satisfaction with marketing for student staff selection? Do you have for improving the marketing strategies for the process? Probe: Who responsibility do you think it is to market these position? And who should be included in this plan? Probe: What are your thoughts regarding the marketing timeline? (Starting 8 weeks in advance)


Goal 2: Application Process 1. What suggestions do you have for

improving the application? 2. What suggestions do you have for

improving the interview process? Probe: How informed were you after attending the informational sessions? . Probe: Do you think the informational sessions were effective in preparing you for the selection process?

Goal 3: Group Process

1. How supported did you feel by the graduate assistants, professionals, and returning staff members? Probe: How was your interaction in the activities in the group process? How would you describe the environment and comfort?

Goal 4: Diversity

1. Do you feel there is a decrease or increase in diversity within the applicant pool? Probe: How did you feel in the interview setting? Did you feel informed by the panel? Probe: How comfortable were you

with the questions being asked by the

interviewer? and why?

2. How do you see the Department of Residence life meeting the needs of these specific populations? White/ Black/Hispanic/Pacific Islander? Probe: What do you see as challenges for staff members that identify with these identities?


Wrapping up Overall:

1. Is there anything else related to the Student Staff Selection process that we did not cover that you would like to share?

Appendix F


Consent Form

Project Title: Department of Residence Life Student Staff Selection Evaluation

Evaluator: (Name)

Purpose:

The Department of Residence Life at Loyola University Chicago is conducting a program assessment under the supervision of the Associate Director for Staff and Programs. You are invited to participate. The purpose of the study is to examine the Student Staff Selection Process. Specifically, we want to understand what learning outcome are achieved during the selection process/if the student staff reflections the overall student populations at Loyola University Chicago. We will use this information to improve the student staff selection process as we continue this process for coming years.

Procedures:

If you participate in this focus group, you will be in a group of approximately 8-10 students. There will be a facilitator who will ask questions and facilitate the discussion, and two note-takers to write down the ideas expressed within the group. If you volunteer to participate in this focus group, you will be asked some questions relating to your experience within your Resident Assistant or Learning Community Assistant position at Loyola University Chicago as a new or returning staff member. These questions will help us to better understand the learned outcomes of the Student Staff Selection Process.

Voluntary Participation:

Your participation is completely voluntary. You may withdraw from this study at any time without penalty.

Benefits and Risks:

Your participation may benefit you and other Loyola University Chicago students by helping to improve the selection process, experience during the process, and expected learning outcomes. No risk greater than those experienced in student staff selection process are anticipated. However, if something during the group causes discomfort, you will have received a list of campus resources where you can seek counseling.

Everyone will be asked to respect the privacy of the other group members. All participants will be asked not to disclose anything said within the context of the discussion, but it is important to understand that other people in the group with you may not keep all information private and confidential.

Confidentiality:

Anonymous data from this focus group will be analyzed by the Department of Residence Life staff and reported to Student Affairs administrators. No individual participant will be identified or linked to the results. Study records, including this consent from signed by you, may be inspected by the administrators. The results of this focus group may be presented at departmental however; your identity will not be disclosed. All information obtained in this focus group will be kept strictly confidential. All materials


will be stored in a secure location within the Department of Residence Life and access to files will be restricted to professional staff.

Contacts and Questions:

If you have any questions about the focus group you can contact (name) at (email address).

Statement of Consent:

By signing this consent form, you are indicating that you fully understand the above information and agree to participate in this focus group. You will be given a copy of this form to keep for our records.

Participant's signature: ___________________________________________

Printed name: ________________________________________

Date: _____________________________________________

Appendix G



Appendix H


Appendix I


Appendix J

Email to all RAs with Qualtrics Links to Student Staff Selection Survey Dear [Insert Name], We hope that you are having a great semester and everything is going smooth. You are receiving this email because the Department of Residence Life is constantly seeking ways to develop students holistically and making every interaction and experience transformative. Since you currently represent the Department this process begins with you. One particular aspect that we are seeking to improve and get feedback is the Student Staff Selection that not too long wrapped up. In the past the Department has tried to get honest feedback. The data collected has allowed the Department to make changes in modifying length and structure of the group process day. . This year we are providing you with a new survey, which is shorter and targeted at more than just satisfaction with Student Staff Selection. We want your input Marketing Strategies and how we can increase diversity amongst the student staff. At the end of the survey, RAs will also seek the opportunity to take part in a focus group, which will help the Department of Residence Life truly understand the selection process from your perspective. The focus group will last no more than one hour and lunch from Jimmy Jones will be provided for those participating. The link to the survey is attached below. All feedback you provide will remain anonymous and will not be tied to your personal information. We ask that you be honest and constructive with your feedback. LINK Again, we truly value your input and feedback for Student Staff Selection. Should you have any questions feel to email Crystal Norwood at [email protected] or Tyeishia Banks at [email protected] Best, Crystal Norwood and Tyeishia Banks Email Reminder1: Subject: Reminder: Student Staff Selection! Dear [Insert Name], We hope that you are going well for you and your semester is going great.


Our records indicate that you have not completed the Student Staff Selection Survey. As a department, we truly value the input and feedback that you offer with your recent experience of Student Staff Selection. This is your chance to let us know what you liked and did not like about training, as well as how we can better tailor the information so it does not feel overwhelming. Please take some time to complete the survey so that we can continue to improve the experience here at Loyola University Chicago especially within the Department of Residence Life. At the end of the survey, you will be able to take part in a focus group, where you can voice your opinions more in depth. The one hour focus group will happen on a later date with food provided from Jimmy Jones. Below is the personalized link that you have received previously. Link If you feel like you received this email in error, please contact Crystal Norwood at [email protected] or Tyeishia Banks at [email protected] Again, we truly value your input and feedback with Student Staff Selection. Best, Crystal Norwood and Tyeishia Banks Email Reminder # 2 Subject: Final Reminder: RA Fall Training Survey! Dear [Insert Name], Our record indicates that you have still not completed the Student Staff Selection survey. We would really like to stress the importance of the survey as your chance to have a large impact on future Student Staff Selection processes. This is your last chance to take part in our evaluation of Student Staff Selection. Below is the personalized link that you have received previously. Link If you feel like you received this email in error, please contact Crystal Norwood at [email protected] or Tyeishia Banks at [email protected] Again, we truly value your input and feedback for Student Staff Selection Best, Crystal Norwood and Tyeishia Banks


Appendix K Focus Group Email Template 1: Email Templates for Focus Group Subject: INVITATION TO PARTICIPATE IN RESIDENCE LIFE FOCUS GROUP Hello from the Department of Residence Life The department of Residence Life invites you to participate in a lunch dialogue focus group as part of an assessment of the Department of Residence Life Student Staff Selection Process. We are constantly working to improve the selection process as well as the student experience who participate in the process. The purpose of the focus group will be to address the following areas:

• To explore the satisfaction of Student Staff Selection marketing strategies. • Explain in depth the ways in which the Selection process has prepared them for each

component. • Identify was in which areas of the Selection process can be improved.

The focus group will take place in the Regis Hall Conference Room. Lunch will be provided in appreciation of your participation and the focus group will be no more than 60 minutes. Please respond to this email if you would like to participate in a focus group with your name and available dates and times by (date) from the following list: (dates) between 11:00am-12:00pm, 12:00pm-1:00pm, and 12:30pm-1:30pm. Once we receive your list of available dates and times, you will receive a final confirmation email from us with one of the dates and times you have been selected for. Please confirm your participation by (date). If you have any questions, please do not hesitate to contact (name graduate assistant) at (phone number) or (email address). We look forward to hearing your valued thoughts and opinions on your experience within the Residence Life Student Staff Selection Process. Have a great day, (Signature) Focus Group Email Template 2: Focus Group Second Invitation Hello from the Department of Residence Life When you completed the survey for the Department of Residence Life, you told us you might be interested in doing a focus group with our office. This is our second attempt to be in touch with you about participating---we hope you’re still interested in joining us! The Department of Residence Life Student Staff Selection Process. We are constantly working to improve the selection process as well as the student experience who participate in the process. The purpose of the focus group will be to address the following areas:

• To explore the satisfaction of Student Staff Selection marketing strategies. • Explain in depth the ways in which the Selection process has prepared them for each

component.


• Identify was in which areas of the Selection process can be improved. The focus group will take place in the Regis Hall Conference Room. Lunch will be provided in appreciation of your participation and the focus group will be no more than 60 minutes. Please respond to this email if you would like to participate in a focus group with your name and available dates and times by (date) from the following list: (dates) between 11:00am-12:00pm, 12:00pm-1:00pm, and 12:30pm-1:30pm. Once we receive your list of available dates and times, you will receive a final confirmation email from us with one of the dates and times you have been selected for. Please confirm your participation by (date). If you have any questions, please do not hesitate to contact (name graduate assistant) at (phone number) or (email address). We look forward to hearing your valued thoughts and opinions on your experience within the Residence Life Student Staff Selection Process. Have a great day, (Signature) Focus Group Email Template 3: Focus Group Acceptance Email: Dear (Student Name), Thank you for agreeing to participate in this interview, this email serves as a confirmation that the focus group you will be attending will be held on (date) at (time) . The focus group will take place in the Regis Hall Conference Room. The focus group will be no longer than 60 minutes. Please let us know if there are any accommodations you may need to participate in this focus group. As a reminder you will need to complete a consent form upon participating. If you have any questions feel free to contact me at (email) or (phone number). We look forward to the focus group. Thank you, (Signature) Focus Group Email Template 4: Focus Group Reminder for those who signed up Dear [insert name], You have signed up to participate in our focus group on [insert date] at [insert time] in [insert location]. Please arrive and we will orient you, invite you to eat, and jump in to the conversation! If your availability has changed for the focus group, please let us know by replying to this email so we can adjust our planning; we request at least 24 hours in advance if possible. Thank you—we will see you soon! Sincerely, (Signature)


Focus Group Email Template 5: Focus group Thank you THANK YOU! Your willingness to spend the time in conversation with us is so important to being able to build on and improve the Student Staff Selection process at Loyola. Without your dedication, we would not be able to keep what you value and modify what you suggest we change. Thanks for your commitment to our residential population. Have a good rest of your semester! If you have any questions, please feel free to be in touch with (Name) at (Email)


Appendix L














Documents

Student Staff Selection Eval