EdWeb Analysis and Design Document - Weeblygregmason.weebly.com/uploads/2/7/7/3/2773728/edweb... · 2020. 1. 28. · assessment scores for Near Hire applicants and increase the number

Greg Mason

August 2, 2009

Executive Summary

Teletech is a global corporation that specializes in business-process outsourcing. Our company partners with businesses in vectors ranging from telecommunications to healthcare to provide customer service operations. Since many of the customer management centers in which these operations take place are all across the world and most of the customers they serve are English speaking, it is necessary to ensure that our customer care associates are satisfactory in English language communication.

When someone applies to Teletech, they must pass a language proficiency interview with a score of at least 2.1 out of 3.0. If the applicant fails within a few points of this score, they are referred to as "Near Hires". The goal of this project is to improve English language assessment scores for Near Hire applicants and increase the number of New Hires by at least 20% of the number of Near Hires. The educational website will be external from Teletech's network and will provide online instruction for Near Hire applicants to improve their English-language skills to pass the interview on the second attempt.

Participants range in age and education. However, since a high school diploma is required to be considered for employment, learners are expected to have at least this level of education. It is also expect for learners to have the minimal computer skills necessary to take this course. It is important to note that the learners will come from a variety of cultural backgrounds and therefore extensive research is being done into delivering eLearning to this international audience.

There are three parts of the MatchPoint assessment interview: Listen and Repeat, Listening Comprehension and Conversational English. There will be four lessons total in this course. All lessons will address the parts of the MatchPoint assessment interview. The first part of this course will focus on listening comprehension. The two lessons within this module are listening for the main idea, listening for details. The second part of the course will focus on speaking skills. The two lessons for this module will include: intonation and word and syllable stress

EdWeb Analysis and Design Document Global English Connections: Helping you reach your goal

This EdWeb project has already been approved by the Director of Language Readiness, Tim Weber, who has not only encouraged the idea, but also contributed constructive feedback to the analysis of the project. Although I will be the primary instructional designer and developer on this project, I will have access to SMEs who will be able to provide their expertise in language learning as well as eLearning design and development if necessary.

The courses design is constructivist at heart because it includes a great deal of learner-centered features. In addition to the two lesson modules as mentioned earlier, the website will include job aids addressing cultural issues and grammar topics. There will also be a social feature on the homepage that will allow learners to create discussion threads and network about the learning topics.

Analysis

1. What is the instructional need or opportunity? What business or learning problem needs to be addressed?

Teletech is a global corporation that specializes in business-process outsourcing. Our company partners with businesses in vectors ranging from telecommunications to healthcare to provide customer service operations. Since many of the customer management centers in which these operations take place are all across the world and most of the customers they serve are English speaking, it is necessary to ensure that our customer care associates are satisfactory in English language communication.

A recruiting process called HirePoint contains an assessment designed to ensure that all new hires demonstrate this proficiency in English. The assessment consists of three primary parts: and is scored on a three point scale. The minimum score to be considered for employment with Teletech is 2.1. Applicants who score 1.8-2.0 put into a category called "Near Hires" and are encouraged to take the assessment again after additional English language training. Teletech currently has a Near Hire eLearning English tutorial, but it is considered to be ineffective. There is a demand for an external educational website that reaches out to the local community.

Why is it important to address this instructional need or opportunity? What will happen if you do not address it?

By up-training these applicants and making them employable, Teletech will increase its labor pool and saves on recruiting costs. Attempts to develop applicant skills also helps build Teletech's reputation in the community.

Assuming this instruction is online (i.e., hosted on a website), what hardware and software (including access speeds and browsers) do potential learners have?

Hardware and software varies according to the geographic location of the target audience. Since call centers are primarily located in urban areas, internet access is generally available.

It's possible that internet speeds are slow in certain areas. To participate in this course, participants must have a PC with at least 500 MHz processor speed and Windows 2000, XP, or Vista. Macs with comparable specs are acceptable but no expected with learners in the target countries. Computers should have at least 256 MB of RAM and speakers or plug-in headphones. Learners require internet speeds of at least 120 kbps.

2. Why did you decide to deliver this instruction online? What are the benefits and potential drawbacks of that decision?

Because our global associates have limited access to authentic EFL instruction in their countries, online instruction is a cost-effective way to connect them with instruction from native English speakers. Near Hire applicants have the opportunity to learn at their own pace. This educational website will require learners to register to take the eLearning courses. This registration component will be a valuable asset to Teletech since it will provide demographic information as well as a gauge of our applicants' level of motivation.

Online language learning does have its drawbacks. It doesn't, for example, provide learners with personal interaction and can't be used alone for effective conversational language acquisition. We will try to compensate for this drawback by providing video simulation. Online learning is not setup in this case to capture the entire audience base, but to capture as many as Near Hire applicants as possible. Because an alternate in-class, instructor-led course is not a feasible option at this point, online instruction is simply the best resource we can provide the target audience.

3. Will this be a self-paced, online course or a group-paced, online course? For example, IT5660 is a group paced, online course. What is your rationale for that decision?

This will be set up as a self-paced online course but will have the capability to be embedded into a larger group-paced course. By making the course self-paced, stakeholders are given the flexibility to adapt it to their needs.

4. Who are the stakeholders? For example, in a K-12 setting, the stakeholders are probably students, teachers, parents, and perhaps the school board and/or the community. In a corporate setting the stakeholders are probably learners, managers, perhaps HR, and the CEO or board of directors.

Stakeholders are the managers in the Learning and Performance department, Teletech clients, and the Near Hire applicants.

5. What other instructional context issues, challenges or problems are important in this situation?

Language is a very important aspect to consider in the development of this course. For this reason, content will need to be carefully selected to accommodate the learners' language comprehension. It is also important to consider cross-cultural values when delivering online instruction to an international audience. Some cultures respond better to online delivery than others. For example, because some cultures are very communal learners, it may be a challenge for them to learn independently.

1. What does each group of stakeholders need to see to consider this instruction successful? These “results” might be standards set by stakeholders, such as the State or your school district, competencies set by a professional organization or licensing agency, etc. In a corporate setting, it might be an increase in sales or a decrease in error reports. Use the table below to answer this question.

2. What indicators will you use to determine if the instruction meets the desired outcomes of each group of stakeholders? For example, if one of the outcomes a group of stakeholders needs to see is an increase in sales, how will you measure that increase? Do you need to collect baseline data in order to calculate the increase in sales? Use the table below to answer this question.

Stakeholder group

Results stakeholders need to see to believe the EdWeb is successful

How you will collect data and measure changes

Learning and Performance Managers

Increase in new hire associates of at least 25%

Baseline data collected from the number of new-hire associates in the labor poor prior to the start of the EdWeb course. This baseline will be compared with the total number of new-hire associates one month after completion of the course.

Teletech clients Increase in new hire associates of at least 25%

Baseline data collected from the number of new-hire associates in the labor poor prior to the start of the EdWeb course. This baseline will be compared with the total number of new-hire associates one month after completion of the course

Associates Increase in MatchPoint assessment score of at least 0.2 points

Baseline data will be taken from the failed assessment scores of all participants registered in the EdWeb course. This baseline will be compared with the participants' assessment scores from the second MatchPoint assessment.

3. What other outcomes do you need to measure? N/A

1. What are the demographics of the learners? a. Age range, e.g., 10 – 11 years old

Age of learners primarily range from 18 -45 years old.

b. Education levels and/or types of college degrees. Since the target audience mostly deals with trouble shooting technical problems over the

phone, the education ranges from high school diploma to advanced college degrees.

c. Other important demographics, e.g., gender, race, ethnicity, languages, disabilities? This course will be delivered to a very diverse international audience.

2. What prior experiences do potential learners have with this content or related areas? For example, perhaps your students have no prior experiences with this content, which is a specific piece of equipment, but they have a lot of prior experience with a piece of equipment that is being replaced, e.g., the content is panoramic photography and the students are photographers who use digital SLRs or the content area is how to use a smart board and the students have extensive experience using traditional whiteboards.

Learners are expected to have a high-intermediate level of English of a Foreign Language. This corresponds with the MatchPoint assessment range of 1.8-2.1. Since potential learners are applying for customer service positions, which require basic computer literacy skills, it is expected that they will possess the minimal computer literacy skills necessary for taking this course.

3. What are the technology skills of the learners? a. Approximately what percentage of learners have taken online instruction before? We won't really know this until the applicant applies for the company. As mentioned

earlier, computer literacy is a necessary qualification for the position and the applicants will be required to take more online instruction upon hire. Therefore, it may be the applicant's first exposure to online learning, but if the applicant is hired, it won't be the applicant's last.

b. Do the learners have the needed technology skills to be successful in this course? As mentioned earlier, computer literacy is a necessary qualification for the position and the

applicants will be required to take more online instruction upon hire. Therefore, it may be

the applicant's first exposure to online learning, but if the applicant is hired, it won't be the applicant's last.

4. What are the learning styles and skills of the learners? a. Generally, what are the learning styles of your learners? If you have no idea,

how/where can you find that information? I will research the audience by having new hire associates complete a Best Learning

Experience writing assignment. This assignment will be distributed to associates in Mexico, Argentina, South Africa, and the Philippines. I will then assess the common instructional values from the stories and apply these values to the course. Although this Common Instructional Values assessment doesn't focus on learning values on an individual scale, it will provide useful pedagogical insights on a societal scale.

b. Do they have self-directed learning skills? The CIV assessment mentioned above will determine this question. c. What other learning styles or skills issues are important to consider for the design

of this instruction? It's important to consider those of our audience with an interpersonal learning intelligence.

Although this is a very valuable and sought after intelligence for our company, interpersonal learners may find the "intrapersonal" design of the course to be somewhat challenging. To compensate for this, I plan to include a social networking feature on the website.

5. What are the reading skills/level of the learners? Generally, American corporations assume a 10th grade reading level.

Since the target audience reads English as a Foreign Language, it's necessary to assess their reading comprehension according to TESOL (Teaching English to Speakers of Other Language) standards of assessment. Most learners in this program will have a reading comprehension of at least High-Intermediate.

6. What are the expectations and/or assumptions your learners have about instruction and/or this topic? a. Do they have a positive attitude toward instruction? The learners generally have a positive attitude toward instruction pertaining to the

company because it is connected to their success in their jobs. b. Do they have a positive attitude toward this topic?

The learners have many extrinsic motivations for improving their English. Attitude in language learning, however, is always a very fragile thing, so it will be very important to provide as much constructive feedback and encouragement in the course.

c. What do they want or expect from instruction?

Learners expect improvement in English communication.

7. What are the incentives and disincentives for learners to successfully complete this instruction? a. Will learners take this instruction in their free time or during their regular work

hours? Learners will take this instruction in their free time. b. Do learners need the instructional information? Do they get rewarded for using it? As mentioned earlier, learners have major extrinsic motivations to succeed in this course.

The most important short-term reward is getting hired onto the company. Long-term rewards include bonuses or career advancement within the company.

8. What other learner needs or characteristics are important and/or relevant to this project? Cultural sensitivity is probably the most important characteristic to consider in this project.

1. What is the scope of this instructional product, i.e., is it a lesson, a group of lessons, a full semester course, a multi-course curriculum, etc.? If it is more than one lesson, list the topics you plan to cover.

There are three parts of the MatchPoint assessment interview: Listen and Repeat, Listening Comprehension and Conversational English. There will be four lessons total in this course. The first part of this course will focus on listening comprehension. The two lessons within this module are listening for the main idea, listening for details. The Listening for Details topic will address the "Listen and Repeat" part of the assessment and the Listening for the Main Idea will address the "Listening Comprehension" part of the assessment.

The second part of the course will focus on speaking skills. The two lessons for this module will include: intonation and word and syllable stress. These lessons address the "Conversational English" part of the assessment. .

Active Listening

In this course, participants learn the active listening strategies that are essential to effectively communicating with a customer.

Main Points and Important Details

In this course, participants learn how to identify the main points and important details of a customer‟s query.

Word Stress and Intonation

In this course, participants learn how stressing the correct words and using the correct intonation in conversation can help to make you sound clearer and more natural.

Pronunciation Challenges

In this course, participants learn how to correctly pronounce words containing difficult sounds by distinguishing the sounds in minimal pairs.

Asking for Information

In this course, participants learn how to use the correct grammar structures commonly used when asking a customer for information.

Giving Information

In this course, participants learn how to use the correct grammar structures commonly used when give a customer information.

2. What are the formal instructional objectives for the instruction? Formal instructional objectives contain three specific elements: a) condition b) one (and only one) action verb (behavior) c) passing criteria and/or standard and/or degree

Objective

#

Condition

Behavior

Degree

1 Given five different four minute call recordings,

Identify the main idea(s) in each call.

To receive full credit for this objective, students must correctly identify 80% of each call's main idea.

2 Given five different four minute call recordings,

Recognize the important details of each call…

To receive full credit for this objective, students must correctly recognize 80% of each call's details.

3 Given ten oral responses of a customer service agent,

Analyze the stress patterns of each sentence.

To receive full credit for this objective, students must correctly identify 80% of the stress patterns in the ten sentences.

4 Given ten written responses of a customer service agent,

Apply the correct intonation of each sentence.

To receive full credit for this objective, students must correctly apply 80% of the intonations to the ten sentences.

5 Given ten customer scenarios…

Apply the correct active listening strategy…

To receive full credit for this objective, students must correctly apply the strategies to 80% of the scenarios.

6 Given a three sentence paragraph

Recognize the incorrect pronunciation.

To receive full credit for this objective, students must correctly identify 80% of the mistakes in the paragraph.

7 Given ten sentences of an agent‟s call

Identify the correct grammar structures.

To receive full credit for this objective, students must apply the correct grammar to 80% of the ten sentences.

3. Complete the following table. Refer to Module 3 of the Assessment and Instructional Alignment course (identified above). Identify where each objective fits in this table by putting the number of the objective in the appropriate cell. For example, objective #1 may be a “Apply Procedure” objective so put the number 1 in the Apply Procedure cell (see #1 in the appropriate cell below).

Knowledge

Cognitive Process Domain

dimension

Remember

Understand

Apply

Analyze

Evaluate

Create

Fact

1,2

Concept

5, 7 3,4, 6

Procedure

Metacognition

4. Write the mastery test question(s) for each objective, using the table below.

Objective

Mastery test question(s)

Given five different four minute call recordings students will be able to identify the main idea(s) in each call. To receive full credit for this objective, students must correctly identify 80% of each call's main idea.

1. What was the specific cause of the connection issue? a) The ADSL codes were incorrect. b) The ADSL codes had been changed. c) There weren't any ADSL codes on the line. d) There shouldn't be ADSL codes on the line. 2. Why did the CSR have to call her supervisor? a) The CSR didn't know how to resolve the problem. b) The supervisor needed to change the ADSL codes. c) The supervisor needed to put new ADSL codes on the line. d) The supervisor had access to the necessary information about the ADSL line.

Given five different four minute call recordings, students will be able to recognize the important details of each call. To receive full credit for this objective, students must correctly recognize 80% of each call's details.

1. What was the customer's password? a) kqbb8jdh b) kqdd8jbh c) kqbd8jbh d) kqbd8jdh 2. What was the customer's email address? a) [email protected]

b) [email protected] c) [email protected]

Given ten oral responses of a customer service agent, students will be able to analyze the stress patterns of each sentence. To receive full credit for this objective, students must correctly identify 80% of the stress patterns in the ten sentences.

Which of the following words in the agent's response are stressed? Select all options that apply. 1. I know what's causing your (a) (b) (c) (d) (e) problem. (f) 2. How can I help you today? (a) (b) (c) (d) (e) (f)

Given ten written responses of a customer service agent, students will be able to apply the correct intonation of each sentence. To receive full credit for this objective, students must correctly apply 80% of the intonations to the ten sentences.

Which diagram fits the agent's intonation? 1. Do you have cable or ADSL? ↓ ↑ a) [Do you have cable] [or ADSL]? ↑ ↓ b) [Do you have cable] [or ADSL]? ↓ ↑ c)[ Do you have] [cable or ADSL]? ↑ ↓ d) [Do you have] [cable or ADSL]? 2. I have reset your password. Is there anything else I can help you with? ↑ ↓ a) [I have reset your password.] [ Is there anything else I can help you with?] ↓ ↑ b)[ I have reset your password. ] [ Is there anything else I can help you with?] ↓ ↓

c) [I have reset your password.] [Is there anything else I can help you with?] ↑ ↑ d)[ I have reset your password.] [Is there anything else I can help you with?

Given ten customer scenarios, apply the correct active listening strategy. To receive full credit for this objective, students must correctly apply the strategies to 80% of the scenarios.

Read the customer text below and choose the response that would demonstrate active listening. ““I‟m getting married, so I need to make a reservation for next Friday.”

a) That‟s great! How long have you known each other? Where did you two meet?

b) I‟m not sure if you can get a discount, but I‟ll check for you.

c) I don‟t know what‟s available for that date. Let me check to see what we have.

d) That‟s wonderful news! Congratulations! Let me see what we have available.

Given a three sentence paragraph, recognize the incorrect pronunciation. To receive full credit for this objective, students must correctly identify 80% of the mistakes in the paragraph.

Listen to the agent‟s speech and select the three words that were mispronounced. “Ok sir I have you booked for Thursday, November Third. You are getting the government discount. The hotel has room service, a fitness centre and business centre.”

a) booked b) business c) discount d) fitness e) government f) November g) reservation h) Thursday

Given ten sentences of an agent‟s call, identify the Complete the sentence with the correct grammar structure.

correct grammar structures. To receive full credit for this objective, students must apply the correct grammar to 80% of the ten sentences.

___________ already _______rebooting your computer?

a) Did you already try b) Are you already trying c) Have you already tried d) Were you already trying

5. What learning resources (databases, tools, documentation, articles, books, job aids,

people, etc.) will learners need during this instruction? How will you obtain these learning resources? Learners will receive an electronic job aid for American-English pronunciation. The website will also include grammar job aids and exercises to supplement the primary objectives.

6. What other instructional content issues are important or relevant to this project?

Possible additional topics include: active listening for listening and vowel phonetics for speaking. Because I'm not confident in how I would assess these topics, I am leaving them off the course curriculum. However, they may be added in the future. Grammar topics will also be included as a supplement to the lessons. Because these topics are aimed at helping the learner succeed at the listening and speaking objectives, they will not include assessments.

1. For the purposes of the Certificate Program, your EdWeb should be ready to implement at the end of the third course, IT5680. During the second course (IT5670), you will conduct a formative evaluation on a prototype of your EdWeb. Do those milestones fit with the needs and/or expectations of the organization sponsoring your EdWeb? If not, describe how you plan to handle the problem.

This time frame works well for my organization because we are currently in the process of revamping our language readiness curriculum. By the time, my course is ready to implement, there's a good chance our department will be in favor of adopting it into the standard training curriculum.

2. Are you the content expert? a. If no, is a content expert available to help you develop the content and review it for

accuracy? Do you anticipate any problems in working with this expert, e.g., time available, schedules, time zone differences?

b. If yes, you may still want a “second set of eyes” to review your content. Who will be available to provide this support?

I have a few colleagues who would be available to serve as SMEs. One is my counterpart

in Mexico and the other is my instructional design mentor at my office.

3. What other types of experts do you need for this project? For example, do you need to work with the IT department when you are ready to implement your EdWeb?

Although I hope to be able to develop the entire program on my own, my design will have to pass through media quality control for it to become a part of our company's training curriculum.

4. When do you plan to discuss this project with each expert? Use the following table if there are more than two experts with whom you need to coordinate.

Name/Title of Expert Approximate Date for Discussion With This Expert

Amy Harry (Instructional Design Manager)

8/2009

Brent Barkley (Media lead) 9/2009

5. Who has to approve this project? For example, does someone in your school need to

approve your EdWeb design before you develop it? Does someone in HR need to approve it? When do you need to get their approval?

I have discussed this project with Tim Weber, the Director of Language Readiness, and he has given me his full support.

6. Organizational change issues: a. Are you breaking new ground in your organization with your EdWeb? If yes, who

may be threatened or intimidated by it? An external educational website for potential Teletech employees has not been done yet.

Because the CEO of the company is looking for new ideas that include some type of engagement with the community, the director and I are both very optimistic about the project's success.

b. Who directly supervises the learners who will take your EdWeb? Does this person

support and/or value the instruction and/or your approach to the instruction? The learners are not subject to any formal supervision. c. Are there people who want this project to fail? If yes, list their titles and why you

think they want this project to fail. There aren't any people I am aware of who want this project to fail. d. How and when will you inform each group of stakeholders about the project? My director and I will inform the stakeholders of my project in June, 2009 through a

MeetingPlace conference call.

7. What other resources or project management issues are important or relevant to this project?

I don't see any other issues right now.

Design (part 1)

8. What is your theory, model or definition of learning? For example, some say learning is the process of personalizing new information. Others say learning is the process of memorizing new information. What is your definition?

This project takes a constructive approach by providing learners with real-world video and audio models to emulate. Language learning is very experiential in nature, requiring that learners are constantly immersed in the language. This course allows for language immersion because it will be completely structured in English. Therefore, taking the course itself will be a language-learning experience for the participants. Learners are provided with a variety tools to reach the language goals at their own pace. This puts the responsibility on learners to create their own learning experience throughout the course. The social tool of the website also fosters a learning environment where learners explore the topics together.

9. Other than the CIVs and Horton’s Absorb, Do, Connect model, what, if any, other instructional design models do you plan to use, i.e., Kolb’s Model of Experiential Learning or Problem-Based Learning? See http://carbon.cudenver.edu/~mryder/itc_data/idmodels.html. Note: You are only required to use the CIVs and Horton.

The CIVs and Horton's Absorb, Do, Connect model will provide a sufficient framework for this course. Since the CIVs are primarily rooted in constructivist thinking, the course will essentially take a constructivist approach.

10. What other instructional design issues are important or relevant to this project?

As mentioned in the analysis, cultural values most be considered in the design of this course. These values determine how well the participants will respond to the instruction. Andrea Edmundson's research on Globalized E-learning Cultural Challenges will help in this challenge.

11. How do you intend to integrate the “Learner Centered” Common Instructional Value into your EdWeb? Provide at least three specific examples.

The objectives in this EdWeb are directly connected to the learners' extrinsic motivations, that is, obtaining a position within the company. Learners have a vested interest in this course. The first tool that learners have to reach their goal is the collection of listening models. This will help them to adapt to the content that they will be exposed to on the job. Another tool that they will have is the module containing pronunciation tutorials. These pronunciation tutorials will help learners clarify their speech for better communication with native-English speakers. Finally, the additional job aids will provide learners with grammar and cultural tools to help them communicate better with the customers.

12. How do you intend to integrate the “Social” Common Instructional Value into your EdWeb? Provide specific examples.

This EdWeb will contain a social-networking wiki that will allow learners to communicate their questions or concerns about the learning material to other learners. Learners will be able to create their own discussion threads about language learning topics.

13. How do you intend to integrate the “Contextual” Common Instructional Value into your EdWeb? Provide at least three specific examples.

http://carbon.cudenver.edu/~mryder/itc_data/idmodels.html

All the material used for this EdWeb will be framed in a customer-service context. The listening materials will taken based on conversations between customer service representatives and customers. The pronunciation materials will be based on audio content taken from customer service representative. Finally, a job aid on culture will be provided to help Near Hire applicants better understand the customer base.

14. How do you intend to integrate the “Active” Common Instructional Value into your EdWeb? Provide at least three specific examples.

This EdWeb will contain the "Active" CIV first by requiring learners to analyze and respond to situations in the listening modules. Second the learners will use the pronunciation tutorials to actively practice speech patterns. Third, the learners will be encouraged to explore the additional language-learning resources that will be provided.

15. How do you intend to integrate the “Supportive” Common Instructional Value into your EdWeb? Provide at least three specific examples.

This EdWeb will first demonstrate the "Supportive" CIV through encouraging content. For example, throughout the course, I will provide appropriate language-learning tips to provide additional assistance to learners. There might be a note in one of the listening exercises that tells learners how to be better active listeners. Another example will be the job aids provided. Finally, the discussion threads will give learners the chance to ask for help.

16. Complete the following chart to describe the activities you plan to include for each objective. In the first column, enter your objective. For each objective you may have one, two, three or more activities. Briefly describe those activities in the second column, entitled Activities. Put a checkmark in the Absorb, Do, or Connect column for each activity. That is, each activity is an Absorb activity, or a Do activity or a Connect activity. Reference: Horton chapters 2, 3, and 4.

Objective Activities Absorb √ Do √ Connect √

Given five different four minute call recordings students will be able to identify the main idea(s) in each call. To receive full credit for this objective, students must correctly identify 80% of each call's main idea.

50 practice multiple-choice questions. For each of the five calls that are played, there are 10 questions. Each call focuses on a unique customer service theme. Learners answer correctly by identifying the main ideas of the call. Feedback is given at the end of each call exercise.

√

Given five different four minute call recordings, students will be able to recognize the important details of each call. To receive full credit for this objective, students must correctly recognize 80% of each call's details.

10 multiple-choice questions to complete a call script. There are five call scripts available for the learner to complete bringing the total number of questions to 50. Learners listen to a call and then complete sections of a call script by clicking on a drop-down field in the missing text. The text appears in the script under the learner's response to provide feedback.

√

Given ten oral responses of a customer service agent, students will be able to analyze the stress patterns of each sentence. To receive full credit for this objective, students must correctly identify 80% of the stress patterns in the ten sentences.

From the five calls selected for the listening activities, clips will be taken to focus on the stress patterns. Clips will be responses from customer service representatives. There will be 10 practice sentences from each call. Since there are five calls, there will be a total of 50 practice sentences for the entire module. For each practice sentence, the learner must select the stressed words in the sentence. The correct words will appear highlighted after the exercise. The learner will

√

listen to the sentence again and repeat out loud while focusing on the stressed words

Given ten customer scenarios, apply the correct active listening strategy. To receive full credit for this objective, students must correctly apply the strategies to 80% of the scenarios.

Learners will have the opportunity to apply the active learning strategies they have learned to customer interaction scenarios. In this activity, the learner analyzes the customer‟s speech and chooses the correct response from a list.

√

Given a three sentence paragraph, recognize the incorrect pronunciation. To receive full credit for this objective, students must correctly identify 80% of the mistakes in the paragraph.

After compare the featured sounds in the minimal pair activities, the leaner will listen to a text read by an agent and identify the pronunciation errors.

√

Given ten sentences of an agent‟s call, identify the correct grammar structures. To receive full credit for this objective, students must apply the correct grammar to 80% of the ten sentences.

The learner chooses the correct grammar structure to complete the agent‟s sentence.

√

Given ten written responses of a customer service agent, students will be able to apply the correct intonation of each sentence. To receive full credit for this objective, students must correctly apply 80% of the intonations to the ten sentences.

From the five calls selected for the listening activities, clips will be taken from each call to focus on the intonation. Clips will be responses from customer service representatives. There will be 10 practice sentences from each call. Since there are five calls, there will be a total of 50 practice sentences for the entire module. Each practice sentence will be divided up into parts. For each part of the practice sentence, the learner must choose an

√

up or down arrow to indicate the correct intonation. Once the learner has submitted the answer, green or red check marks will appear next to the arrows to indicate the correct answer. The learner will listen to the sentence again and repeat out loud while focusing on intonation.

This formative evaluation was conducted by email. Six participants in the Philippines were chosen to provide a cultural perspective of the dominant target audience. While no Filipino call center associates were available to participant in this evaluation, six Filipinos were available from the language readiness team at various sites in the Philippines. The participants include:

Two Language Quality specialists from the Cainta site. (one male, one female).

Two Language Readiness trainers from the Bacolod site. (two males).

Two Language Readiness coaches from the Bacolod site (two females). Feedback from these participants is very helpful not only because they represent the culture of the target audience, but also because they have a language background and can therefore act as subject-matter experts.

Questions Asked Summary of Data Changes to be Made

1. The lesson topics

provided for listening

comprehension are useful

for improving my

conversational English.

Suggestions?

Strongly Disagree (1)-Strongly

Because listening comprehension is the only module I have designed so far, this question only applies to the lesson topics of that module. Two participants responded with a 4 and four responded with a 5, resulting in a mean of 4.7. One participant suggested including a vocabulary bank as

I have considered providing a vocabulary bank since the beginning of my analysis. I need to develop the majority of the listening scripts before I put together some sort of vocabulary bank.

Agree (5) a job aid for the listening module.

2. The website‟s layout was

inviting and made me

want to learn more about

the topics.

Suggestions?

Strongly Disagree (1)-

Strongly Agree (5)

One participant responded with a 3 two responded with a 4 and three responded with a 5 resulting in a mean of 4.3. The participant with the lowest score suggested including more visible resources on each page. Another participant suggested provide narration for the text throughout the website.

I plan to narrate the lesson content of this website.

3. The website‟s color

scheme was appealing to

me.

Suggestions?


Strongly Agree (5)

Three participants responded with a 4 and three responded with a 5 resulting in a mean of 4.5 One participant suggested using darker colors for the background with white text on top.

Given the positive responses from most of the evaluators regarding the color scheme, I can‟t really justify changing the background based on one comment. I‟ll continue to ask for feedback on that issue and change it is the comments persist.

4. The website‟s fonts were

easy to read.

Suggestions?


Strongly Agree (5)

Two participants responded with a 4 and four responded with a 5 resulting in a mean of 4.7 The participant, who suggested changing the background color, suggested that the fonts might be easier to read if they were white on a dark background. Another participant

I‟m going to include more headers and different fonts to provide contrasts throughout the lessons.

recommended using more different fonts to make the page look more interesting.

5. The website‟s overall

layout was well organized

Suggestions?


Strongly Agree (5).

Three participants responded with a 3 and three responded with a 4 resulting in a mean of 3.5. Many of the participants admitted that they would need to see more of the website to judge on its organization.

I need to develop more of the website.

6. The most interesting topic

listed on this website is:

a. Conversational

Style

b. Listening

Comprehension

c. Conversational

Structure

One participant responded with a (c) one responded with a (a) and four responded with (b). It‟s possible that listening comprehension was chosen as most interesting because it was the module that was actually presented in this evaluation. However, listening comprehension has been a favored topic with many of our learners in the past.

I think the development of the listening comprehension module is going to take first priority in this project, mostly because he listening scripts and recorded dialogues will take up a lot of bandwidth.

Because of the participants‟ cultural and occupational backgrounds, the feedback for this evaluation was credible and helpful. Much of the feedback that I‟ve gotten on and off the record has been consistent with my research of language issues at TeleTech. Most of the ideas that have emerged from this process have pertained to auxiliary components of this project such as job aids or narration. Although I have decided to implement these ideas, they will most likely be developed in the beta stage of the project. Moreover, the analysis I‟ve done in the last few months cultural psychology has led me to some very interesting insights that I‟d like to include in my EdWeb in the future. When the three modules are developed, I‟d like to start working on a supplemental cross-cultural awareness module. This module would contain lessons focusing on the cultural dimensions that impact communication. My supervisors are very enthusiastic about including more cross-cultural communication content into our curricula. The questions of this evaluation focused mostly on the visual design and instructional concepts. By the time my functional prototype is ready to present I‟ll be able to get more in-depth with my questions as there will be more content and functionality to evaluate. The next round of questions will focus more on specific activities and hopefully some of the additional resources. The biggest concerns I have at this point in my EdWeb project concern the actual time I have to develop these modules. The activities will be fairly media rich for my first EdWeb project so I hope I haven‟t bitten off more I can chew. Thankfully, I have had the opportunity to design a lot of similar courses in my current job at TeleTech, so some of the content I will be able to reuse with some minor modifications.

This table illustrates the color scheme that I plan to use throughout my entire website. The boldest colors will be used for Menu bars and footers. The medium hues are reserved for the headings for modules and activities and hyperlink text. Finally, the lightest color will be used for back grounds. Each learning module is assigned a color and the colors will remain consistent for their respective pages. All text laid over the colored boxes will be in white and all text laid over the backgrounds will be in black.

There are four fonts that will be used throughout the website:

Bookman Old Style font size 20 is used for titles and headers

Arial font size 14 is used for paragraph text in the main body.

Arial Rounded MT Bold font size 18 is used for larger text in the main body

Bolded and Underlined Arial font size 14 is used for all hyperlink and footer text

Headings and hyperlink text

#/RGB Menu bars and footers

#/RGB Backgrounds #/RGB

# 777CAS RGB – 119.124.165

#404162 RGB – 64.65.98

#DEE0E9 RGB – 222.224.233

#A58E77 RGB – 165.142.119

#625240 RGB – 98..82.64

#E9E89E RGB – 233.232.222

#A5A177 RGB – 165.161.119

#626040 RGB – 98.96.64

#E9E1D3 RGB - 226.215.211

Home

Lessons

Conversational Style

Conversational Structure

Listening Comprehension

Active Listening

Listening for Main Points and Important Details

Word Stress and Intonation

Pronunciation Challenges

Asking for Information

Giving Information

Resources Blog Spot About Us

Global English Site Map

\

EdWeb Analysis Document Page 33 of 48

I ran the Flesch Reading Ease test for each page of the Active Listening lesson and got the following average results:

Each page of this lesson contains no more than 700 words. This word count per page is the standard I am following to prevent cognitive overload. At first blush, it may appear that the readability level is high for non-native speakers. It‟s important to note that although the learners are non-native speakers, they must already be at a high-intermediate/low-advance English level to pass the TeleTech language assessment. This roughly equates to eighth grade reading level which is slightly below the results shown on the readability test. One thing that may assist in this reading

discrepancy is the use of „global English” throughout the website. Global English is English that is free of idioms and cultural references that non-native speakers may not be familiar with. However, aside from this, I still think it‟s important for me to review the lesson content again and try increase the easy of readability.


While certain disabilities aren‟t exactly applicable to the target audience given the job requirements of the position for which they are applying, accessibility has been a concern. In light of this, the ADA guidelines have been very helpful to consider in the development of my prototype. The two primary accessibility issues concern: 1) inadequate technology on the user side and 2) English being a foreign language for the target audience. Technological inadequacies vary depending on income and location. Users with lower incomes may depend on older computers with out-dated devices and software. This may cause complications when accessing the audio and video files from the website. Lower income households as well as areas far from urban centers may struggle with low internet speeds. To address these technological challenges, I will include a web page that explains the technological requirements of the website. Furthermore, image, audio, and video files will be kept at the lowest possible file sizes to minimize these technological requirements. Language issues will also be a major accessibility factor. On a positive note, these language issues are consistent across the target audience so the content will only have to accommodate users within a certain range of language comprehension. To accommodate this range, only “global English” is used throughout the website. Moreover, lesson narration will be provided through out all of the lessons to 1) build on listening comprehension and pronunciation skills and 2) accommodate users with a more audio intelligence.


Formative Evaluation Process

For this evaluation, SMEs and target audience representatives will be given the questions via email. The form will be set up in a way that participants can type their answers in and send them directly back via email.

Participants

SMEs who will be participating in this evaluation are all on the TeleTech language readiness team. They include the language readiness manager, instructional designer, and language analyst. While the target audience representatives won‟t actually be the learners who will take these courses, they are non-native speakers who currently work in the call-center environment and therefore match the demographic profile of the future learners for this course. My point of contact in the Philippines has selected three TeleTech employees who have agreed to participant in the evaluation.

SME Feedback


1. Do you feel that the lesson topics are appropriate for helping learners achieve success in the module topics- Conversational Style, Listening Comprehension, and Conversational Structure?

a) If no, please write the

names of the topics

you feel are

inappropriate and

state why you think

so.

b) Are there any topics

that you think should

LR (Language Readiness) Manager: “I think you should add lessons about cross-cultural communication. Often we find that our agents have more problems with this than language issues. Maybe add topics that address time consciousness and natural communication.” LR Instructional Designer: “I think the topics look great. We really emphasize all of them in our current language curriculum, so I think it‟s good

1. Course content will have less of a customer service context and focus on contexts that learners are more than likely already familiar with. 2. I‟ve already made the decision to include topics on cross-cultural communication but haven‟t included them on the functional prototype. Cross-cultural communication will be a forth module to the EdWeb and will include a lesson on Cultural Dimensions and a lesson on Strategies for Cross-cultural Communication.


be added? If so,

please write

suggested topic

names and why you

think they would be

helpful.

to prepare potential agents ahead of time.” LR Analyst (Global Language Champion): “Topics look good. I would try to make the content more general since they aren‟t really focused on the customer service context yet.”

2. Do you think the

website is easy enough

for a TeleTech trainee to

navigate? If not, what

recommendations do you

have?

LR (Language Readiness) Manager: “I would provide a site map so you can reference all the pages.” LR Instructional Designer: “You might want to provide links to let the learners skip directly to certain pages in the lessons.” LR Analyst (Global Language Champion): “I don‟t think there will be any major issues.”

1. A site map will be added. 2. Links to each lesson page will be included below the navigation path.

3. Do you think it‟s

important to provide audio

narration throughout the

courses?

LR (Language Readiness) Manager: “Yes. This is a great idea. We do that for our eLearning courses. I think it‟s just a best practice.” LR Instructional Designer: “Absolutely. You might want to explain how it works on a technical requirements page.”

Audio will be included for all lesson content.


LR Analyst (Global Language Champion): “I like the idea of narration but I don‟t think you need to narrate all the content. Users should definitely be able to turn it off if they wish.”

Target Audience Representative Feedback


1. What did you think of the navigation?

a) The navigation was very difficult to

understand. My suggestions for

improvement are…

b) The navigation was okay. I think it

would be better if you…

c) The navigation was good but it would

be better if you …

d) The navigation was excellent. I have

no suggestions for improvement.

Respondent 1 d) The navigation was excellent. I have no suggestions for improvement Respondent 2 b) The navigation was okay. I think it would be better if you… It‟s better if the fonts are more attractive. The fonts looked like they have been stretched. The colors also need to be improved on. Respondent 3 d) The navigation was excellent. I have no suggestions for improvement The presentation is concise,

I will look into using better fonts throughout the site.


precise, and very informative. Correct responses were identified and defined for the audience. There‟s simply nothing to improve on the Listening Skills portion.

2. Rank the lesson topics in the order of interest to

you- 1 being the most interesting and 6 being

the least interesting.

Respondent 1 1. Active Listening - This topic is the most interesting because… It has a directly relates to the other topics 6. Word Stress and Intonation - This topic is the least interesting because… Understanding is more important but communicating response is not to be neglected Respondent 2 1. Pronunciation Challenges - This topic is the most interesting because… for a lot of non- native speakers this is a challenge. 6. 6) Giving information - This topic is the least interesting because…

It appears that conversational structure is the least interesting for all participants. In light of this, I am considering replacing the conversational structure module with cross-cultural communication.


This topic is the most interesting because… I think of all the topics, I am most knowledgeable in this field. Respondent 3 1. Main Points and Important Details - This topic is the most interesting because we communicate in order to understand and be understood. It is important to get the substance of every topic, which is the very reason why we converse. 6. Asking for Information…This is the least interesting topic for me because if one can apply the skills in all five preceding topics, he or she will be able to effectively ask information in any way he/she does.

3. Overall, what do you think of the selection of the

images used in the active listening lesson.

a) The images didn‟t seem relevant to the

lesson topic.

b) The images were okay. They were

relevant to the topic but didn‟t really help

me to make a connection with the

material.

Respondent 1 b) The images were fine. Respondent 2 b) The images were fine. Respondent 3

The active listening lesson images were not all present for the prototype. Instead, place holder buttons were used to represent the active listening strategies. I will include images for these strategies that create direct associations.


c) The images were good. They helped me

to make a connection to the course

material.

a) The images were great.

4. I feel like the content presented in the active

listening lesson could help an associate with

listening comprehension.

a) Completely agree

b) Somewhat agree

c) Somewhat disagree

d) Completely disagree

Respondent 1 a) Completely agree Respondent 2 a) Completely agree Respondent 3 a) Completely agree

No changes to be made.

5. Below are three test questions to see how

much you learned in the active listening

lesson. Read what the customer says, and

then identify the most appropriate response.

1) “I‟m getting married, so I need to make a

reservation for next Friday.”

a) That‟s great! How long have you known each other? Where did you two meet? b) I‟m not sure if you can get a discount, but I‟ll check for you. c) I don‟t know what‟s available for that date. Let me check to see what we have. d) That‟s wonderful news!

Respondent 1 1. correct 2. correct 3. correct Respondent 2 1. correct 2. incorrect 3. correct Respondent 3 1. correct 2. correct

The participants seem to respond well to this lesson. Because these test questions are to ultimately test active listening skills, the customer speech for each question will be recorded in an audio file for each question.


Congratulations! Let me see what we have available.

2) “I‟d like to go for the cheapest plan you‟ve got”

a) So to clarify, you‟d like to sign up for the plan with the best rate? b) So if I understand correctly, you‟d like to go for the cheapest plan we‟ve got? c) What do you mean by “cheap plan” exactly? d) To “go for” means to “sign up for” right?

3) “You really don‟t understand what I‟m

saying, do you?”

a) I‟m sorry. I wasn‟t paying attention. There are a lot of distractions here. b) I apologize for the miscommunication. Let me ask you a few clarifying questions c) I‟m sorry, but your accent is very difficult for me to understand. d) I would like to understand you better. Can you repeat what you just said?

3. correct


Given that this was my first time developing a website, I have to admit that I was rather nervous getting feedback from both SMEs and the target audience representatives. However, the feedback was very constructive and more focused on the conceptual design and navigational functionality. It‟s worth noting that I had originally planned on getting responses from four target audience representatives. However, because one was not available, I had to manage with three. Much of the feedback that I‟ve received concerning the topics was fairly positive with the exception of Conversational Structure. I believe this is largely due to the aversion to English grammar, created by years of tedious grammar instruction in the Filipino school system. I need to decide if this module is worth keeping in the program. Furthermore, because the respondents represented the Filipino population, it‟s not clear how our learners in areas such as Latin America or South Africa will feel about topic. I believe multimedia interactivity is an absolute necessity to make this EdWeb not only effective but interesting as well. I hope that user accessibility and my own media development capability will allow me to provide a fully functional and enjoyable multimedia EdWeb. Reactions to all respondents have led me to believe that the homepage needs to be changed. This includes the layouts and the use of colors. I‟d really like flash interaction on the homepage that will allow learners to click on a module tab and read more about the module topics. I think this interaction will help to motivate learners from the beginning and better organize the homepage.


All pages Organizational





Active listening 00-10

Organizational


Organizational

Listening comprehension, Conversational Style, Active listening 00-10

Organizational

Representative (place holder for appropriate clip art)








Global English, Conversational Structure

Organizational

Listening Comprehension, Active listening 00-10

Organizational



Organizational

Active listening 02

Representative

Active listening 03

Representative

Active listening 03

Representative

Active listening 03

Representative

Active listening 04

Representative


Active listening 04

Representative

Active listening 03

Representative


Representative

Conversational Structure

Representative





Decorative

Global English Decorative


Decorative

Global English Representative

Global English Representative

Global English, Conversational Structure

Representative


Representative

Listening Comprehension, Active Listening 00-10

Representative


The feedback I received on the Functional Prototype of my EdWeb was very constructive. I should also add that reviewing other EdWebs gave me a great deal of insight into my own. Fortunately, the corrections that I need to make are not too labor intensive. Here is a short list of the changes and additions that I need to make to my site:

Contact Information: This includes adding contact information in the footer and developing the About Us page.

Site Map: Feedback indicated that the navigation path below the navigation bar was not enough for a point reference in the website. I think it would be worth adding this as an option in the navigation bar.

Lesson page links: I think full list of lesson page links below the navigation bar would be very helpful for learners to quickly reference certain pages and quickly return to the page that they were on.

Technical requirements: While none of my peer reviews suggested this, it become obvious when I was going through the ADA checklist that I should add a requirements page. This page will not only list the technical requirements but also the language skills necessary for completing the lessons in the EdWeb.

More consistent alignment: Some of the alignment issues in my site can really be resolved by simply modifying the HTML code. I‟ve learned some important lessons with Dream Weaver that should allow me to build a cleaner page.

More Effective Typology: I need to revisit some of the headings and experiment with different fonts, especially from the home page.

The most important thing I learned from critiquing two of my colleagues' Functional Prototypes was how easily information might be misinterpreted from your website. I‟ve realized clearly identifying the purpose of each webpage in your site is essential. We can‟t assume that our audience knows what we want them to do or why we want them to do it. This is why peer review is so important in instructional design. Sometimes a designer has spent so much time looking over the content of their website that they lose a sense of clarity and objectivity.

Developing this prototype has given me a lot of insight into what needs to be developed and even what needs to be dismantled. Feedback from my functional prototype evaluation as well as comments made off the records indicates to me that learners might not respond too enthusiastically to the grammar module. So I‟m seriously considering putting a Cross-Cultural Communication module in its. Perhaps a job aid would suffice for the grammar points. I think the website will really take off once I add the multimedia interactions. Also, I‟m really looking forward to developing the “blog spot” area of my site. This area will be a wonderful opportunity for me to communicate with the learners and learners to communicate with each other.

Documents

EdWeb Analysis and Design Document - Weeblygregmason.weebly.com/uploads/2/7/7/3/2773728/edweb... · 2020. 1. 28. · assessment scores for Near Hire applicants and increase the number