List of Suggested Reviewers or Reviewers Not To Include …ceick/EDM/NSF_SPP-SAS_Proposal-submitted.pdf · 2018. 6. 18. · Rizkallah Rejane: Eagle SAL Thesis Advisor: Decarlo Paul:

List of Suggested Reviewers or Reviewers Not To Include (optional)

SUGGESTED REVIEWERS:Not Listed

REVIEWERS NOT TO INCLUDE:Not Listed

1

Investigator: Christoph F. Eick

Collaborators and Co-EditorsLast Name First Name Affiliations ConflictDing Wei UMASS Boston CollaboratorChen Guoning Univerisity of Houston CollaboratorPaul Amalaman Universtity of Houston CollaboratorFatih Akdag Unversity of Houston CollaboratorSujing Wang Lamar University, Beaumont CollaboratorYongli Zhang University of Houston CollaboratorTianxing Cai Lamar University, Beaumont CollaboratorZechun Cao University of Houston Collaborator

Graduate Advisors and Postdoctoral SponsorsLast Name First Name Affiliations ConflictLockemann Peter Uni Karlsruhe, Germany Graduate Advisor

Thesis Advisor and Postgraduate-Scholar SponsorLast Name First Name Affiliations ConflictAmalaman Paul Not known Thesis AdvisorCheng Chun-sheng Price Line Thesis AdvisorRuth Miller Washington University Thesis AdvisorDing Wei UMASS Boston Thesis AdvisorRinsurkawong Vadeerat Not known Thesis AdvisorJianthapthaksin Rachsuda Asumption Un., Thailand Thesis AdvisorZeidat Nidal IBM Thesis AdvisorRui Christopher Cal State Fullerton Thesis AdvisorTuttle Sharon Humboldt State Univesity Thesis AdvisorWang Jing AOL Thesis AdvisorDavis Clifton Not known Thesis AdvisorThomas Justin John Hopkins University Thesis AdvisorWang Sujing Lamar University, Beaumont Thesis AdvisorWu Meikang Microsoft Thesis AdvisorWang Chong Apple Thesis Advisor

Collaborators & Other Affiliations Information

1

Investigator: Sara G. McNeil

Collaborators and Co-EditorsLast Name First Name Affiliations ConflictRobin Bernard University of Houston CollaboratorGronseth Susie University of Houston CollaboratorLee Mimi University of Houston CollaboratorLittle Julie EDUCAUSE Collaborator

Graduate Advisors and Postdoctoral SponsorsLast Name First Name Affiliations ConflictLin Meng-Fen (Grace) University of Hawaii Graduate AdvisorStill Beverly Houston Community College Graduate AdvisorPerez Juan San Jacinto College Graduate AdvisorPhan Trang Cal. State University, Fresno Graduate AdvisorWilson Dawn Houston Baptist University Graduate AdvisorBump Douglas Houston Community College Graduate Advisor

Thesis Advisor and Postgraduate-Scholar SponsorLast Name First Name Affiliations Conflict


1

Investigator: Nouhad J. Rizk

Collaborators and Co-EditorsLast Name First Name Affiliations ConflictChoueiri Elias Lebanese University CollaboratorGarbajal Santiago Univerisity of Madrid CollaboratorSayah Margot Lebanese University CollaboratorSalem Abd Badih Ein Shams University CollaboratorDecarlo Paul University of Houston Collaborator

Graduate Advisors and Postdoctoral SponsorsLast Name First Name Affiliations ConflictBusher Hugh Leicester University Graduate Advisor

Thesis Advisor and Postgraduate-Scholar SponsorLast Name First Name Affiliations ConflictJidagam Rohith Microsoft Thesis AdvisorRizkallah Rejane Eagle SAL Thesis AdvisorDecarlo Paul University of Houston Thesis AdvisorSalameh Tony TT LLC Thesis AdvisorBouFarhat Hadi Ogero Thesis AdvisorKhalifeh Antoine Arc-en-ciel Thesis Advisor


Not for distribution

COVER SHEET FOR PROPOSAL TO THE NATIONAL SCIENCE FOUNDATIONFOR NSF USE ONLY

NSF PROPOSAL NUMBER

DATE RECEIVED NUMBER OF COPIES DIVISION ASSIGNED FUND CODE DUNS# (Data Universal Numbering System) FILE LOCATION

FOR CONSIDERATION BY NSF ORGANIZATION UNIT(S) (Indicate the most specific unit known, i.e. program, division, etc.)

PROGRAM ANNOUNCEMENT/SOLICITATION NO./DUE DATE Special Exception to Deadline Date Policy

EMPLOYER IDENTIFICATION NUMBER (EIN) ORTAXPAYER IDENTIFICATION NUMBER (TIN)

SHOW PREVIOUS AWARD NO. IF THIS ISA RENEWALAN ACCOMPLISHMENT-BASED RENEWAL

IS THIS PROPOSAL BEING SUBMITTED TO ANOTHER FEDERALAGENCY? YES NO IF YES, LIST ACRONYM(S)

NAME OF ORGANIZATION TO WHICH AWARD SHOULD BE MADE ADDRESS OF AWARDEE ORGANIZATION, INCLUDING 9 DIGIT ZIP CODE

AWARDEE ORGANIZATION CODE (IF KNOWN)

IS AWARDEE ORGANIZATION (Check All That Apply) SMALL BUSINESS MINORITY BUSINESS IF THIS IS A PRELIMINARY PROPOSAL(See GPG II.C For Definitions) FOR-PROFIT ORGANIZATION WOMAN-OWNED BUSINESS THEN CHECK HERE

NAME OF PRIMARY PLACE OF PERF ADDRESS OF PRIMARY PLACE OF PERF, INCLUDING 9 DIGIT ZIP CODE

TITLE OF PROPOSED PROJECT

REQUESTED AMOUNT

$

PROPOSED DURATION (1-60 MONTHS)

months

REQUESTED STARTING DATE SHOW RELATED PRELIMINARY PROPOSAL NO.IF APPLICABLE

THIS PROPOSAL INCLUDES ANY OF THE ITEMS LISTED BELOWBEGINNING INVESTIGATOR (GPG I.G.2)

DISCLOSURE OF LOBBYING ACTIVITIES (GPG II.C.1.e)

PROPRIETARY & PRIVILEGED INFORMATION (GPG I.D, II.C.1.d)

HISTORIC PLACES (GPG II.C.2.j)

COLLABORATIVE STATUSVERTEBRATE ANIMALS (GPG II.D.6) IACUC App. DatePHS Animal Welfare Assurance Number

HUMAN SUBJECTS (GPG II.D.7) Human Subjects Assurance Number

Exemption Subsection or IRB App. Date

INTERNATIONAL ACTIVITIES: COUNTRY/COUNTRIES INVOLVED (GPG II.C.2.j)

FUNDING MECHANISM

PI/PD DEPARTMENT PI/PD POSTAL ADDRESS

PI/PD FAX NUMBER

NAMES (TYPED) High Degree Yr of Degree Telephone Number Email Address

PI/PD NAME

CO-PI/PD

CO-PI/PD

CO-PI/PD

CO-PI/PD

Page 1 of 3

DUE - IUSE- Exploration & Design: Engaged Student Learni

NSF 15-585 11/02/16

746001399

University of Houston

0036525000

University of Houston4800 Calhoun BoulevardHouston, TX. 772042015

University of HoustonUniversity of Houston University of Houston

TX ,772042015 ,US.

Design and Implementation of a Student Performance Prediction and Self-Assessment System to Enhance Student Retention

299,879 24 06/01/17

Department of Computer Science

713-743-3335

4800 Calhoun

Houston, TX 77204United States

Christoph F Eick PhD 1984 713-743-3345 [email protected]

Sara G McNeil EdD 1995 713-743-4975 [email protected]

Nouhad Rizk DSc 2007 713-743-3710 [email protected]

036837920

Not a collaborative proposalResearch - other than RAPID or EAGER


CERTIFICATION PAGE

Certification for Authorized Organizational Representative (or Equivalent) or Individual Applicant

By electronically signing and submitting this proposal, the Authorized Organizational Representative (AOR) or Individual Applicant is: (1) certifying that statements made herein are true and complete to the best of his/her knowledge; and (2) agreeing to accept the obligation to comply with NSF award terms and conditions if an award is made as a result of this application. Further, the applicant is hereby providing certifications regarding conflict of interest (when applicable), drug-free workplace, debarment and suspension, lobbying activities (see below), nondiscrimination, flood hazard insurance (when applicable), responsible conduct of research, organizational support, Federal tax obligations, unpaid Federal tax liability, and criminal convictions as set forth in the NSF Proposal & Award Policies & Procedures Guide,Part I: the Grant Proposal Guide (GPG). Willful provision of false information in this application and its supporting documents or in reports required under an ensuing award is a criminal offense (U.S. Code, Title 18, Section 1001).

Certification Regarding Conflict of Interest

The AOR is required to complete certifications stating that the organization has implemented and is enforcing a written policy on conflicts of interest (COI), consistent with the provisionsof AAG Chapter IV.A.; that, to the best of his/her knowledge, all financial disclosures required by the conflict of interest policy were made; and that conflicts of interest, if any, were,or prior to the organization’s expenditure of any funds under the award, will be, satisfactorily managed, reduced or eliminated in accordance with the organization’s conflict of interest policy.Conflicts that cannot be satisfactorily managed, reduced or eliminated and research that proceeds without the imposition of conditions or restrictions when a conflict of interest exists,must be disclosed to NSF via use of the Notifications and Requests Module in FastLane.

Drug Free Work Place Certification

By electronically signing the Certification Pages, the Authorized Organizational Representative (or equivalent), is providing the Drug Free Work Place Certification contained in Exhibit II-3 of the Grant Proposal Guide.

Debarment and Suspension Certification (If answer "yes", please provide explanation.)

Is the organization or its principals presently debarred, suspended, proposed for debarment, declared ineligible, or voluntarily excluded from covered transactions by any Federal department or agency? Yes No

By electronically signing the Certification Pages, the Authorized Organizational Representative (or equivalent) or Individual Applicant is providing the Debarment and Suspension Certification contained in Exhibit II-4 of the Grant Proposal Guide.

Certification Regarding LobbyingThis certification is required for an award of a Federal contract, grant, or cooperative agreement exceeding $100,000 and for an award of a Federal loan or a commitment providing for the United States to insure or guarantee a loan exceeding $150,000.

Certification for Contracts, Grants, Loans and Cooperative AgreementsThe undersigned certifies, to the best of his or her knowledge and belief, that:(1) No Federal appropriated funds have been paid or will be paid, by or on behalf of the undersigned, to any person for influencing or attempting to influence an officer or employee of any agency, a Member of Congress, an officer or employee of Congress, or an employee of a Member of Congress in connection with the awarding of any Federal contract, the making of any Federal grant, the making of any Federal loan, the entering into of any cooperative agreement, and the extension, continuation, renewal, amendment, or modification of any Federal contract, grant, loan, or cooperative agreement.(2) If any funds other than Federal appropriated funds have been paid or will be paid to any person for influencing or attempting to influence an officer or employee of any agency, a Member of Congress, an officer or employee of Congress, or an employee of a Member of Congress in connection with this Federal contract, grant, loan, or cooperative agreement, the undersigned shall complete and submit Standard Form-LLL, ‘‘Disclosure of Lobbying Activities,’’ in accordance with its instructions.(3) The undersigned shall require that the language of this certification be included in the award documents for all subawards at all tiers including subcontracts, subgrants, and contracts under grants, loans, and cooperative agreements and that all subrecipients shall certify and disclose accordingly.

This certification is a material representation of fact upon which reliance was placed when this transaction was made or entered into. Submission of this certification is a prerequisite for making or entering into this transaction imposed by section 1352, Title 31, U.S. Code. Any person who fails to file the required certification shall be subject to a civil penalty of not lessthan $10,000 and not more than $100,000 for each such failure.

Certification Regarding Nondiscrimination

By electronically signing the Certification Pages, the Authorized Organizational Representative (or equivalent) is providing the Certification Regarding Nondiscrimination contained in Exhibit II-6 of the Grant Proposal Guide.

Certification Regarding Flood Hazard Insurance

Two sections of the National Flood Insurance Act of 1968 (42 USC §4012a and §4106) bar Federal agencies from giving financial assistance for acquisition or construction purposes in any area identified by the Federal Emergency Management Agency (FEMA) as having special flood hazards unless the: (1) community in which that area is located participates in the national flood insurance program; and(2) building (and any related equipment) is covered by adequate flood insurance.

By electronically signing the Certification Pages, the Authorized Organizational Representative (or equivalent) or Individual Applicant located in FEMA-designated special flood hazard areas is certifying that adequate flood insurance has been or will be obtained in the following situations: (1) for NSF grants for the construction of a building or facility, regardless of the dollar amount of the grant; and(2) for other NSF grants when more than $25,000 has been budgeted in the proposal for repair, alteration or improvement (construction) of a building or facility.

Certification Regarding Responsible Conduct of Research (RCR) (This certification is not applicable to proposals for conferences, symposia, and workshops.)

By electronically signing the Certification Pages, the Authorized Organizational Representative is certifying that, in accordance with the NSF Proposal & Award Policies & Procedures Guide, Part II, Award & Administration Guide (AAG) Chapter IV.B., the institution has a plan in place to provide appropriate training and oversight in the responsible and ethical conduct of research to undergraduates, graduate students and postdoctoral researchers who will be supported by NSF to conduct research. The AOR shall require that the language of this certification be included in any award documents for all subawards at all tiers.

Page 2 of 3


CERTIFICATION PAGE - CONTINUED

Certification Regarding Organizational Support

By electronically signing the Certification Pages, the Authorized Organizational Representative (or equivalent) is certifying that there is organizational support for the proposal as required by Section 526 of the America COMPETES Reauthorization Act of 2010. This support extends to the portion of the proposal developed to satisfy the Broader Impacts Review Criterion as well as the Intellectual Merit Review Criterion, and any additional review criteria specified in the solicitation. Organizational support will be made available, as described in the proposal, in order to address the broader impacts and intellectual merit activities to be undertaken.

Certification Regarding Federal Tax Obligations

When the proposal exceeds $5,000,000, the Authorized Organizational Representative (or equivalent) is required to complete the following certification regarding Federal tax obligations. By electronically signing the Certification pages, the Authorized Organizational Representative is certifying that, to the best of their knowledge and belief, the proposing organization: (1) has filed all Federal tax returns required during the three years preceding this certification; (2) has not been convicted of a criminal offense under the Internal Revenue Code of 1986; and (3) has not, more than 90 days prior to this certification, been notified of any unpaid Federal tax assessment for which the liability remains unsatisfied, unless the assessment is the subject of an installment agreement or offer in compromise that has been approved by the Internal Revenue Service and is not in default, or the assessment is the subject of a non-frivolous administrative or judicial proceeding.

Certification Regarding Unpaid Federal Tax Liability

When the proposing organization is a corporation, the Authorized Organizational Representative (or equivalent) is required to complete the following certification regarding Federal Tax Liability: By electronically signing the Certification Pages, the Authorized Organizational Representative (or equivalent) is certifying that the corporation has no unpaid Federal tax liability that has been assessed, for which all judicial and administrative remedies have been exhausted or lapsed, and that is not being paid in a timely manner pursuant to an agreement with the authority responsible for collecting the tax liability.

Certification Regarding Criminal Convictions

When the proposing organization is a corporation, the Authorized Organizational Representative (or equivalent) is required to complete the following certification regarding Criminal Convictions: By electronically signing the Certification Pages, the Authorized Organizational Representative (or equivalent) is certifying that the corporation has not been convicted of a felony criminal violation under any Federal law within the 24 months preceding the date on which the certification is signed.

Certification Dual Use Research of Concern

By electronically signing the certification pages, the Authorized Organizational Representative is certifying that the organization will be or is in compliance with all aspects of the United States Government Policy for Institutional Oversight of Life Sciences Dual Use Research of Concern.

AUTHORIZED ORGANIZATIONAL REPRESENTATIVE SIGNATURE DATE

NAME

TELEPHONE NUMBER EMAIL ADDRESS FAX NUMBER

fm1207rrs-07

Page 3 of 3

NATIONAL SCIENCE FOUNDATIONDivision of Undergraduate Education

NSF FORM 1295: PROJECT DATA FORM

The instructions and codes to be used in completing this form are provided in Appendix II.

1. Program-track to which the Proposal is submitted: IUSE- Exploration & Design: Engaged Student Learni

2. Name of Principal Investigator/Project Director (as shown on the Cover Sheet):

Eick, Christoph

3. Name of submitting Institution (as shown on Cover Sheet):


4. Other Institutions involved in the project’s operation:

Project Data:

A. Major Discipline Code: 31

B. Academic Focus Level of Project: LO

C. Highest Degree Code: D

D. Category Code: --

E. Business/Industry Participation Code: NA

F. Audience Code:

G. Institution Code: PUBL

H. Strategic Area Code:

I. Project Features: 1 3

Estimated number in each of the following categories to be directly affected by the activities of the projectduring its operation:

J. Undergraduate Students: 450

K. Pre-college Students: 0

L. College Faculty: 2

M. Pre-college Teachers: 0

N. Graduate Students: 0

NSF Form 1295 (10/98)

PROJECT SUMMARY

Overview:Undergraduate education is challenged by high dropout rates and by delayed student graduation due todropping courses or having to repeat courses due to low academic performance. In this context, an earlyprediction of student course performance may help to identify students who need special attention toreduce course drop rates by providing appropriate interventions, such as mentoring and conductingreview sessions. Moreover, it is critical that such skill-based assessment occurs in the early stages of thesemester and is provided in real-time. In addition to providing external assessment capabilities thatidentify student deficiencies in particular skills, it is desirable that students are taught to learn how toassess their own skills as such capabilities will frequently enable them to take proper measures to enhancetheir academic performance in a particular course. The proposed project addresses these two issues; itsgoal is the design and implementation of a Student Performance Prediction and Self-Assessment System(SPP-SAS) to reduce retention rates. SPP-SAS is an interactive, web-based system that consists of a Self-assessment Platform, a Student Performance Prediction System that uses data mining techniques tointerpolate student performance into the future, and an Early Warning System the identifies lowperforming students in courses. A prototype system of SPP-SAS will be developed for 2 undergraduateComputer Science courses, and its benefits will be assessed. Parts of the system design will include a coresystem that is reusable across courses; methodologies will be developed that facilitate to augment thecore prediction and self-assessment system with course-specific knowledge. Another important outcomeof this process is a unified representation of student background and student performance data that issuitable to be used to assess student performance across multiple courses, which is critical for thegeneralizability of the prototype systems.

Intellectual Merit:The project develops course-based rather than a general student performance prediction models andsupports making predictions in the presence of a lot of missing values, allowing to predict studentperformance very early in the semester. Designing and developing this Performance Prediction and Self-Assessment System will provide much needed insights about how to individually support students incourses and improve their self-assessment skills, which will increase course completion and ultimatelystudents' academic achievement. To the best of our knowledge, this is the first project that studies theinteractions between student self-assessment and data-driven student performance prediction models.Moreover, this project has the potential to transform this field by creating a responsive system that can beused by instructors in other courses to provide real time feedback to students about their progress inmeeting learning goals and to suggest specific resources to improve students' performance and skills.

Broader Impacts:This project intends to create a system with three critical areas that operate in unison: an Early WarningSystem that operates on the top of the Self-Assessment Platform and the Student Performance PredictionSystem. This will result in a more effective, student-centered way to increase student retention in criticalSTEM courses, Understanding the needs of these diverse students and how to improve their academicachievement will positively affect student success and create increased opportunities for students fromtraditionally underrepresented groups to succeed in STEM careers, such as computer science. During thecourse of the project data-driven, course-based student performance prediction models will be developed.In addition to predicting student course performance, the obtained models can be used to identify factorsthat are associated with poor academic performance. Knowing such factors is of critical importance toselect interventions that reduce the dropout rates.

TABLE OF CONTENTSFor font size and page formatting specifications, see GPG section II.B.2.

Total No. of Page No.*Pages (Optional)*

Cover Sheet for Proposal to the National Science Foundation

Project Summary (not to exceed 1 page)

Table of Contents

Project Description (Including Results from Prior

NSF Support) (not to exceed 15 pages) (Exceed only if allowed by aspecific program announcement/solicitation or if approved inadvance by the appropriate NSF Assistant Director or designee)

References Cited

Biographical Sketches (Not to exceed 2 pages each)

Budget (Plus up to 3 pages of budget justification)

Current and Pending Support

Facilities, Equipment and Other Resources

Special Information/Supplementary Documents(Data Management Plan, Mentoring Plan and Other Supplementary Documents)

Appendix (List below. )

(Include only if allowed by a specific program announcement/solicitation or if approved in advance by the appropriate NSFAssistant Director or designee)

Appendix Items:

*Proposers may select any numbering mechanism for the proposal. The entire proposal however, must be paginated.Complete both columns only if the proposal is numbered consecutively.

1

1

15

4

6

5

3

3

7

1

Project Description “Design and Implementation of a Student Performance Prediction and Self-Assessment System

to Enhance Student Retention”

1 Introduction

1.1 Need for the Project

Despite a strong and intensive effort by colleges and universities to improve student retention and academic performance over the past three decades, retention rates remain flat. The first to second year retention rate for undergraduate students at four-year public institutions in 2005 was 72% [1]. Ten years later, in 2015, the retention rate for first to second year students was 73%; over a quarter of the students who start college leave after their first year, and only 32% of students who start college graduate within four years [2].

Graduation rates for students in STEM majors is even lower. According to a report by the National Center for Education Statistics [12], about 28% of bachelor’s degree students entered a STEM field (i.e., chose a STEM major) at some point within 6 years of entering postsecondary education in 2003−04; however, 48% left the field either by changing majors or leaving college without completing a degree. Other studies indicated that many of the students who left the STEM fields were actually high-performing students who might have made valuable additions to the STEM workforce had they stayed [23, 37]. To produce more graduates in STEM fields, some recent U.S. policies have focused on reducing students’ attrition from STEM fields in college, arguing that increasing STEM retention by even a small percentage can be a cost-efficient way to contribute substantially to the supply of STEM workers [12, 13, 16]. Retention and graduation rates are also a major challenge for the University of Houston, as can be seen in Table 1.

Table 1. University of Houston graduation rates for full-time first time in college undergraduates 2008-2011.

Year Freshman Enrolled at

UH

Cumulative Graduation rate

in 4 years

Cumulative Graduation rate

in 5 years

Cumulative Graduation

rate in 6 years 2008 3,486 18.3% 38.8% 48.1% 2009 3,100 19.7% 41.6% 51% 2010 3,453 22.7% 42.5% 2011 3,556 25.2%

According to the 2016 National Freshman Motivation to Complete College Report [20], many students have indicated that they need support to help them succeed in college. Support preferences of respondents at 4 year public institutions in this study were as follows:

80.5% would like to receive instruction in the most effective ways to take college exams. 65.9% would like to receive help in improving study habits. 53.8% would like to receive tutoring in one or more of my courses.

2

It is critical to identify students who are struggling as early in the semester as possible, so that such students can be warned, and appropriate interventions can be implemented. Faculty members need tools that will allow them to quickly identify at-risk students and devise ways of supporting their learning. Early Warning Systems (EWSs) have been used to identify students who are most likely to fail academically or to encounter serious problems adapting to the college environment. Many colleges purchase a EWS in order to identify these high-risk students and provide the support they need before they leave school due to failure or social adjustment problems. These EWSs use “predictors” such as scores from standardized tests, high school grade point averages and class rank to identify struggling students. The problem with a typical EWS is that they are used across courses at the end of a semester and cannot be used by instructors during a course to quickly respond to course-specific problems. In addition, since a EWS is used for all courses across campus, they are designed with a general predictive model that cannot address the complexity of specific courses.

Moreover, there is general consensus among educational researchers that students benefit from self-assessment skills. It is desirable that student have a realistic and accurate sense of their own achievements to direct their studying in productive directions [9]. Studies about self-assessment suggest that students who perform well academically are more accurate in predicting and evaluating their academic performance than those who perform poorly. While consistently viewed as a necessary and valuable skill, self-assessment is not well understood, particularly regarding its role in improving academic performance and retention in STEM courses.

This proposal addresses the two issues that were discussed in the last two paragraphs; its objective is the design and implementation of a Student Performance Prediction and Self-Assessment System (SPP-SAS) to reduce retention rates. An overview of the goals and scope of its proposed research will be given next.

1.2 High Level Goals and Scope of the Proposed Project

As mentioned earlier, undergraduate education is challenged by high dropout rates and by delayed student graduation due to dropping courses or having to repeat courses due to low academic performance. In this context, an early prediction of student course performance may help to identify students who need special attention to reduce course drop rates by providing appropriate interventions, such as mentoring and conducting review sessions. In addition to providing external assessment capabilities that identify student deficiencies, it is desirable that students are taught to learn how to assess their own skills as such capabilities will frequently enable them to take proper measures to enhance their academic performance in a particular course.

The goal of this proposal is to alleviate student drop out and to reduce student graduation times along the lines we discussed in the previous paragraph. The proposed research investigates student performance assessment taking two perspectives: First, we will investigate if emphasizing student self-assessment in our curriculum leads to better student performance in courses. In particular, an interactive web-based platform will be developed for student self-assessment with respect to the skills taught in a particular course, measures will be designed and implemented to assess to which extent a student’s self-assessment matches his/her actual

3

performance in particular course, and a feedback system will be developed to communicate such findings and performance-related findings to students.

The second perspective of student performance that is explored in this proposal is the data-driven prediction of student performance in courses. One particular goal of this proposal is the design and implementation of a real-time early warning system that helps to identify low performing students early in the course. Using a data mining approach the proposed early warning system learns different classification and prediction models to predict student course performance based on their attendance, performance in assignments, quizzes, in-class group projects, exams, their self-assessment skills and information about student socio-economic background, learning styles, available resources, and other data.

The student performance prediction and self-assessment system will initially be developed for two Computer Science Undergraduate courses, COSE 2440 and COSC 1320, and a prototype implementation has already started this semester. Parts of the system design will include a core system that is reusable across courses; moreover, methodologies will be developed that facilitate to augment the core prediction and self-assessment system with course-specific knowledge. As far as the prototype system is concerned, raw data are collected from multiple knowledge sources, integrated, and ultimately stored in a relational database. One important outcome of this process is a unified representation of student background and student performance data that is suitable to be used to assess student performance across multiple courses, which is critical for the generalizability of the prototype system, we develop.

1.3 Alignment of the Proposed Project with the NSF Solicitation 15-585

The proposal is submitted to the program track “Exploration and Design: Engaged Student Learning”. The project addresses an immediate and significant challenge that is facing undergraduate STEM education: the retention and support of STEM students from all demographic groups particularly in introductory courses in the first two years of college. The focus of the project is on the development, use, and testing of the Student Performance Prediction and Self-Assessment System (SPP-SAS) with the goal of increasing the retention and academic progress of STEM students in computer science. The application of this system is designed to increase the engagement of undergraduate students in their STEM learning by making them active participants in evaluating their progress and improving their ability to self-assess their learning in order to study more effectively.

Since the University of Houston (UH) is one of the nation’s most diverse universities in one of the most diverse cities in the United States, UH is especially cognizant of our mission “to offer nationally competitive and internationally recognized opportunities for learning, discovery and engagement to a diverse population of students in a real-world setting.” Like the University of Houston, this project is designed to embrace diversity and foster an open, welcoming environment where students of all backgrounds can collaboratively learn, work and serve.

In this project we seek to establish the basis for the development and implementation of a three parts of a system to support undergraduate STEM education by increasing the retention and academic progress of students: 1) a Self-Assessment Platform to improve students’ self-

4

assessment of their content knowledge and skills; 2) a Student Performance Prediction System to predict student achievement; and 3) an Early Warning System to communicate feedback to students and instructors about individual students’ progress in meeting learning goals and to suggest specific resources for support.

The overarching goal of this project is to inform policy, practice, and future design of tools to support the retention of undergraduate STEM students. Project results will be shared at education and data mining conferences and will provide the foundation for expanding and adapting the SPP-SAS model into other STEM courses.

2 Project Tasks and Research Themes

The proposal centers on the design, implementation, and evaluation of a Student Performance Prediction and Self-Assessment System (SPP-SAS). SPP-SAS consists of a Self-Assessment Platform, a Student Performance Prediction System and an Early Warning and Feedback System which operates on the top of the Student Prediction System. Figure 1 shows the architecture of SPP-SAS. Moreover, in order for those three components to work, data collection, standardization, and integration procedures have to be developed which is the 4th task of the proposed research. Moreover, SPP-SSA will be deployed, tested and evaluated in a pilot study for two Computer Science courses: COSC 2440 and COSC 1320 which is 5th task of the project. Finally, in Task 6, we will quantitatively assess how much effort it will take to “re-use” SPP-SAS for a new course. The remainder of this section describes the six tasks in more detail.

Figure 1: Architecture of SPP-SAS

5

2.1 Task 1: Self-Assessment Platform (SAP)

Researchers have found that there is little correlation between students' self-evaluation and their actual performance on exams [21]. Many students are over-optimistic about their performance, and this inaccurately inflated confidence is related to poorer academic performance [22]. This project will examine if by supporting students in developing self-assessment skills academic performance can be improved, particularly for first time in college (FTIC) students. A Self-Assessment Platform to facilitate self-assessment will be developed. In addition to improved academic performance, students who practice self-assessment learn metacognitive competencies such as self-judgment, self-reaction, task analysis, self-motivation and control [43]. Students also improve their self-regulation skills such as setting targets, evaluating progress relative to target criteria, and improving the quality of learning outcomes [6, 5, 10]. In addition, self-assessment is associated with improved motivation, engagement, and efficacy [15, 19, 30, 36], and reducing dependence on the teacher [33]. Using a Student Performance Prediction and Self-Assessment System is also seen as a potential way for teachers to reduce their own assessment workload, making students more responsible for tracking their progress [34, 39]. Moreover, the self-assessment process provides a framework for metacognition, and reflecting on their skills helps each student to clarify, consider, and prioritize his learning habits and behaviors. Most importantly, it allows students to begin feeling a sense of autonomy throughout their learning processes.

The self-assessment process will be led by the teacher and emphasizes skill knowledge; its purpose is to improve performance with meaningful motivation where students develop a critical skill to identify, self-monitor, self-evaluate, skills that are taught in a particular course. In our preliminary work, we deployed three self-assessments as part of a Computer Science course: before Exam1, Exam2 and the final exam. The process starts with a self-assessment survey, created for students, which provides questions that address proficiency in specific skills to be tested in an upcoming exam. Students respond with clickers and indicate their confidence in performing each of the skills. The response choices to given to students to assess their proficiency with respect to a particular skill were:

I cannot do it. I know how to do it some of the time, but I often need help. I know how to do it most of the time, but sometimes I need help. I know it very well. I know how to do it. I know it very well. I can teach it to others.

This self-assessment for each individual student will be collected through a web-based interface and uploaded into a self-assessment database. Subsequently, proportions of exam points associated with each skill will be identified for each exam question and also uploaded. Using the self-assessment platform, a student’s self-assessment will be compared with his/her actual exam performance, and the accuracy of his/her self-assessment will be determined. Quantitative self-assessment measures to assess how well the student’s self-prediction matches his/her actual performance in an exam, homework, or assignment will be developed for this purpose. The web-

6

based self-assessment system will also keep track of student self-assessment data that have been computed using the developed self-assessment measures. Feedback to the students about the accuracy of their prediction will be provided in an effort to improve student self-assessment accuracy. We claim that this dynamic process helps students self-evaluate their progress toward learning, reflect on their learning, and generate strategies to enhance their learning.

2.2 Task 2: Student Performance Prediction System (SPPS)

One specific challenge that universities face is high course drop rates. An early prediction of students’ failure may help to identify students who need special attention to reduce course drop rates by providing appropriate interventions, such as continuous mentoring and conducting review sessions. To provide such capabilities, a Student Performance Prediction System will be designed and implemented that learns models to predict student course performance based on his/her attendance, and performance in assignments, quizzes, in-class group projects, and exams. The investigated prediction approaches for this task include Multi-Layer Neural Networks, Support Vector Machines, Decision and Regression Trees, and Random Forests. We will determine experimentally which of those approaches performs best and then use this approach in SPPS. Data that have been collected from the previous teaching of the same course will be used as training data to learn prediction models that are then used to predict the performance of students that take the same course in the next semester.

One particular challenge is that the obtained prediction models have to deal with a lot of missing data, particularly if predictions have to be made early in the semester, when students have only completed a few of the course assignments, quizzes and exams. To deal with this problem, we will learn different models for different subsets of attributes that are associated with the progression of the teaching of the particular course. Moreover, particular inputs might be missing for particular students, e.g. a student might have missed Exam2 as she was sick; in this case the missing attribute will be replaced by the student average, if feasible or by the course average, otherwise.

Predicting student performance has been an active area of research [24, 18, 30, 8, 42, 28, 27, 41, 31, 25] in education data mining. Our proposed approach differs from what has been proposed in the literature in the following aspects:

The previous work discovers valuable knowledge that a faculty or an educational system can use to improve student performance, whereas our proposed approach identifies students that are more likely to fail so that corrective measures can be taken.

Most of the proposed prediction models are generic so that they can be used across courses, whereas our work learns prediction model not only from general performance data, but also from course specific data and the learnt models are tailored to predict student performance in a particular course, and not for all courses. Moreover, compared with the past work, we use much richer knowledge sources to learn prediction models; e.g. we consider student learning styles. We believe this is important to obtain accurate prediction models.

As we are developing an Early Warning System, our approach has to be capable to deal with missing data; all referenced work above assumes complete data.

7

Student self-assessment data will play a key role in student performance prediction.

2.3 Task 3: Early Warning and Feedback System (EWS)

Sarlo emphasized that developing an early warning system (EWS) within a larger framework is essential to prevent academic skill deficits and disengagement from occurring and to more effectively respond to these issues when they occur [35]. Many early warning systems have been developed to serve this need [38, 14, 32, 29] and data collected by such systems have been used to identify factors that are associated with student dropout [40, 32] Neild and Balfanz [7] identified three major challenges in devising the early intervention strategies to prevent dropout:

1. To figure out which signals to look for and when to look for them to identify at risk students;

2. To develop a set of structures and practices within schools that enable educators to review data and pinpoint those students who are sending signals; and

3. To determine the help that students need, on the basis of the signals they send and their responses to previous interventions.

Our proposed project focuses on dealing with the first two challenges. Most past EWS are developed based on pre-defined warning signs such as irregular attendance, poor academic performance, behavior problems and grade retention. However, the EWS we will develop takes into consideration social, economic, learning styles, self-assessment and study habits indicators in consideration, and also takes course specific indicators into consideration, and is more geared towards interpolating student performance into the future. Finally, as we use a data mining approach, our learnt models consider combinations of indicators to predict course dropout. For example our system is capable to learn complex signals, such that for a student coming from a poor socio-economic background, low attendance is a strong signal for course dropout—but not in general. The proposed early warning system operates on the top of the Self-Assessment Platform and the Student Performance Prediction System (as shown in Fig. 1) and will be web-based and interactive to enable students and teachers to track progress and to help students to be better assessors of their own knowledge. In order to develop the EWS, we will first identify communication and feedback strategies that are suitable for the EWS, design web interfaces to support these strategies, and implement the EWS. Finally, EWS will be enhanced based on feedback obtained from deploying EWS in two Computer Science undergraduate courses (see Section 2.5). 2.4 Task 4: Data Collection, Standardization, and Integration (DCSI)

All data produced during the course of the project will be stored in a relational database that will be designed for the needs of the project. One important outcome of this process is the relational database schema that gives a unified representation of student background and student performance data that is suitable to be used to assess student performance across multiple courses, which is critical for the generalizability of the prototype system that we develop. Moreover, software to input data that is collected in various formats into the relational database,

8

as well as software that exports data from the relational database to the Self-Assessment Platform, Student Performance Prediction System and Early Warning System will be developed.

For this project, we will collect data from three different sources: Cross sectional student data, longitudinal student data, and Student skill based self-assessment data:

Source 1: Cross sectional data will be collected via an online survey. This survey is divided into 5 subcategories, which are: economic factors, social factors, learning factors, institutional factors, general factors and demographic factors. Each category will have a set of questions for that particular domain.

Source 2: Longitudinal data is dependent on time. More data become available as the course progresses. We will collect in-class assessments of students every two weeks.

Source 3: Skill based self-assessment data will also collected from students in two phases. In Phase 1, students will be asked to assess themselves with respect to set of course related skills (i.e., polymorphism, inheritance, abstraction, encapsulation for an OOP’s course). In Phase 2 we will evaluate their performance by using a test with these skill related questions, and we collect their skill-related scores in the exams.

2.5 Task 5: Pilot Study—Deployment and Evaluation of SPP-SAS in two Computer Science Courses (PIL)

We will initially create and deploy the Student Performance Prediction and Self-Assessment System (SPP-SAS) for two Computer Science Undergraduate courses, COSE 2440 and COSC 1320. This deployment will be used test the three main software components we will develop. It will also be used to assess the accuracy of the Student Performance Prediction System. Moreover, we will obtain feedback from students and instructors concerning the usefulness of the Early Warning System and the Self-Assessment Platform and their user interfaces. This feedback will be used to enhance SPP-SAS.

Finally, we conduct controlled experiments that evaluate the usefulness of SPP-SAS in general, and of its components. In one section of each course, we will use the SPP-SAS to monitor student progress in the course and display this data in real time to the student and instructor (Experimental group). In the control section of each course, we will not provide this support to enrolled students. We will compare the two groups with respect to their course performance. Consistent with prior research studies, we expect that the students in the experimental group will perform better in the course than those in the control group. Table 2 summarizes the planned experimental design:

9

Table 2: Experimental Design to Evaluate SPP-SAS

Experimental Group Control Group Data from the UH Early warning System (SCC), including risk level for each enrolled student, are uploaded into the database.

Data from UH SSC, including risk level for each enrolled student, are uploaded into the database.

Information is added to the SPP-SAS about student progress including scores from in-class work, group work, homework, exams, etc.

Information is added to the SPP-SAS about student progress including scores from in-class work, group work, homework, exams, etc.

Instructor views real time data about individual student progress using the SPP-SAS Instructor Dashboard.

Instructor can view real time data about individual student progress using the SPP-SAS Instructor Dashboard.

Student views real time data about progress using SPP-SAS Student Dashboard.

No student view of SPP-SAS

Student receives recommendations for specific support resources based on their skill deficiencies

Student has access to all support resources, but none are specifically recommended.

In addition, we will survey the students who were in the Experimental Group about their perceptions of the Early Warning System and the Self-Assessment Platform. The survey is designed to elicit responses to two key questions: 1) What actions, if any, did the students take as a result of viewing the EWS information / SAS Information (i.e., “study more,” “talk to instructor,” “attend a tutoring session”) and 2) What feelings did the students have when they interacted with EWS / SAS (i.e., “scared,” “relieved that someone was paying attention,” “angered”)? Finally, the survey will also ask students specific questions how the EWS / SAS could be improved. 2.6 Task 6: Analysis of the Reusability of the Proposed System Across Courses (RAC)

The Student Performance and Self-Assessment will initially be developed for a two Computer Science courses; however, part of the system design will include a core system that is reusable across courses; moreover, methodologies will be developed that facilitate to augment the core system with course-specific knowledge. In Task 6, we will investigate how reusable the two prototype systems we developed are, and we will conduct a quantitative analysis what effort it will take to adapt the developed systems for a different course, and, near the end of the project a white paper will be written that summarizes our findings with respect to reusability of SPP-SAS.

3 The Research Team and Their Tasks

Dr. Christoph F. Eick from the Department of Computer Science at the University of Houston is the PI of the proposal. He is an expert in data mining, data sciences, artificial intelligence and geographic information systems. He is also the Director of the UH Data Analysis and Intelligent Systems Lab and serves in the program committee of top data mining conferences, such as ICDM

10

and PAKDD 2016. He will supervise the project and will lead the research and development associated with project tasks 2, 4, and 6. Dr. Sara McNeil is the program coordinator, Learning, Design and Technology Program, University of Houston, and she will be a Co-PI in the project. She is an associate professor and serves as Instructional Technology Graduate Program Coordinator at Houston University where she has won numerous awards for teaching and organizing distance learning programs. Her current research interests include: Instructional Design and Development, Learning Communities, Multimedia Learning, Collaboration and Collaborative Design. Dr. McNeil will be responsible for the Design of the Self-Assessment Platform (Task 1) and for the Pilot Study (Task 5). Dr. Nouhad Rizk who is also a Co-PI in the project has a very strong academic background in computer science combined with over twenty eight years work experience in industry as a chief information officer. She is the founder and counselor of IEEE-NSM branch at University of Houston. After completing her PhD in E-learning at Leicester University, her research focused on information retrieval and educational data mining. Dr. Rizk also serves as the Director of Undergraduate Studies in the Department of Computer Science. Dr. Rizk will supervise the development of the Early Warning and Feedback System (Task 3) and contribute to the Pilot Study and the design of the Self-Assessment Platform (Tasks 1 and 5). Moreover, the project Pilot studies will be conducted in the courses she teaches at UH. Moreover, the project will support two PhD students and one undergraduate student and 2-3 Master’s students will contribute to the project as part of their Master Thesis research.

4 Advisory Board

The project will be guided by an advisory board comprised of four members, but others may be added as necessary. We will meet with the advisory board bi-monthly in person or by video conferencing. They will be appraised of the progress of the tasks and their input solicited. The board will also assist with the analysis and interpretation of the data collected and will play an important role in the dissemination of the results. Projected duties and areas of expertise of its members are as follows: Dr. Arnon Hershkovitz will provide guidance in educational data mining. He is a Senior Lecturer in the Mathematics, Science and Technology Education Department, School of Education, Tel Aviv University (Israel). He earned his Ph.D. in Science Education from the School of Education, Tel Aviv University. Dr. Melissa Pierson will provide guidance in early warning systems. She is the Assistant Vice Provost for Undergraduate Student Success at UH, and she administers the EAB Student Success Collaborative (SSC), the system that UH currently uses to inflect student outcomes across all undergraduate courses. Her research interests include the appropriate use of technology in teacher education, the development of technology integration abilities in novice teachers, and the

11

relationship between teaching excellence and expertise with technology.

Dr. Joe Scandura will provide guidance in database design and organization. He has served as a professor in mathematics and education at Syracuse, SUNY Buffalo, Florida State and University of Pennsylvania with senior visiting and/or postdoctoral appointments in experimental & mathematical psychology, artificial intelligence and software engineering at Indiana, Michigan, Stanford, MIT, ETS and Drexel Universities. He is the author of over 200 scientific publications and is perhaps best known as developer of the Structural Learning Theory. Margaret Kuzynski is a statistician in the Institutional Effectiveness office in the College of Education. She will provide expertise in statistical analysis of our evaluation data.

5 Dissemination Plan

This project will design and implement an Early Warning System, a Self-Assessment Platform and a Student Performance Prediction System. Moreover, a relational database will be designed that stored the data of SPSS. We will make the software of these three system and the relational database design available to other researchers and instructors. We feel that this type of system can be applied to other STEM fields and will generate useful data for further research and exploration. We will disseminate our work by publishing in professional and practical journals and by presenting this project at both education and STEM conferences, as well as data mining conferences. The SPP-SAS portal will also have public information about project methods and results. We expect to regularly publish our experience and results in international technical conferences, and speak at regional and national meetings. The software developed will be freely made available to the academic community. The overall project is expected to last well beyond the timeline of proposed research. With the assistance of the Office of Undergraduate Studies at the University of Houston, we will also seek other STEM courses to test and evaluate the Student Performance Prediction and Self-Assessment System. In addition, the Advisory Board will play an active role in disseminating project results.

6 Intellectual Merit

This project will produce a course-based Student Performance Prediction and Self-Assessment System that is currently not available to students or instructors at the University of Houston and will result in techniques for providing specific, focused resources for students who need additional support to meet course goals and objectives. Designing and developing this Performance Prediction and Self-Assessment System will provide much needed insights about how to individually support students in courses and improve their self-assessment skills, which will increase course completion and ultimately students’ academic achievement. The research about student self-assessment will also be improved by the additional evaluation data obtained from the course-based Student Performance Prediction System. This project has the potential to transform this field by creating a responsive system that can be used by instructors in other

12

courses to provide real time feedback to students about their progress in meeting learning goals and to suggest specific resources to improve students’ performance and skills.

We claim that there are several contributions that unique and novel about this proposal to reduce retention rates for STEM majors, improving the state of the art. The most important contributions of the project are:

The project develops course-based rather than general student performance prediction models and support making predictions in the presence of a lot of missing values, allowing to predict student performance very early in the semester

It develop a course-based early warning system rather than a general, generic early warning system.

It develops a Self-Assessment Platform that supports skill-based self-assessment and evaluates its benefits of using this platform in low-level Computer Science courses.

To the best of our knowledge, this is the first project that studies the interactions between student self-assessment and data-driven student performance prediction models that have been obtained using data mining techniques.

7 Broader Impacts

Promoting Teaching and Learning: This project is focused on improving student learning and progress in specific course objectives and skills. It will result in insights about a Student Performance Prediction and Self-Assessment System that can improve student self-assessment skills and automatically suggest specific resources to correct learning deficiencies for individual students. This study will consider multiple sections of two freshman-level computer science courses, COSC 2440 and COSC 1320.

Increasing Retention of Diverse Students in STEM Courses: The University of Houston is the second most ethnically diverse major research university in the United States and a Hispanic Serving Institution. Our student body comprises 40,914 undergraduate and graduate students many of whom are first generation college students. Understanding the needs of these diverse students and how to improve their academic achievement will positively affect student success and create increased opportunities for students from traditionally underrepresented groups to succeed in STEM careers such as computer science.

Enhancing Existing Retention and Academic Performance Tools: This project builds on the EAB Student Success Collaborative (SSC), the system that UH currently uses to inflect student outcomes across all undergraduate courses. Information from the SCC such as college, major and degree, GPA and earned credits as well as the level of risk factor will be uploaded and used in student performance prediction models. Parts of the system design will include a core system that is reusable across courses; methodologies will be developed that facilitate to augment the core prediction and self-assessment system with course-specific knowledge. One important outcome of this process is a unified representation of student background and student performance data that is suitable to be used to assess student performance across multiple courses, which is critical for the generalizability of the prototype system.

13

Determining Factors that Determine Course Drop Out and Poor Course Performance: During the course of the project data-driven, course-based student prediction models will be developed. In addition to predicting student course performance accurately, the obtained models can be used to identify factors that are associated with poor academic performance. Knowing such factors is a critical importance to select interventions that reduce the dropout rate.

Benefits to Society: This project intends to create a system with three critical areas that operate in unison: an Early Warning System that operates on the top of the Self-Assessment Platform and the Student Performance Prediction System. This will result in a more effective way to increase student retention in critical STEM courses, provide support for teaching students critical self-assessment skills for both college and their career, and support instructors’ knowledge of students’ progress so effective interventions may be implemented. Improving retention rate for undergraduate students and especially for students in STEM majors would make a major impact on society since increasing STEM retention by even a small percentage can be a cost-efficient way to contribute substantially to the supply of STEM workers. In addition, the project will increase our knowledge of how to most effectively support diverse students to succeed in STEM courses. This is critical in order to increase the diversity in STEM careers.

8 Preliminary Work

Some research on some of the proposal goals already started since late 2015, and data collection and some prototype implementation already began for 2 Computer Science courses COSC 1430 and COSC 2440. Online cross-sectional online surveys were conducted that collect student data for the two courses COSC 1430 and COSC 2440. Moreover, a skill-based self-assessment survey is currently conducted in which before each exam students are asked to self-assess their skills which are then compared with their actual exam performance. Self-assessment measures are currently implemented that provide feedback to students how well the student can self-assess his skill knowledge and self-assessment data are currently collected in the two courses; moreover, student feedback with respect to the value of self-assessment is also currently collected. Moreover, a prototype student performance prediction system was developed for the same two courses as part of a Master Thesis [17]. The prototype system predicts the final grade of students based on attendance, homework and exam performance and socio-economic student data, about two weeks before the last day to drop the course. Student data from the teaching of the same course in the previous semester were used to learn models that predict the final exam grade of students in the current semester. In our planned research we will extend the prototype system to use more knowledge sources including self-assessment data, and to provide the capability of making predictions in the presence of a lot of missing data. Moreover, we like to mention the PI of the project developed a lot of prediction and classification models in his past work [3, 4, 11]. Dr. McNeil has designed and developed multiple websites in which the content is generated through a relational database. These websites have been active for as long as 15 years and generate up to 20,000 unique web visitors a day. She has written and presented about the systems that were configured for these websites at national and international conferences.

14

9 Project Management Plan

9.1 Resource Allocation

Table 3 describes when particular research themes are investigated during the 24 months of project funding; moreover, an estimation what percentage of project resources is allocated to each of the six project tasks is given. Table 3: Resource Allocation and Timeline of the Project

9.2 Project Deliverables and Subtasks

These are the subtasks and the deliverables of the six project tasks, and the resources allocated to each task and subtask: Task 1: Self-Assessment Platform (SAP) ......................................................................................... 25%

a. Define the approach (Develop Self-Assessment Approach, Develop Self-Assessment Measures, Develop Approach how to Present Self-Assessment Results to Students, and how to Make Student Better Self Assessors) ............................................................ 8%

15

b. Design and Implementation of an Interactive Web-based Self-Assessment Platform .............................................................................................................................. 10%

c. Enhance Self-Assessment Platform based on Feedback from the Pilot Study ............ 7% Task 2: Student Performance Prediction System (SPP) ................................................................ 25%

a. Collecting Training Data for the SPP System .................................................................. 7% b. Design and Implementation of the SPP system .......................................................... 12% c. Evaluation and Enhancement of the SPP Prediction Models........................................ 6%

Task 3: Early Warning System (EWS) .............................................................................................. 15%

a. Selection of communication and feedback strategies models for EWS ....................... 5% b. Design and Implementation of EWS ................................................................................ 7% c. Enhance EWS based of Feedback from the Pilot Study ................................................. 3%

Task 4: Data Collection, Standardization, and Integration (CSI) ............................................... 12%

a. Design a Relational Database that stores all SPP-SAS Data .......................................... 4% b. Develop Software that Collects Student Performance and other Student Data and

imports it into the Relational Database ............................................................................ 6% c. Develop Software that exports Data from the Relational Database to SAP, EWS, and

SPS and imports analysis results of the three systems into the Relational Database. 2% Task 5: Pilot Study (PIL) ..................................................................................................................... 20%

a. Deploy SPSS for 2 Computer Science Courses ............................................................... 14% b. Evaluate the Early Warning System .................................................................................... 3% c. Evaluate the Self-Assessment Platform ............................................................................... 3%

Task 6: Task 6: Analysis of the Reusability of the Proposed System across Courses (RAC) ............................................................................................................................ 3%

Write white paper that summarizes the finding of this study and identifies the reusable parts of the developed system and outlines procedures that have to be taken when using SPP-SAS for a new Computer Science course

10 Results of Prior NSF Support

None

1

Project Description

“Design and Implementation of a Student Performance Prediction and Self-Assessment System

to Enhance Student Retention”

References

[1] ACT Institutional Data File, 2015. Retrieved from https://www.ruffalonl.com/documents/shared/Papers_and_Research/ACT_Data/ACT_2015.pdf

[2] ACT Institutional Data File, 2005. Retrieved from http://www.act.org/content/dam/act/unsecured/documents/retain_2005.pdf

[3] Paul K. Amalaman and Christoph F. Eick. HC-edit: A hierarchical clustering approach to data editing. In Proceedings of the International Symposium on Methodologies for Intelligent Systems (ISMIS): 160-170, 2015.

[4] Paul K. Amalaman, Christoph F. Eick, and Nouhad Rizk. Using turning point detection to obtain better regression trees. in Proceedings of the International Conference on Machine Learning and Data Mining (MLDM), New York City, New York, July 2013.

[5] H. L. Andrade, Y. Du, and K. Mycek (2010). Rubric-referenced self-assessment and middle school students' writing. Assessment in Education: Principles, Policy & Practice, 17(2): 199-214, 2010.

[6] H. L. Andrade, Y. Du, and X. Wang. Putting rubrics to the test: The effect of a model, criteria generation, and rubric-referenced self-assessment on elementary school students’ writing. Educational Measurement: Issues and Practice, 17(2): 3-13, 2008.

[7] Robert Balfanz and Joanna Fox. Early warning systems: Foundational research and lessons from the field. National Governors Association Meeting, Philadelphia, PA., 2011.

[8] G. Ben-Zadok, A. Hershkovitz and R. Nachmias. Examining online learning processes based on log files analysis: A case study. In A. Méndez-Vilas, A. Solano Martin, J.A. Mesa González, & J .Mesa González (Eds.), Research, Reflections and Innovations in Integrating ICT in Education: Proceedings of the Fifth International Conference on Multimedia and ICT in Education, Volume 2, 55-59, 2009.

[9] D. Boud. The role of self assessment in student grading. Assessment & Evaluation in Higher Education, 14(1): 20–30, 1989.

2

[10] S. M. Brookhart, M. Andolina, M. Zuza, and R. Furman, R. Minute math: An action research study of student self-assessment. Educational Studies in Mathematics, 57: 213-227, 2004.

[11] O.U. Celepcikay and C. F. Eick. REG^2: A regional regression framework for geo-referenced datasets. In Proceedings of the 17th ACM SIGSPATIAL International Conference on Advances in GIS (ACM-GIS), Seattle, Washington, November 2009.

[12] X. Chen, (2013). STEM attrition: College students’ paths into and out of STEM fields (NCES 2014-001). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. Washington, DC, 2013.

[13] R.G. Ehrenberg. Analyzing the factors that influence persistence rates in STEM field majors: Introduction to the symposium. Economics of Education Review, 29(6): 888-891, 2010.

[14] Alyce Emerson. Dropout prevention: A study of transition education programs in the Fayette County schools. University of Kentucky, ProQuest Dissertations Publishing, 2010. 3472541.

[15] M. Griffiths and C. Davies. Learning to learn: Action research from an equal opportunities perspective in a junior school. British Educational Research Journal, 19(1): pp. 43-58, 1993.

[16] S. Haag and J. Collofello. Engineering undergraduate persistence and contributing factors. Paper presented at the 38th ASEE/IEEE Annual Frontiers in Education Conference, Saratoga Springs, NY, 2008.

[17] Rohith Jidagam. Design and implementation of faculty support system to reduce course dropout rates. Master Thesis, University of Houston, May 2016.

[18] Dorina Kabakchieva. Predicting student performance by using data mining methods for classification. Cybernetics and Information Technologies, 13.1: 61-72, 2013.

[19] V. Klenowski, V. (1995). Student self-evaluation processes in student-centred teaching and learning contexts of Australia and England. Assessment in Education: Principles, Policy & Practice, 2(2): 145-163, 1995.

[20] R. Levitz. 2016 National Freshman Motivation to Complete College Report. Cedar Rapids, Iowa. Retrieved from http://www.ruffaloNL.com

[21] M. Lew, W. M. Alwis and H. G. Schmidt. Accuracy of students' self-assessment and their beliefs about its utility. Assessment & Evaluation In Higher Education, 35(2): 135-156, 2010. doi:10.1080/02602930802687737

3

[22] R. Lewine and A. A. Sommers. Unrealistic Optimism in the Pursuit of Academic Success. International Journal For The Scholarship Of Teaching & Learning, 10(2): 1-3, 2016.

[23] B. L. Lowell, H. Salzman, H. Bernstein, and E. Henderson. Steady as she goes? Three generations of students through the science and engineering pipeline. Paper presented at the Annual Meeting of the Association for Public Policy Analysis and Management, Washington, DC., November 7, 2009.

[24] Gomez Martinez. Contributions from data mining to study academic performance of students of a tertiary institute. ACM Trans. Journal of Educational Research, 2.9: 713-726, 2014.

[25] K. Mishra and S. Gupta. Mining students data for prediction performance. In Advanced Computing & Communication Technologies (ACCT), Fourth International Conference on IEEE, 255-262, 2014.

[26] G. Munns and H. Woodward. Student engagement and student self-assessment: The REAL framework. Assessment in Education: Principles, Policy and Practice, 13(2): pp. 193-213, 2006.

[27] N. Nguyen, J. Paul, and H. Peter. A comparative analysis of techniques for predicting academic performance. In Proceedings of the 37th ASEE/IEEE Frontiers in Education Conference, 7-12, 2007.

[28] K. Pittman. Comparison of data mining techniques used to predict student retention. Ph.D. Thesis, Nova Southeastern University, 2008.

[29] Philadelphia Education Fund. Early warning response system toolkit: School-Based deliberations for the dropout crisis. Philadelphia, PA., 2013.

[30] Shanmuga Priya and Senthil Kumar. Improving the student’s performance using educational data mining. International Journal of Advanced Networking and Applications 4.4: 1806, 2013.

[31] Bhaskaran Ramaswami. CHAID based performance prediction model in educational data mining. IJCSI International Journal of Computer Science Issues, 7(1): 212-217, 2010.

[32] M. Ryan. Early warning indicator systems. Denver, CO: Education Commission of the States. 2011. Retrieved from http://www.ecs.org/clearinghouse/94/36/9436.pdf

[33] D. R. Sadler. Formative assessment and the design of instructional systems. Instructional Science, 18: 119-144, 1989.

[34] P. Sadler and E. Good. The impact of self- and peer-grading on student learning. Educational Assessment, 11(1): 1-31, 2006.

4

[35] R. Sarlo. Early warning systems: Moving from reaction to prevention. University of South Florida, Florida. Retrieved October 22, 2016, from http://www.rtinetwork.org/learn/rti-in-secondary-schools/early-warning-systems-moving-from-reaction-to-prevention

[36] D. H. Schunk. Commentary on self-regulation in school contexts. Learning & Instruction, 15: 173-177, 2005. doi:10.1016/j.learninstruc.2005.04.013

[37] E. Seymour and N. M. Hewitt. Talking about leaving: Why undergraduates leave the sciences. Boulder, CO: Westview Press, 1997.

[38] Loralyn Taylor and Virginia McAleese. Beyond retention: Supporting student success, persistence and completion rates through a technology-based, campus-wide, comprehensive student support program. Retrieved from http://www2.ed.gov/documents/college-completion/beyond-retention-supporting-student-success.doc

[39] L. Towler and P. Broadfoot. Self-assessment in the primary school. Educational Review, 44(2): 137-151, 1992.

[40] K. Uekawa, S. Merola, F. Fernandez, and A. Porowski. Creating an early warning system: Predictors of dropout in Delaware. Regional Educational Laboratory Mid-Atlantic, 2010. Retrieved from http://www.doe.k12.de.us/infosuites/ddoe/p20council/docs/MA1275TAFINAL508.pdf

[41] M. Wook, Y. Yahaya, N. Wahab, M. Isa, N. Awang, and H. Seong. Predicting NDUM student’s academic performance using data mining techniques. In Proceedings of the Second International Conference on Computer and Electrical Engineering, IEEE Computer Society, 2009.

[42] S. K. Yadav, B.K. Bharadwaj and S. Pal. Data mining applications: A comparative study for predicting student’s performance. International Journal of Innovative Technology and Creative Engineering, 1(12): 13-19, 2011.

[43] B. J. Zimmerman. Becoming a self-regulated learner: An overview. Theory Into Practice, 41(2): 64-70, 2002.

Biographical Sketch—Christoph F. Eick PROFESSIONAL PREPARATION M.S., Computer Science, University of Karlsruhe (Germany), May 1979. Dr. nat., Computer Science, University of Karlsruhe (Germany), July 1984. APPOINTMENTS 1985-1992: Assistant Professor, Department of Computer Science, University of Houston. 1992-…: Associate Professor, Department of Computer Science, University of Houston. 9/2004-…: Director Data Analysis & Intelligent System Lab, University of Houston PRODUCTS 5 Products Most Closely Related to the Project 1. Paul K. Amalaman, Christoph F. Eick: HC-edit: A Hierarchical

Clustering Approach to Data Editing, in Proc. International Symposium on Methodologies for Intelligent Systems (ISMIS), 2015: 160-170.

2. F. Akdag and Christoph F. Eick: An optimized interestingness hotspot discovery framework for large gridded spatio-temporal datasets, in Proc. IEEE International Conference on Big Data, 2015: 2010-2020.

3. S. Wang and C. F. Eick, A polygon-based clustering and analysis framework for mining spatial datasets. GeoInformatica 18(3): 569-594 (2014).

4. Zhejun Cao, Sujing Wang, Germain Forestier, Anne Puissant, Christoph F. Eick: Analyzing the composition of cities using spatial clustering, in Proceedings of the 2nd ACM SIGKDD International Workshop on Urban Computing. 2013: 40-47

5. Rachsuda Jiamthapthaksin, Christoph F. Eick, Seungchan Lee: GAC-GEO: a generic agglomerative clustering framework for geo-referenced datasets. Knowl. Inf. Syst. 29(3): 597-628 (2011).

Other Significant Products (out of 160 peer-reviewed publications) 1. Fatih Akdag, Christoph F. Eick and Goning Chen: Creating Polygon

Models for Spatial Clusters, in Proc. International Symposium on Methodologies for Intelligent Systems (ISMIS), 2014: 491-499.

https://scholar.google.com/citations?view_op=view_citation&hl=en&user=qfCKBuoAAAAJ&sortby=pubdate&citation_for_view=qfCKBuoAAAAJ:XiVPGOgt02cC

https://scholar.google.com/citations?view_op=view_citation&hl=en&user=qfCKBuoAAAAJ&sortby=pubdate&citation_for_view=qfCKBuoAAAAJ:XiVPGOgt02cC

http://www.informatik.uni-trier.de/%7Eley/pers/hd/w/Wang:Sujing.html

http://www.informatik.uni-trier.de/%7Eley/db/journals/geoinformatica/geoinformatica18.html#WangE14

http://www.informatik.uni-trier.de/%7Eley/pers/hd/j/Jiamthapthaksin:Rachsuda.html

http://www.informatik.uni-trier.de/%7Eley/pers/hd/l/Lee:Seungchan.html

http://www.informatik.uni-trier.de/%7Eley/db/journals/kais/kais29.html#JiamthapthaksinEL11

2. W. Ding, C. F. Eick, X. Yuan, J. Wang, J.-P. Nicot, A framework for regional

association rule mining and scoping in spatial datasets. GeoInformatica 15(1): 1-28 (2011).

3. Chun-Sheng Chen, Nauful Shaikh, Panitee Charoenrattanaruk, Christoph F. Eick, Nouhad J. Rizk, Edgar Gabriel: Design and Evaluation of a Parallel Execution Framework for the CLEVER Clustering Algorithm, in Proc. PARCO 2011: 73-80

4. C. F. Eick, B. Vaezian, D. Jiang, and J. Wang, Discovery of Interesting Regions in Spatial Data Sets Using Supervised Clustering. PKDD 2006: 127-138.

5. Christoph F. Eick, Nidal M. Zeidat, Zhenghong Zhao: Supervised Clustering - Algorithms and Benefits. ICTAI 2004: 774-776

SYNERGETIC ACTIVITIES 1. Served on the program committee for the IEEE International Conference for

Data Mining (2006-2013, 2016), International Conference on Machine Learning and Data Mining (MLDM, 2005-2016), and Symposium on Methodologies for Intelligent Systems (ISMIS, 2014-2016).

2. Developed a new course “Data Mining” for the Computer Science Curriculum at the University of Houston.

3. Director of the UH Data Analysis and Intelligent System Lab (UH-DAIS) 4. Served as the Director of Graduate Studies 2001-2004, during my tenure, the

PhD program grew from 20 students in 2001 to more than 60 students at the beginning of 2004.

5. Served as the Associate Chairman for the Department of Computer Science 2011-2014.

6. Received the Faculty Research Engagement Grant from Yahoo! Labs, California for a project, “Regional Knowledge Discovery for Geo-targeting” in August 2010.

7. Served as the Dissertation advisor for 13 PhD students and as the Master Thesis advisor for more than 80 M.S. students.

http://www.informatik.uni-trier.de/%7Eley/pers/hd/d/Ding_0003:Wei.html

http://www.informatik.uni-trier.de/%7Eley/pers/hd/y/Yuan:Xiaojing.html

http://www.informatik.uni-trier.de/%7Eley/pers/hd/w/Wang_0007:Jing.html

http://www.informatik.uni-trier.de/%7Eley/pers/hd/n/Nicot:Jean=Philippe.html

http://www.informatik.uni-trier.de/%7Eley/db/journals/geoinformatica/geoinformatica15.html#DingEYWN11

http://www.informatik.uni-trier.de/%7Eley/pers/hd/v/Vaezian:Banafsheh.html

http://www.informatik.uni-trier.de/%7Eley/pers/hd/j/Jiang:Dan.html

http://www.informatik.uni-trier.de/%7Eley/pers/hd/w/Wang_0007:Jing.html

http://www.informatik.uni-trier.de/%7Eley/db/conf/pkdd/pkdd2006.html#EickVJW06

Biographical Sketch - Sara G. McNeil

PROFESSIONAL PREPARATION

University of Georgia, Athens, GA Art B.F.A., 1969University of Tennessee Knoxville, TN Education M.S., 1992University of Tennessee Knoxville, TN Education - Emphasis in Technology Ed.D., 1995

APPOINTMENTS

2001-present Associate Professor, Department of Curriculum and Instruction, University of Houston1995-2001 Assistant Professor, Department of Curriculum and Instruction, University of Houston

PUBLICATIONS

Five Publications Most Closely Related to the Proposed Project

1. Shireen, W., & McNeil, S. (2008). AC 2008-1227: A modern dsp-based laboratory for power electronics education. American Society of Engineering Education, 13, pp. 13.62.1-13.62.7.

2. McNeil, S. (2015). Visualizing mental models: Understanding cognitive change to support teaching and learning of multimedia design and development. Educational Technology Research and Development, 63(1), 73-96.

3. McNeil, S., & Robin, B. (2015). A student-centered collaborative design model for the development of MOOCs. In J. Corbeil, M. Valdes-Corbeil, and B. Khan (Eds.), MOOC Case Book: Case Studies in MOOC Design, Development and Implementation. Linus Publishers: Ronkonkoma, NY. pp 295-310.

4. McNeil, S., Mintz, S., & White, C. (2002). The design and development of an interactive web site for teaching and learning about American history. In D. Willis, J. Price, & J. Willis (Eds.), Technology and Teacher Education Annual 2002. Charlottesville, VA: Association for the Advancement of Computing in Education.

5. McNeil, S., & Chernish, W. (2001). Collaborative approach to multimedia courseware design and development. In G. Williams, W. Chernish, & B. McKercher (Eds.) The Internet and Travel and Tourism Education (pp.107-123). New York: The Haworth Press.

Other Significant Publications

1. Robin, B., McNeil, S., Cook, D., Agarwal, K., & Singhal, G. (April, 2011). Preparing for the changing role of instructional technologies in medical education. Academic Medicine, 86(4), 435-439.

2. McNeil, S. (2015). 21st Century Technology Skills. In The SAGE Encyclopedia of Educational Technology. J. Michael Spector, Editor.

3. Robin, B., & McNeil, S. (2015). The collaborative design and development of MOOCs for teacher professional development. In C. Bonk, M. Lee, T. Reeves, and T. Reynolds (Eds.), MOOCs and Open Education Around the World (Chapter 16). New York: Routledge.

4. Robin, B, & McNeil, S. (2014). Webscapes: An academic vision for digital humanities projects on the web. Book 2.0, 4(1+2), 121–141.

5. White, C., McNeil, S., Burke, V., Liss, N., & McCormack, S. (2004). PATH (Project for the Active Teaching of History): A history education community partnership. Journal of College Teaching & Learning. Western Academic Press.

SYNERGISTIC ACTIVITIES

1. Designed and co-developed Digital History, a multimedia history resourcehttp://www.digitalhistory.uh.eduThe Digital History website was developed through partnerships with the Chicago Historical Society, the Gilder Lehrman Institute of American History in New York, the National Park Service and other cultural institutions, and provides students and teachers with high quality historical resources for free. Named to the NEH’s Edsitement list of exemplary resources in the humanities, Digital History is also listed as one of the Top Five resources in U.S. History by Best of History Websites.

2. PI and/or Co-PI on seven grants from the U.S. Department of Education totaling over $7 million and a curriculum materials grant from the National Endowment of the Humanities for $200,000.

3. Designed, developed and taught two Massive Open Online Courses (MOOCs) for a global audience (over 20,000 total students) last year through Coursera.

4. Received academic and professional honors and awards for teaching.University of Houston Distinguished Leadership in Teaching Award (one award given per year)University of Houston Teaching Excellence AwardUniversity of Houston Distance Education Teaching Excellence AwardCollege of Education Faculty Teaching Excellence AwardOutstanding Faculty member in a Distance Education program, (national teaching award) University Continuing Education Association

5. Editorial ConsultantPast Managing Editor, Technology, Instruction, Cognition and Learning JournalMember of Journal of Engineering Systems Simulators Editorial Board.Member of IJEL (International Journal on E-Learning) Editorial Board. Charlottesville, VA: Association for the Advancement of Computing in Education.Member of the Journal of Technology and Teacher Education Editorial Board. Charlottesville, VA: Association for the Advancement of Computing in Education.Member of the WebNet Journal: Internet Technologies, Applications & Issues Editorial Board.Charlottesville, VA: Association for the Advancement of Computing in Education

1

Biographical Sketch – Nouhad J. Rizk

PROFESSIONAL PREPARATION BS, 1979–1981 Ecole Polytechnique Supérieure d'Informatique Liban (E.P.S.I.L.), Lebanon Computer Science MS, 1981–1983 Ecole Polytechnique Supérieure d'Informatique Liban (E.P.S.I.L.), Lebanon Computer Science DESS, 1983–1984 Université de Nancy, Lorraine, France Computer Science Ph.D., 2002–2007 Leicester University, UK Education Leadership & Management/Computer Science

APPOINTMENTS 2007–Present Instructional Assistant Professor, Department of Computer Science,

University of Houston 1994–2006 Senior Lecturer/Sr. Project Supervisor, Department of Computer Science,

University of Notre Dame 1990–1993 Instructor and trainer, Department of Computer Science, Annunciation

Technical School, Lebanon 1989–1990 IT Consultant/Project Leader, Department of Computer Science, International

Technical Institute, Lebanon 1985–1994 Chief Information Officer, E.A.G.L.E Enterprise

PRODUCTS 5 Products Most Closely Related to the Project 1. R. Jidagam, N. Rizk, Evaluation of Predictive Data Mining Algorithms in Student Academic

Performance, INTED Proceedings, 2016.

2. R. Jidagam, N. Rizk, FSS: A Faculty Support System for Student Academic Performance Analysis, ICERI Proceedings, pp.852-861, 2015

3. D.J. Kim, H. Al-Mubaid, K-B Yue, and N. Rizk, From Expectation to Actual Perception after Experience: A Longitudinal Study of the Perceptions of Student Response System, in the proceedings of Americas Conference on Information Systems (AMCIS 2011), pp 401. Detroit, USA, 2011.

4. N. Rizk, N. Shaikh, The Usage of A Hybrid Course to Enhance Student Engagement and Success, in the proceedings of “The International conference on E-learning on the Workplace” (ICELW2011), New York, USA, 2011.

5. N.Rizk, P. DeCarlo, Z. Mughal, The overview and evaluation of an expert system for delivering dynamic curriculum assessments in college institutions, in the proceedings of “The International Conference of Education, Research and Innovation” ICERI 2010, Madrid- Spain, 2010.

2

Other Significant Products 1. P. K. Amalaman, C.F. Eick, N. J. Rizk, Using Turning Point Detection to Obtain Better

Regression Trees. MLDM 2013: pp. 325-339

2. C.-S. Chen, N. Shaikh, P. Charoenrattanaruk, C.F. Eick, N. Rizk, and E. Gabriel, Design and Evaluation of a Parallel Execution Framework for the CLEVER Clustering Algorithm, presented at Parallel Computing Conference 2011 (ParCo), acceptance rate: 31%, Ghent, Belgium, September 2011; published in K. De Bosschere et al. (Eds.), Application and Tools and Techniques on the Road to Exascale Computing, IOS Press, pp. 73-80, May 2012

3. C. Chen, C. F. Eick, N. Rizk “Mining Spatial Trajectories Using Non-parametric Density Functions”, in the Proceedings of International Conference on Machine Learning and Data Mining (MLDM 2011), pp.496-510. New York, USA, 2011. http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/r/Rizk:Nouhad_J=.html

4. N. Rizk, Intelligent Parallel and Evolutionary Approaches to Computational Biology, in the book “Parallel Computing for Bioinformatics” by Albert Zomaya Wiley and Sons Publishers. New Jersey, 07030 5774

5. N. Rizk, 3D-E-learning model: a new model for integrating ICT in higher education curricula, in the proceedings of “Society for Information Technology and teacher Education” (SITE 2007), San Antonio, USA, 2007.

SYNERGETIC ACTIVITIES 1. Serving on the Computer Science Search Committee for new faculty.

2. Serving as ex-officio on Computer Science Undergraduate Committee. Serving on College of Natural Sciences and Mathematics (NSM) Undergraduate Curriculum Committee. Serving as the Director of Undergraduate Studies in the Department of Computer Science.

3. Served as the Chair of the Computer Science Undergraduate Committee 2012-2013. Served as the Chair of NSM Grievance 2016.

4. Received Teacher Excellence Award from The Computer Science Department at the University of Houston, 2012.

5. Served as thesis advisor for more than 27 M.S. students.

SUMMARYPROPOSAL BUDGET

FundsRequested By

proposer

Fundsgranted by NSF

(if different)

Date Checked Date Of Rate Sheet Initials - ORG

NSF FundedPerson-months

fm1030rs-07

FOR NSF USE ONLYORGANIZATION PROPOSAL NO. DURATION (months)

Proposed Granted

PRINCIPAL INVESTIGATOR / PROJECT DIRECTOR AWARD NO.

A. SENIOR PERSONNEL: PI/PD, Co-PI’s, Faculty and Other Senior Associates (List each separately with title, A.7. show number in brackets) CAL ACAD SUMR

1.

2.

3.

4.

5.

6. ( ) OTHERS (LIST INDIVIDUALLY ON BUDGET JUSTIFICATION PAGE)

7. ( ) TOTAL SENIOR PERSONNEL (1 - 6)

B. OTHER PERSONNEL (SHOW NUMBERS IN BRACKETS)

1. ( ) POST DOCTORAL SCHOLARS

2. ( ) OTHER PROFESSIONALS (TECHNICIAN, PROGRAMMER, ETC.)

3. ( ) GRADUATE STUDENTS

4. ( ) UNDERGRADUATE STUDENTS

5. ( ) SECRETARIAL - CLERICAL (IF CHARGED DIRECTLY)

6. ( ) OTHER

TOTAL SALARIES AND WAGES (A + B)

C. FRINGE BENEFITS (IF CHARGED AS DIRECT COSTS)

TOTAL SALARIES, WAGES AND FRINGE BENEFITS (A + B + C)

D. EQUIPMENT (LIST ITEM AND DOLLAR AMOUNT FOR EACH ITEM EXCEEDING $5,000.)

TOTAL EQUIPMENT

E. TRAVEL 1. DOMESTIC (INCL. U.S. POSSESSIONS)

2. FOREIGN

F. PARTICIPANT SUPPORT COSTS

1. STIPENDS $

2. TRAVEL

3. SUBSISTENCE

4. OTHER

TOTAL NUMBER OF PARTICIPANTS ( ) TOTAL PARTICIPANT COSTS

G. OTHER DIRECT COSTS

1. MATERIALS AND SUPPLIES

2. PUBLICATION COSTS/DOCUMENTATION/DISSEMINATION

3. CONSULTANT SERVICES

4. COMPUTER SERVICES

5. SUBAWARDS

6. OTHER

TOTAL OTHER DIRECT COSTS

H. TOTAL DIRECT COSTS (A THROUGH G)

I. INDIRECT COSTS (F&A)(SPECIFY RATE AND BASE)

TOTAL INDIRECT COSTS (F&A)

J. TOTAL DIRECT AND INDIRECT COSTS (H + I)

K. SMALL BUSINESS FEE

L. AMOUNT OF THIS REQUEST (J) OR (J MINUS K)

M. COST SHARING PROPOSED LEVEL $ AGREED LEVEL IF DIFFERENT $

PI/PD NAME FOR NSF USE ONLYINDIRECT COST RATE VERIFICATION

ORG. REP. NAME*

*ELECTRONIC SIGNATURES REQUIRED FOR REVISED BUDGET

1YEAR

1


Christoph

ChristophChristoph

Eick

Eick Eick

ChristophChristophChristoph F F F Eick Eick Eick - PI 0.00 0.00 1.00 12,324Sara G McNeil - Co-PI 0.00 0.00 1.00 9,163Nouhad Rizk - Co-PI 0.00 0.00 1.00 8,577

0 0.00 0.00 0.00 03 0.00 0.00 3.00 30,064

0 0.00 0.00 0.00 00 0.00 0.00 0.00 02 45,1501 7,2000 00 0

82,4147,936

90,350

07,200

0

0000

0 0

2,50000000

2,500 100,050

50,525MTDC on Campus (Rate: 50.5000, Base: 100050)

150,5750

150,5750


FundsRequested By

proposer

Fundsgranted by NSF

(if different)



fm1030rs-07


Proposed Granted



1.

2.

3.

4.

5.









6. ( ) OTHER





TOTAL EQUIPMENT


2. FOREIGN


1. STIPENDS $

2. TRAVEL

3. SUBSISTENCE

4. OTHER







5. SUBAWARDS

6. OTHER










ORG. REP. NAME*


2YEAR

2


Christoph

ChristophChristoph

Eick

Eick Eick


0 0.00 0.00 0.00 03 0.00 0.00 3.00 30,104

0 0.00 0.00 0.00 00 0.00 0.00 0.00 02 45,1501 8,0000 00 0

83,2547,951

91,205

07,200

0

0000

0 0

80000000

800 99,205

50,099MTDC on Campus (Rate: 50.5000, Base: 99205)

149,3040

149,3040


FundsRequested By

proposer

Fundsgranted by NSF

(if different)



fm1030rs-07


Proposed Granted



1.

2.

3.

4.

5.









6. ( ) OTHER





TOTAL EQUIPMENT


2. FOREIGN


1. STIPENDS $

2. TRAVEL

3. SUBSISTENCE

4. OTHER







5. SUBAWARDS

6. OTHER










ORG. REP. NAME*


Cumulative

C


Christoph

ChristophChristoph

Eick

Eick Eick


0.00 0.00 0.00 03 0.00 0.00 6.00 60,168

0 0.00 0.00 0.00 00 0.00 0.00 0.00 04 90,3002 15,2000 00 0

165,66815,887

181,555

014,400

0

0000

0 0

3,30000000

3,300 199,255

100,624

299,8790

299,8790

1

University of Houston BUDGET JUSTIFICATION

A. Senior Personnel

Christoph Eick, Principal Investigator: 1.0 summer month/year, for which salary is requested. Dr. Eick is an Associate Professor in the Department of Computer Science. Dr. Eick also serves as the Director for the University of Houston Data Analysis and Intelligent Systems Lab. His specific expertise is in data mining, data analysis, artificial intelligence and geographical information systems He will supervise the project and will lead the research and development associated with project tasks 2, 4, and 6. He will also supervise three graduate research assistants. Additionally, Dr. Eick will direct the analysis and dissemination of the Project results.

Sara McNeil, Co-Principal Investigator: 1.0 summer month/year, for which salary is requested. Dr. McNeil is an Associate Professor in the Department of Curriculum & Instruction in the College of Education. She is an expert in curriculum development and the instructional design, development and evaluation of technology-enhanced learning environments. She will lead research and development for project Task 1 and jointly with Dr. Rizk for Task 5; moreover, she will be responsible for evaluating the benefits of the Self-Assessment Platform and the Early Warning System that is developed during the course of the project.

Nouhad Rizk, Co-Principal Investigator: 1.0 summer month/year, for which salary is requested. Dr. Rizk is an Instructional Associate Professor in the Department of Computer Science. She is an expert in information retrieval and educational data mining; moreover, she currently serves as the Director of Undergraduate Studies in the Department of Computer Science. She will lead research and development for project Task 3 and jointly with Dr. McNeil for Task 5

B. Other personnel

TBN Graduate Student 1: 9.0 calendar months/year, for which $2,150 in monthly support is requested. His/her research will focus on Project Tasks 1, 3 and 5.

TBN Graduate Student 2: 12.0 calendar months/year, for which $2,150 in monthly support is requested. His/her research will focus on Project Tasks 2, 4, and 5.

TBN Undergraduate Student: 9.0 and 10.0 calendar months for years 1 and 2, for which $800 in monthly support is requested. The objective of his/her research will focus on Project Tasks 3, 4, and 5

C. Fringe Benefits

The Department of Health and Human Service (DHHS) has given its approval to budget fringe at actual cost rather than a percentage of the salary. A fringe benefits calculator has been developed as a tool to assist in calculating the fringe benefits for sponsored research budgets. The calculator can be found at the University of Houston’s Division of Research Website. Fringe benefits are calculated as follows for Year 1: Dr. Eick ($2,237), Dr. McNeil ($1,507), Dr. Rizk ($1,411), Grad Students ($129/position/month), and Undergraduate Student ($72).

D. Equipment

None requested.

2

E. Travel

Domestic – Funds ($7,200/year) are requested to support the cost of travel to important, scientific conferences, such as KDD, ICDM, and AERA, where the results from this research can be presented. Funds will be used to support four trips per year. The PI as well as the Co-PI as well as the supported graduate students may attend these domestic conferences. The money will be used to pay for airfare and other transportation costs, per diem, lodging, conference registration and various incidental costs incurred. Average costs for travel from Houston, TX are as follows: $500 transportation + $400 registration + $284 per diem + $600 lodging + $16 incidentals = ~$1,800/trip.

F. Participant Support

None requested.

G. Other Direct Costs

Materials and Supplies The amount requested for this project ($2,500 in year 1 and $800 in year 2) will be dedicated to related software licenses, data collection materials and miscellaneous research supplies. Moreover, two lab tops will be acquired that will be used by the two graduate students supported in the project.

H. Total Direct Costs

Total direct costs for each year are as follows: Year 1: $100,050 Year 2: $99,205

I. Indirect Costs

Indirect costs are budgeted in accordance with the University of Houston’s current federally negotiated rate agreement, dated 1/12/2016. The University of Houston F&A rate of 50.5 for FY 2017 is applied to modified total direct costs. Modified total direct costs do not include equipment, participant costs, and subcontracts over $25,000.

J. Total Direct and Indirect Costs

Total Costs for this proposal are as follows: Year 1: $150,575 Year 2: $149,304

Current and Pending Support: [Christoph F. Eick] Current: None. Pending: Project Title: Design and Implementation of a Student Performance Prediction and Self-Assessment

System to Enhance Student Retention (this proposal) Source of Support: NSF Total Award Amount: $299,879 Total Award Period: 06/01/2017 – 05/31/2019 Location of Project: University of Houston Month/Yr Committed: [x] Cal [x] Acad [1.00] Sumr Project Title: The Port of the Future is a Smart Port Source of Support: Department of Homeland Security Total Award Amount: $500,000 Total Award Period: 01/02/2017 – 01/01/2020 Location of Project: University of Houston Month/Yr Committed: [x] Cal [x] Acad [0.60] Sumr

Current and Pending Support: [Sara G. McNeil] Current: None. Pending: Project Title: Design and Implementation of a Student Performance Prediction and Self-Assessment

System to Enhance Student Retention (this proposal) Source of Support: NSF Total Award Amount: $299,879 Total Award Period: 06/01/2017 – 05/31/2019 Location of Project: University of Houston Month/Yr Committed: [x] Cal [x] Acad [1.00] Sumr

Current and Pending Support: [Nouhad J. Rizk] Current: None. Pending: Project Title: Design and Implementation of a Student Performance Prediction and Self-Assessment

System to Enhance Student Retention (this proposal) Source of Support: NSF Total Award Amount: $299,879 Total Award Period: 06/01/2017 – 05/31/2019 Location of Project: University of Houston Month/Yr Committed: [x] Cal [x] Acad [1.00] Sumr

Facilities, Equipment, and Other Resources

UH Data Analysis and Intelligent Systems Lab: The Lab of Dr. Eick’s research group has five desks, and will be used to house the three students that are supported by the NSF grant. Offices: Moreover, Dr. Eick has an office near the lab and the two Co-PIs have offices as well. Computer: Dr. Eick’s Lab houses two high performance desktop computers; moreover, the IT services of the Department of Computer Science and the College of Education are available for the participants of the NSF project, which will be describe next.

Major IT Services of the Department Computer Science

• Lab Access for Students and Faculty - Both Unix and Windows. This also includes support for roaming profiles and persistent disk storage for long-term data/code storage.

• Research systems installation and support for all CS faculty and research teams • E-mail accounts for students, staff and faculty (IMAP, POP, Webmail and SMTPS)

• DNS server for name resolution in the CS subnetwork • DHCP services for easy use of faculty and students' desktops and laptops • Subversion repositories for code revision control for research groups • Database systems for teaching and research work • Web page support for all department students and faculty

• Printing and quota support from all the CS systems • Remote login and SSH access to programming platform • Instructional platform for coding assignments • Shared memory systems, Distributed memory clusters and multiple Terabyte-scale storage

systems to enable research computations and data store

• Video conferencing system for world-wide communication and research meetings • Firewalls to filter and securely allow access to CS intranet and Internet • Managing course software and features for teaching support • Software distribution and media creation via Microsoft MSDNAA software alliance for

students

• Auxiliary systems to support the departmental activities such as faculty search, graduate applications and others

Summary of CS Hardware and Software

CS-IT considers energy efficiency highly and hence we have moved all our research servers out of PGH and housed them at the Research Computing Center (RCC). This has helped us reduce the energy costs at PGH as a result of CS activities. A summary of the size of our hardware infrastructure is as below:

IT servers 40 Student lab computers 125 Staff and Faculty PC 50

Research Clusters 20 Storage arrays 10 (~60 Terabytes)

Database systems 5 Layer 2 and 3 switches 30

Cisco 7700 1 Port expanders 15

Approximate total CPU count 1200

Major specialized software for CS research and teaching

MATLAB, Windriver Tornado, Pathscale Compilers, Sun Studio Compilers, CURRY Medical imaging software, Grid computing software (e.g., Globus), Cloud Computing software (e.g., Hadoop), Subversion for code revision control, Rapid term rewriting software, ZFS for filesystems, Parallel programming tools (MPI, OpenMPI, MPICH), Shared memory software (OpenMP), Visual numerics software, BOINC for volunteer computations, License Manager, Squirrelmail, SiCortex compilers, OpenSSL for certificate authority, MySQL, SQL Server, ORACLE, MSDNAA, PhotoShop, In-design, Maya design software

College of Education Resources

For faculty, staff and students in the College of Education (COE) the following resources are available:

• Computing and multimedia environments; • Computer and Equipment support in five dedicated labs; • Web services including server space; • Hardware/software recommendations, solutions, upkeep, and repair; • Consultation and support for Instructional Design; • Training on academic software and technology; • Core and non-core technology application programs and ad-hoc programs; • Administer the local area network for the college, including the installation,

configuration, maintenance and upgrading of the LAN; • Insure compliance with university and division policies and procedures regarding

network/computing acquisitions; and

• Collaborate with administration and campus leaders to implement information technology support services.

Specifications of COE Computing Environments:

Location Purpose Computers Available Room 300N General Purpose Open Lab 6 PC computers Room 313 General Purpose Open Lab 16 PC computers Room 324 General Purpose Open Lab 40 PC computers Room 326 Statistics and Instructional Classroom 26 PC computers Room 327 Instructional Classroom 21 PC computers Room 328 Statistics and Instructional Classroom 21 PC computers Room 416 Statistics and Instructional Classroom 15 PC computers Laboratory for Innovative Technology in Education 12 PC computers Faculty and students also have access to hardware and software that they may check out. This equipment includes headphones, microphones, scanner, camcorders, digital cameras, voice recorders and tripods. The Laboratory for Innovative Technology in Education (LITE) will provide space for this project for faculty and doctoral students. http://lite.coe.uh.edu Other Resources: none

1

Data Management Plan Title of the Project: Design and Implementation of a Student Performance Prediction and Self-Assessment System to Enhance Student Retention Principal Investigators: Christoph Eick (PI), Sara McNeil (Co-PI) and Nouhad Rizk (Co-PI) I. Types of Data: This project will generate data about students enrolled in computer

science courses. This data will include demographic data and course specific data from assignments, group work, homework and exams.

II. Data and Metadata Standards: The file formats that this project will generate are as follows: .txt .docx .xlsx .crd .dcd .xyz. The use of the Dublin Core metadata will fit with the types of data to be collected because it includes metadata fields important to the researcher and future users. Metadata will be captured and created at the time of initial data creation by Matlab, Xmgrace, Origin, and Excel. Sample scripts will be formatted in an open source code in Perl and C that will allow for ease of transformation if needed. Documentation of data will be written in the Dublin Core metadata standard which is standard for digital objects. The following fields will be kept: TITLE, CREATOR, SUBJECT, FUNDERS, RIGHTS, DATES, SOURCES, LIST OF FILE NAMES, FILE FORMATS, VERSIONS, CHECKSUMS, METHODOLOGY, DATA PROCESSING FILE STRUCTURE, VARIABLE LIST, and CODE LISTS.

III. Policies for access/sharing and provisions for appropriate protection/privacy: The reports generated by this project will be made available through the University of Houston Libraries Institutional Repository (UHLIR) and a web address maintained by the PIs. The original data and metadata produced by the project will be made anonymous to protect the anonymity of the students. De-identified data will be made available immediately following the grant end date or be made available upon the PIs’ consent, whichever comes first. Others can gain access to data through a web address maintained by the PIs. There is no fee for this access. The PIs will retain the right to use the data before opening it up to a wider audience. These data and metadata are crucial for the development of students’ thesis. The development of any systems or software through this proposal is covered by copyright. No action need be taken to license the dataset. This project has data that will describe individuals, and we will obtain IRB approval before the research is conducted. All of the data will be de-identified. The IRB Protocol requires that personal data not be included in final summary of the project.

IV. Polices and provisions for re-use and re-distribution: Permission restrictions do not need to be placed on the data. Students and scholars and practitioners in democracy-promoting organizations will use the information in this database. Upon the completion of the project development, we present our findings at education conferences. The system plans may be shared with interested researchers.

2

V. Plans for archiving and preservation of access: The PI will be responsible for maintaining the data, applying proposed metadata, uploading the data with associated metadata. The members of the PI’s group will deposit data into PI’s group cluster regularly maintained and backed up by the University IT staff who works in the shift of 24 hours and 7 days a week year round. Metadata will be archived on external hard drives and DVD by the PI and the students in her group. Another copy of original data executed on XSEDE computing resources will be archived by the students in her group at the High Performance Storage System (HPSS) of the National Institute of Computational Sciences (NICS) through Kraken. The PI will be responsible for maintaining the data, applying proposed metadata, uploading the data with associated metadata. The PI plans to retain the original data and metadata until three years after the end of the fund cycle. The original data and metadata will be archived on the group disk immediately after the manuscript is published. Each member in her group will regularly save another copy of metadata to external hard drives and DVD at least once a month. Results will be published in peer-reviewed journals. We will be generating metadata, references, reports, and research papers. The data will be retained for a minimum of 3 years.

October 31, 2016 Dear NSF Review Committee Members, I write with my support of the NSF proposal entitled, “Design and Implementation of a Student Performance Prediction and Self-Assessment System to Enhance Student Retention.” The success of our undergraduate students is a top priority for the University of Houston. Recent self-study efforts conducted through the Foundations of Excellence initiative have highlighted opportunities to systematically allocate staff and resources. We broadly consider the student success recipe to include multiple factors—student knowledge and performance, faculty strength and informed participation, and advisor support and access to data. Among a number of tools that we are adopting to systematically support student success, the University of Houston has implemented EAB’s Student Success Collaborative (SSC). The SSC tool imports relevant student data from our PeopleSoft student management system and operates predictive analytics in order to situate student performance in historical performance data. The university is already making use of this functionality in three key ways: 1) to help advisors more precisely recommend student paths to success, such as timing of course enrollment; 2) to put advanced data query tools in the hands of advisors so that they can customize student messaging based on specific performance profiles, rather than relying on blanket emails; and 3) to inform faculty, departments, and colleges about strategic programmatic and scheduling decisions. Existing student support resources on campus are being re-envisioned to exemplify a proactive approach to student success. One example is our student tutoring center—a “hidden gem” often overlooked by students was renamed LAUNCH to reflect a positive and upward trajectory in student progress. Though logical, the idea that good students seek out ways to strengthen themselves as a natural course as opposed to only when they are struggling can be a tough sell—we find that students simply lack the judgment about when and how they should seek academic assistance. When asked, they tend to see tutoring and academic counseling as what other students do. So, to direct students-in-need to help resources, in the last year we have implemented the Academic Success Referral (ASR) system, through which instructors can click a button next to a student’s name on the grading roster to refer a student for consultation and recommendation for additional resources. In its pilot semester, the quite limited pilot project did lead to improved student performance for students who were referred, responded to the request for consultation, and followed through with the process. However, we found that even these students who were interested enough in their own progress to heed this red flag warning were largely and surprisingly unaware of their own performance and ways they might help it.

As the University of Houston actively pursues ways to maximize our efforts for the success of our students, it seems that the “sweet spot” toward which we aim is the intersection of course-level data with engaged students and informed staff and faculty. With that in mind, the Undergraduate Student Success Center enthusiastically supports any college-level efforts to use tools to inform students and staff. The present proposal for a Student Performance Prediction and Self-Assessment System is a worthy example. The design calls for both student and instructor dashboard access to real course data in a way that students do not currently have available. For students, the specific recommendations for next steps will lead to successful course completion. For course instructors, immediate feedback on student progress will serve as a form of just-in-time professional development. As head of the University of Houston Undergraduate Student Success Center, as well as primary SSC implementation leader on campus, I have been asked and have agreed to serve on the advisory board for this project to ensure a seamless connection between the two systems. I would be happy to assist with additional information and can be reached by email at [email protected] or by phone at 713-743-4961. My best regards,

Melissa E. Pierson, Ph.D. Assistant Vice Provost, Undergraduate Student Success University of Houston

Arnon Hershkovitz, Ph.D.

School of Education

Faculty of Humanities

Tel Aviv University, ISRAEL

October 30, 2016

To whom it may concern,

I am writing this letter in support of the grant proposal submitted to the NSF by Dr. Sara

McNeil of the University of Houston and her colleagues, titled "Design and Implementation of

a Student Performance Prediction and Self-Assessment System to Enhance Student Retention".

In recent years, there has been a tremendous progress in the way at-risk students are being

handled, and new intervention programs have been suggested. Yet, Dr. McNeil and her

colleagues suggest studying this phenomenon from a new, fascinating point of view. Their

proposal of using data-driven self-assessment tools for students to reflect upon their own

learning path brings back the students as main actors in the attempts to attack the

failing/dropping problem. By doing so, not only grades will improve, but also – and most

importantly – students' self-motivation and self-esteem will.

I am very glad that I am given the opportunity to assist with my expertise in educational data

mining and learning analytics to this novel, important and timely proposal. Therefore, I will

gladly serve on this project Advisory Board.

Yours,

Arnon Hershkovitz, Ph.D.

School of Education

Tel Aviv University, Israel