There are three components to our assessment; one is the quantitative assessment, two is the qualitative assessment of the program, and three is the qualitative evaluation of the program. Our discussions started with the subject of the quantitative assessment since this would answer the overarching question of, “does LEAP have an impact on students’ outcomes academically?”, which is really what we want to answer. This question, however is a poorly formed research question, so we must also ask, “How does LEAP have an impact on students’’ academic outcomes?” which requires the addition of a qualitative piece to the assessment. Additionally LEAP already conducts an end of summer program evaluation with each cohort completing LEAP to get the students’ feedback on the program and their immediate experience.
For the quantitative assessment we are looking at students who have participated in LEAP in the past ten years, or from the 2002-2012 entering cohorts. Although LEAP was established in 1996, record keeping in terms of students was incomplete in the early years of the program. Therefore, along with the sheer amount of information and data, it is only financially feasible to gather and process data for the last ten years. We are looking at three main data points; six year graduation rates, cumulative GPA, and first semester GPA. With this sample we will be able to collect and compare six year graduation rates and cumulative GPA’s for five full cohorts of students. The data from the five remaining cohorts will only be used in comparisons of first semester GPA. We are also taking into account other factors such as race, gender, in-state our out-of-state status, high school GPA, SAT/ACT scores, major/college, and first generation status, which will all in part help control for the many factors that could also influence a student’s academic outcomes in college.
The qualitative piece will be from students who are currently serving as LEAP mentors and previously completed LEAP. This is of course a skewed sample because students who completed LEAP and later came back to be a mentor will have had an exceptional experience and feel that going through LEAP had a positive influence on their education. This is why it was important for us to use this, and be looking at both the quantitative and qualitative pieces. Together they can form a whole picture, answering both if LEAP has an impact (quantitative) and how (qualitative).
The final piece of assessment is more evaluation but also a valuable of any programming that happens. Students complete, with supervision of LEAP staff but still anonymously, an evaluation of the program and their experience in it. This has been a consistent evaluation over the years with few changes. However, one of the things we have been working on is a pre-test type evaluation for students to complete before beginning with LEAP. While one was administered last year, Talia decided it needed improved upon before being permanently put in place, which I fully agreed with. Next summer it will be given to students, but not for use by the office. Their results and feedback will be the final step in ensuring that the evaluation is appropriate and will be consistent.