Education & Internships >> Browse Articles >> Education News


Assessing Student Learning: A Work in Progress

Assessing Student Learning: A Work in Progress

Richard Ekman and Stephen Pelletier, Change Magazine

December 02, 2009

3) Share your results. A willingness to share CLA results widely opens up important conversations. “We report both NSSE and CLA data at our opening fall workshop, so everybody hears it, not just the faculty—including, by the way, our student leaders,” Seton Hill’s Gawelek says. The university has also shared test results with the educational policy committee of its board. Trustees, Gawelek says, have asked penetrating questions about how the university uses the test results. Cabrini College’s leaders made a point of discussing the assessments with the president’s cabinet, the enrollment management committee, staff members in financial aid, and others. Talking about the test results, Dean McCormick says, opened a way to engage the whole campus, including campus staff other than faculty members, in conversations about what they could do in their areas to help improve student performance. Barton College, which has reported CLA results even to prospective students and their parents, is relying on the hard evidence the test provides to continue pressing for improvements in student writing and critical-thinking skills.

Widespread reporting of results can affect campus culture. Through such activities as strengthened program review, better institutional effectiveness procedures, participation in the NSSE survey, and hard work by faculty committees, Texas Lutheran University is creating what provost and executive vice president John Masterson calls “a real culture of assessment and continuous improvement” that is more clearly focused on improving the quality and structure of student educational experiences. “Assessment is not just something you do for the accrediting bodies,” says Masterson. At Barton too, the CLA has helped create a “culture of evidence” that goes well beyond the particulars of the instrument.

Practical Challenges
Learning how to use CLA results to influence other aspects of educational effectiveness is, frankly, still a work in progress. The institutions that are wrestling with this challenge face many obstacles—some minor, some significant. At Allegheny College, for example, initial CLA results showed that student performance was not at the level expected, nor was it in alignment with more positive results from other measures. After satisfying themselves that the test itself was not flawed, Allegheny’s leaders looked more closely at the logistics of test administration. They suspected that the CLA results might be skewed by the progressively stronger academic performance of entering first-year students. “Our academic profile has gone up in the last three or four years,” President Cook says, “and so we haven’t had an apples-to-apples comparison. We think that puts uncertainty into the system.” But even if it’s “not a perfect measure,” Cook says, “we think it’s worth pursuing.”

Timing can also skew outcomes. When Bethel University in Minnesota gave first-year students the assessment well into their first semester, they got results that school administrators sensed were too good. The university suspected that students did better on the test with a month of classes under their belts than they might have done before classes started, when they were unaffected by the college experience.

Another challenge some institutions have faced is the requirement to provide SAT or ACT scores for students tested, since these measures are used to control for students’ initial academic ability. Alaska Pacific University and Heritage University in Washington state, both with large numbers of students who come from groups that are historically underrepresented among college-goers, are two universities that do not require the SAT or ACT. CLA administrators worked to find an alternative test to use as a control and settled on the Scholastic Level Exam (SLE), a short-form measure of cognitive ability developed by Wonderlic (an Illinois company with more than 40 years experience in educational testing services), which has turned out to be a reliable substitute. The SLE has proved valuable for other institutions that wanted to administer the CLA to nontraditional adult learners and is now being widely used in community colleges.

But by far the biggest challenge has been getting students to take and do well on the CLA. Because it is not a high-stakes exam like the GRE or MCAT, it has been difficult to recruit students and to be sure that they have given the test a good effort. The Council for Aid to Education learned early that the standard three-hour version of the assessment was unwieldy. The solution was to create a matrix sampling design, so that every student takes a 90-minute version of the test—essentially half the instrument.