To Avoid Nasty Surprises, Higher Ed Turns To Prediction - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Denial-of-service attacks
06:25 PM
Doug Henschen
Doug Henschen
Connect Directly

To Avoid Nasty Surprises, Higher Ed Turns To Prediction

Colleges are turning to predictive analytics for a variety of uses.

InformationWeek Green - Jan. 28, 2013
InformationWeek Green
Download the entire February 2013 issue of InformationWeek Education, distributed in an all-digital format as part of our Green Initiative
(Registration required.)

Lots of mainstream businesses are graduating from rearview-mirror reporting to more advanced, predictive analytics, so it's no surprise that colleges and universities are making the same progress.

The goal, of course, is to figure out what's coming. You know how many students enrolled last fall, but did you accurately forecast how many freshmen would drop out after the first semester? Can you anticipate what revenue will look like next fall? How many dorm rooms, classrooms and dining halls will your college need five years from now? These are just a few examples of the data-driven analyses taking place in higher ed.

The move to analytics comes as colleges and universities face declining enrollments. Nearly half of colleges and universities responding to a recent survey by Moody's Investors Service said they expect enrollment of full-time students to decline. As a result, a third of them expect tuition revenue to decline or to grow at less than the inflation rate.

Compounding the problem, many colleges and universities are experiencing higher drop-out rates. The University of Kentucky ventured into advanced analytics last year hoping to improve student retention. UK is working on a way to identify which students are having trouble so that the university can intervene before they drop out.

UK did a proof-of-concept project in March, using six years' worth of student data going back to 2006, to model the characteristics of students who did and did not drop out after their freshman year. Within two months, the university developed a baseline predictive model using high school GPA and college entrance exam scores.

In May, the university hired a statistician who advanced the hypothesis that UK could do a better job of predicting retention by measuring student engagement, so the team added variables from the university's Blackboard learning management system. That data included the number of times students log in to their class Web pages, check syllabuses, download homework assignments and collaborate online with classmates, and whether they turn in homework assignments on time. UK is using SAP's Predictive Analysis software, in large part because it uses SAP's ERP system and BusinessObjects BI software.

UK's risk models, based on regression analyses, have yet to be proved; that will require more testing and comparative analysis, says Adam Recktenwald, the university's enterprise architect. Nonetheless, Recktenwald says UK is taking a novel approach.

"Schools tend to look at students in large swaths, such as high-performing students and low-performing students," he says. "We want to bring this down to micro segments, so that we can better understand the needs of individual students."

Faith In Numbers

Taylor University, a private Christian university in Indiana with 1,900 undergraduates and 120 grad students, is also experimenting with retention analytics, using the RStat module from Information Builders, its incumbent business intelligence software vendor.

After spending two months compiling data, Taylor's director of institutional research and associate registrar, Edwin Welch, was disappointed to learn that he could come up with only about 6,300 complete and consistent digital student records, 460 of which pertained to students who dropped out after their freshman year -- the outcome Welch wanted to predict. That's not a large sample size, particularly when the software splits that data, randomly choosing 70% of the records to build models and holding aside 30% for testing accuracy.

Despite the thin data sample, Welch built a variety of models using logistic regression, neural networks and other techniques available in the software. The initial effort turned out models that predicted with 78% to 86% accuracy whether a given freshman would either leave the university or continue on to sophomore year.

After testing a number of variables, Welch determined that the best predictors of retention for Taylor boiled down to six measures: high school GPA, SAT and ACT scores, fall-term GPA, percentage of credit hours completed in the freshman fall term, number of midterm grades below a C- in the fall term, and the number of credit hours registered for in the spring term.

Not satisfied with any one model, Welch then combined the results of his six best models and reached 90% accuracy, measured by testing the model against 2011 data. In that year, 68 freshmen dropped out, yet only nine of those students were on academic alert lists (compiled based on faculty and staff observations rather than data analysis). Welch's model predicted that 25 of those 68 students wouldn't return.

Welch says he isn't satisfied with 90% accuracy, but until he can add more data to the model, he can't hope for better. Still, had Taylor used the model in 2011, it would have known that 16 more students were at risk than were identified by conventional means. Retaining even half of those students would have yielded a return on the university's analytics software investment.

Taylor is putting the predictive modeling to use. Students flagged by five or six of Welch's models will be contacted by the university's Academic Enrichment Center, which offers tutoring and seminars on time and stress management. Students flagged by three or four of those models will go on the alert lists of instructors, counselors and hall directors.

The University of Kentucky is in the same boat; it won't have definitive evidence of retention success until the school year comes to a close. Meantime, UK is developing models to predict revenue, insight that will help it predict the number of classrooms, dorm rooms and even buses it will need. "We'll continue to refine these tools," Recktenwald says, "and we're designing methods to bake the findings directly into our resource management process."

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
Melanie Rodier
Melanie Rodier,
User Rank: Apprentice
1/29/2013 | 4:08:06 PM
re: To Avoid Nasty Surprises, Higher Ed Turns To Prediction
It would be interesting to see how effective the extra tutoring, stress management etc is at retaining the students who are deemed to be at high risk of leaving. Maybe these students are looking for a type of instruction/flexibility/etc etc that their current university can't provide, no matter how much tutoring and other "traditional' programs they are offered, and universities as a result still need to look out-of-the-box to meet their students' needs.
User Rank: Apprentice
1/29/2013 | 1:32:07 AM
re: To Avoid Nasty Surprises, Higher Ed Turns To Prediction
I can't resist: Does the University of Kentucky model factor in the cast of one-and-done basketball players?
User Rank: Strategist
1/29/2013 | 1:07:39 AM
re: To Avoid Nasty Surprises, Higher Ed Turns To Prediction
I would urge more analytics -- and more staff training. It's surprising at Taylor U. that faculty and staff identified 13% of potential dropouts in 2011, while an associate registrar's modeling program, 90%. That's testimony to the power of numbers. It's also possible, since he was using 2011 data in 2012, his analytics program's data was more complete than any one staff member's knowledge during the year of 2011. Charlie Babcock, editor at large, InformationWeek
D. Henschen
D. Henschen,
User Rank: Author
1/28/2013 | 3:54:44 PM
re: To Avoid Nasty Surprises, Higher Ed Turns To Prediction
Analytics certainly isn't new to higher ed, but it's becomming more afforadable and accessible to smaller and non-elite schools. The Harvards and Yales of this world can be selective about who they let in and, as a result, their student retention rates are super high. Smaller colleges and universities and state institutions that have mandates to serve a cross-section of state residents are having real trouble with both enrollment and retention. What's more, schools can't just lower standards to fill seats as they'll risk their reputations. The only answer is working smarter and spotting student needs and risks sooner with the aid of data-driven analysis.
2020 State of DevOps Report
2020 State of DevOps Report
Download this report today to learn more about the key tools and technologies being utilized, and how organizations deal with the cultural and process changes that DevOps brings. The report also examines the barriers organizations face, as well as the rewards from DevOps including faster application delivery, higher quality products, and quicker recovery from errors in production.
Data Science: How the Pandemic Has Affected 10 Popular Jobs
Cynthia Harvey, Freelance Journalist, InformationWeek,  9/9/2020
The Growing Security Priority for DevOps and Cloud Migration
Joao-Pierre S. Ruth, Senior Writer,  9/3/2020
Dark Side of AI: How to Make Artificial Intelligence Trustworthy
Guest Commentary, Guest Commentary,  9/15/2020
Register for InformationWeek Newsletters
Current Issue
IT Automation Transforms Network Management
In this special report we will examine the layers of automation and orchestration in IT operations, and how they can provide high availability and greater scale for modern applications and business demands.
White Papers
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Sponsored Video
Flash Poll