Denial-of-service attacks
Commentary
1/24/2013
06:25 PM
Doug Henschen
Doug Henschen
Commentary
Connect Directly
LinkedIn
Twitter
Google+
RSS
E-Mail
50%
50%

To Avoid Nasty Surprises, Higher Ed Turns To Prediction

Colleges are turning to predictive analytics for a variety of uses.

InformationWeek Green - Jan. 28, 2013
InformationWeek Green
Download the entire February 2013 issue of InformationWeek Education, distributed in an all-digital format as part of our Green Initiative
(Registration required.)


Lots of mainstream businesses are graduating from rearview-mirror reporting to more advanced, predictive analytics, so it's no surprise that colleges and universities are making the same progress.

The goal, of course, is to figure out what's coming. You know how many students enrolled last fall, but did you accurately forecast how many freshmen would drop out after the first semester? Can you anticipate what revenue will look like next fall? How many dorm rooms, classrooms and dining halls will your college need five years from now? These are just a few examples of the data-driven analyses taking place in higher ed.

The move to analytics comes as colleges and universities face declining enrollments. Nearly half of colleges and universities responding to a recent survey by Moody's Investors Service said they expect enrollment of full-time students to decline. As a result, a third of them expect tuition revenue to decline or to grow at less than the inflation rate.

Compounding the problem, many colleges and universities are experiencing higher drop-out rates. The University of Kentucky ventured into advanced analytics last year hoping to improve student retention. UK is working on a way to identify which students are having trouble so that the university can intervene before they drop out.

UK did a proof-of-concept project in March, using six years' worth of student data going back to 2006, to model the characteristics of students who did and did not drop out after their freshman year. Within two months, the university developed a baseline predictive model using high school GPA and college entrance exam scores.

In May, the university hired a statistician who advanced the hypothesis that UK could do a better job of predicting retention by measuring student engagement, so the team added variables from the university's Blackboard learning management system. That data included the number of times students log in to their class Web pages, check syllabuses, download homework assignments and collaborate online with classmates, and whether they turn in homework assignments on time. UK is using SAP's Predictive Analysis software, in large part because it uses SAP's ERP system and BusinessObjects BI software.

UK's risk models, based on regression analyses, have yet to be proved; that will require more testing and comparative analysis, says Adam Recktenwald, the university's enterprise architect. Nonetheless, Recktenwald says UK is taking a novel approach.

"Schools tend to look at students in large swaths, such as high-performing students and low-performing students," he says. "We want to bring this down to micro segments, so that we can better understand the needs of individual students."

Faith In Numbers

Taylor University, a private Christian university in Indiana with 1,900 undergraduates and 120 grad students, is also experimenting with retention analytics, using the RStat module from Information Builders, its incumbent business intelligence software vendor.

After spending two months compiling data, Taylor's director of institutional research and associate registrar, Edwin Welch, was disappointed to learn that he could come up with only about 6,300 complete and consistent digital student records, 460 of which pertained to students who dropped out after their freshman year -- the outcome Welch wanted to predict. That's not a large sample size, particularly when the software splits that data, randomly choosing 70% of the records to build models and holding aside 30% for testing accuracy.

Despite the thin data sample, Welch built a variety of models using logistic regression, neural networks and other techniques available in the software. The initial effort turned out models that predicted with 78% to 86% accuracy whether a given freshman would either leave the university or continue on to sophomore year.

After testing a number of variables, Welch determined that the best predictors of retention for Taylor boiled down to six measures: high school GPA, SAT and ACT scores, fall-term GPA, percentage of credit hours completed in the freshman fall term, number of midterm grades below a C- in the fall term, and the number of credit hours registered for in the spring term.

Not satisfied with any one model, Welch then combined the results of his six best models and reached 90% accuracy, measured by testing the model against 2011 data. In that year, 68 freshmen dropped out, yet only nine of those students were on academic alert lists (compiled based on faculty and staff observations rather than data analysis). Welch's model predicted that 25 of those 68 students wouldn't return.

Welch says he isn't satisfied with 90% accuracy, but until he can add more data to the model, he can't hope for better. Still, had Taylor used the model in 2011, it would have known that 16 more students were at risk than were identified by conventional means. Retaining even half of those students would have yielded a return on the university's analytics software investment.

Taylor is putting the predictive modeling to use. Students flagged by five or six of Welch's models will be contacted by the university's Academic Enrichment Center, which offers tutoring and seminars on time and stress management. Students flagged by three or four of those models will go on the alert lists of instructors, counselors and hall directors.

The University of Kentucky is in the same boat; it won't have definitive evidence of retention success until the school year comes to a close. Meantime, UK is developing models to predict revenue, insight that will help it predict the number of classrooms, dorm rooms and even buses it will need. "We'll continue to refine these tools," Recktenwald says, "and we're designing methods to bake the findings directly into our resource management process."

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Melanie Rodier
50%
50%
Melanie Rodier,
User Rank: Apprentice
1/29/2013 | 4:08:06 PM
re: To Avoid Nasty Surprises, Higher Ed Turns To Prediction
It would be interesting to see how effective the extra tutoring, stress management etc is at retaining the students who are deemed to be at high risk of leaving. Maybe these students are looking for a type of instruction/flexibility/etc etc that their current university can't provide, no matter how much tutoring and other "traditional' programs they are offered, and universities as a result still need to look out-of-the-box to meet their students' needs.
FritzNelson
50%
50%
FritzNelson,
User Rank: Apprentice
1/29/2013 | 1:32:07 AM
re: To Avoid Nasty Surprises, Higher Ed Turns To Prediction
I can't resist: Does the University of Kentucky model factor in the cast of one-and-done basketball players?
cbabcock
50%
50%
cbabcock,
User Rank: Strategist
1/29/2013 | 1:07:39 AM
re: To Avoid Nasty Surprises, Higher Ed Turns To Prediction
I would urge more analytics -- and more staff training. It's surprising at Taylor U. that faculty and staff identified 13% of potential dropouts in 2011, while an associate registrar's modeling program, 90%. That's testimony to the power of numbers. It's also possible, since he was using 2011 data in 2012, his analytics program's data was more complete than any one staff member's knowledge during the year of 2011. Charlie Babcock, editor at large, InformationWeek
D. Henschen
50%
50%
D. Henschen,
User Rank: Author
1/28/2013 | 3:54:44 PM
re: To Avoid Nasty Surprises, Higher Ed Turns To Prediction
Analytics certainly isn't new to higher ed, but it's becomming more afforadable and accessible to smaller and non-elite schools. The Harvards and Yales of this world can be selective about who they let in and, as a result, their student retention rates are super high. Smaller colleges and universities and state institutions that have mandates to serve a cross-section of state residents are having real trouble with both enrollment and retention. What's more, schools can't just lower standards to fill seats as they'll risk their reputations. The only answer is working smarter and spotting student needs and risks sooner with the aid of data-driven analysis.
The Business of Going Digital
The Business of Going Digital
Digital business isn't about changing code; it's about changing what legacy sales, distribution, customer service, and product groups do in the new digital age. It's about bringing big data analytics, mobile, social, marketing automation, cloud computing, and the app economy together to launch new products and services. We're seeing new titles in this digital revolution, new responsibilities, new business models, and major shifts in technology spending.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest - August 27, 2014
Who wins in cloud price wars? Short answer: not IT. Enterprises don't want bare-bones IaaS. Providers must focus on support, not undercutting rivals.
Flash Poll
Video
Slideshows
Twitter Feed
InformationWeek Radio
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.