Educational Software Gets Graded

12 Open Educational Resources: From Khan to MIT

Teachers will tell you the problem isn't a paucity of educational software. The problem, they say, is picking between dozens or even hundreds of applications all claiming to teach a subject.

"Right now we live in a system in which nobody knows what works," said Benjamin Jones, an associate professor at Kellogg School of Management and faculty director of the Kellogg Innovation and Entrepreneurship Initiative (KIEI).

"They don't know what to buy or what to implement," Jones told InformationWeek during an interview in his office on the Northwestern University campus.

Hoping to address this fundamental problem, Jones, along with Duke University business professor Aaron Chatterji, is building Edu Star, a cloud-based platform that will, among other things, offer empirical rankings of educational software.

Specifically, Edu Star plans to rate each technology on a scale of one to five stars. It will also provide additional information about the software, such as how many students have used it, how it was tested and user ratings from students and teachers.

Edu Star will be available to academics and application developers alike, an important distinction.

[ New-fangled educational devices didn't start with computers or the Internet. Read Tablets Rock On: Education Tech Through The Ages. ]

This feedback loop between users and developers distinguishes the project from Consumer Reports, to which it has sometimes been compared.

"Edu Star has features of the Consumer Reports model," Jones said. "But in this space, you need more than experts, you need quantitative assessment at low cost."

Jones does appreciate how Consumer Reports' transparent ranking methodology has made it into a trusted brand, something he would like to emulate.

Jones imagines an automated, iterative, A/B testing scheme. These results would be extremely valuable to developers, of course.

"On one side are students, teachers and parents. On the other side are developers," Jones said.

However, just as important, will be keeping these testing costs down. This will give "innovators and entrepreneurs a testing opportunity" they may be denied today, Jones said.

According to Jones, it's incredibly hard for application developers to make money in the K-12 market, in part because procurement cycles are very long. He hopes Edu Star will help shrink that time by giving school procurement managers empirical evidence that an application performs as advertised.

Since good teachers often have their own innovative ideas, Jones assumes some using Edu Star will pair up with developers, helping them generate new tools that can get to market quickly.

Edu Star will be hosted at -- and tap into the users and content of -- PowerMyLearning, a free K-12 online learning platform operated by nonprofit CFY. To date, PowerMyLearning has received grants totaling $7 million from the Bill & Melinda Gates Foundation, The Eli and Edythe Broad Foundation, and the W.K. Kellogg Foundation.

CFY's Digital Learning Program has grown from serving a few hundred students, plus their teachers and parents, in one school in 1999 to now having served over 50,000 students, plus their teachers and parents, in more than 100 schools nationwide.

The goal of Edu Star is to bring rigor to educational applications testing, said Elisabeth Stock, the founder and executive director of CFY, in a phone interview from Jones' office.

"We're trying to show causation, using a random controlled trial, as is done in medicine," she said.

Like Jones, CFY is committed to using digital technology to helps students, educators and parents improve educational outcomes. But more so, both are eager to quantify the impact of educational software on student outcomes.

In this sense, Edu Star mirrors other efforts, such as the Predictive Analytics Reporting (PAR) Framework, which has been aggregating student data from two-year and four-year institutions.

Interest in proving the efficacy of different educational practices has been spurred by the Common Core Standards, the U.S. education initiative to standardize curriculum across the states. Not only do the CCS specify math and language arts requirements, they require schools to move towards computer-based assessment.

Under the CCS mandates, "even the best teachers in the state will have to make some changes," said Stock, adding that this situation presents a unique opportunity for offerings like Edu Star.

Edu Star integration into PowerMyLearning is expected to start early next year.

Education IT directors are leveraging technologies and technology models that allow them to implement the right products and services for schools right now. In this Top Tech Tools For Educators report, we'll look at some of the tools schools are using and how they're being put to work in the classroom to engage students and improve learning. (Free registration required.)