Information Systems Professor Develops Automatic Grading Software

Students and professors know that grading assignments can take weeks. But what if it didn’t? What if grading could take place instantly? BYU professor Gove Allen explains how he took on the problem by developing automatic grading software for use in introductory Excel classes.

Q: What exactly does the software do?

A: The program creates an environment where students can submit assignments and then have them graded immediately. When a student downloads the assignment workbook for Excel, everything the student needs to do that assignment (and everything we need to grade the assignment) is included.

The theory behind the software is that shortening the feedback cycle increases learning. My colleagues and I have thought, ‘What if we could push the feedback cycle down to just a few seconds?’ Doing that gets students feedback while their mind is still engaged on the problem. They don’t have to go back and rethink of the assignment because their mind is still right there in the work.

Q: When was the software developed?

A: Development on the program started more than three years ago when Professor Ball took over the IS 100 class. At that time we were teaching the class with a simulation to Excel, but students who worked on those sites got dissatisfied pretty quickly after realizing they weren’t getting the full experience.

When Professor Ball took over the class — and this has really been a team development since he wrote the textbook and I wrote the software — he came to me knowing that I was doing some automated grading for students in an advanced Excel class. He asked if we could apply what I was doing to the entry-level class, and things grew from there.

Q: How does the software improve the learning experience?

A: Most professors that use the program give students two attempts at the work. That gives students a compelling reason to actually look at feedback, learn from it and re-engage the problem immediately. It gets down to what we already know about learning: reducing the feedback cycle and giving students a reason to take homework seriously improves performance.

Q: When did you start distributing the software to other schools?

A: About a year ago Nick and I really had the product to where we thought we could offer it to other schools. It’s been adopted by about ten schools now. The University of Arizona, Utah State University, Utah Valley University, Arizona State University, West Texas A&M and the University of St. Thomas in Minnesota are some of the institutions currently using the program.

Q: What makes the software stand out from other automatic grading programs?

A: Grading has been one of the big problems with the systems that simulate Excel. With the simulation approach, a server or a cluster of servers is used to process submissions. When you’ve got 1,500 students from BYU and 2,700 students from the University of Arizona who are all submitting the assignment at the same time, server performance can come to a halt. We realized that we could actually have the student’s machine do the grading. Architecturally, it’s a huge advantage because every student that brings extra demand to the system also brings the same amount of processing power to service that demand. It scales beautifully.

The first time that we did an assignment at the University of Arizona, we were a little nervous because we didn’t know how it was going to go. The publisher that we’re working with for the e-textbook put extra people on staff that night to make sure there would be resources for support calls. They said there wasn’t even a blip, the student submissions didn’t slow down anything. That architectural choice is one of the things that sets us apart from anyone else who’s doing automatic grading for Excel.

Q: What do you consider some of your biggest achievements in developing this software?

A: Our software provides a better experience. We’ve actually done tests between students learning the simulation approach and our approach — we’ve run two classes simultaneously — and students learn about 10 percent more using our approach than they do using the simulation approach. They like it more, too. Our research shows they would be more willing to take another class in information systems because they’re actually working in Excel rather than on a webpage service.