Measuring Engagement (With Books)


It has never been too hard to spot the student in class who didn’t do the  assigned reading — she’s usually the one in the back, not raising her hand, and  avoiding eye contact in hopes of not being called on. But a new program could  make identifying unprepared students even easier, and, the creators hope,  improve outcomes and retention.

CourseSmart, the digital textbook provider that is partnered with five major  publishers, announced the launch of CourseSmart Analytics on Wednesday. The  program, which is currently being piloted at three colleges, tracks students’ engagement with their e-textbooks and provides and allows professors and  colleges to evaluate the usefulness of learning materials and to track student  work.

The program looks at metrics of student usage, including page views, time  spent in a textbook, notes taken, highlights made, bookmarks used, and whether  or not the student even opened the book. The information is compiled and  analyzed by CourseSmart’s algorithm, and professors receive an overall  assessment of their students’ engagement through their regular learning  management system.

The emphasis on analytics is part of a larger  push in higher ed for data  linking course content to student outcomes. There is pressure on colleges to  prove their students are learning, and that pressure extends to book publishers  to prove their materials are useful in achieving the desired results.

“The big buzz in higher ed is analytics,” said Cindy Clarke, CourseSmart’s  senior vice president of marketing. “Based on what we had and what issues there  are with institutions around improving the return they’re getting on their  investment in course materials, we realized we had a valuable data set that we  could package up.”

CourseSmart, in fact, already collects information on student usage, so the  next step was to make that information useful to professors and administrators,  Clarke said. Rather than raw data, the CourseSmart program provides an  engagement score, which Clarke envisions being used in a number of ways.

Most obviously, professors can compare across a class to see which students  are most engaged and which are least engaged. Clarke said the program is not  intended for professors to give grades based on whether or not a student did the  reading, but to help professors tailor the kind of help they might offer a  student. For instance, if a student is struggling on tests but has a high  engagement score, the professor might focus on helping that student with study  skills and test preparation. Alternatively, a student with a low engagement  score and low test scores might need to learn active reading strategies, or to  be shown the value of actually doing the reading.

Professors can also evaluate how engaged the class is as a whole. Leading up  to a midterm exam, for example, a professor might notice there was one chapter  for which students had a particularly low engagement score, and so the professor  might choose to spend more time reviewing that section than other sections.

Of course, students might have different ways of studying — maybe they take  notes by hand, rather than highlighting the e-text — that could affect their  engagement scores. Still, Clarke said, when coupled with other metrics, the  score can be a useful indicator.

“Every individual student is different,” she said. “You’re going to hear  about the student who never cracks open the books and aces the test, and then  the other end of the spectrum where the student does all the reading but doesn’t  do well on the test.… The instructor is going to know the personality of the  student, but CourseSmart can provide a valuable set of facts for him to  incorporate into that.”

The hope is that professors will use the engagement score to better spot and  understand problems, thereby improving student outcomes.

“Analytics will be a powerful tool in higher education, especially when  data is reported in easy-to-understand, actionable ways,” Ellen Wagner,  executive director for WICHE’s Cooperative for Educational Technologies, said in  a press release. “The higher education community is hungry for actionable data  that links student engagement to their learning content.”

A professor can also use engagement scores to assess her own effectiveness.  For every textbook they use, professors and administrators will have access to  the overall engagement score, based on data from every college using that  textbook. This will, Clarke hopes, allow them to reflect on whether or not they  are making the best use of a book and supporting materials. She acknowledged  that different professors at different universities might use the same textbook  in various ways — teaching different chapters or spending more time on certain  sections, for example — but she said that given the number of universities  using CourseSmart materials, the aggregate score should account for  anomalies.

At the administrative level, department chairs or academic officers can look  at engagement scores across disciplines or across textbooks, to see, for  example, if there might be a more engaging and effective text for a particular  subject. They can also use the information from CourseSmart to analyze how and  if engagement is linked to outcomes and retention, and eventually, Clarke hopes,  they can use the analytics to develop retention strategies.

“Clearly there’s a lot of data that’s available at the institution level, but  there’s clearly some confusion, too,” Clarke said. “There’s a lot of effort  around figuring out how to use all of this data. We’ve already packaged it up  for them to make it easy for them to implement a useful tool.”

Though the program is only in the beta phase, Clarke said there has been a  lot of interest. CourseSmart is hoping to have a total of ten institutions  involved in the pilot, which will last through the end of the spring 2013  semester, and to launch later that year. There is no cost for access to the  engagement score dashboard during the beta period, and Clarke said it will  remain free for the foreseeable future.

CourseSmart is separately developing a dashboard that will let publishers see  engagement data for their texts and evaluate how they can improve their  materials, Clarke said.


Read more: Inside Higher Ed

This entry was posted in Articles. Bookmark the permalink.

One Response to Measuring Engagement (With Books)

  1. Tina. says:

    Very interesting article!! I think this is very overdue, especially with all the technology being used in education today. Nice job!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s