Once you’ve created your OER and used it, you may be wondering about the impact of using it in your course or the impact on your students. Did you students use it? Did they find it confusing? Did they even know it was an OER? This section will help you identify impact measurements of your OER.

Calculating Impact

The COUP Framework – Open Education Group

Attribution 4.0 International (CC BY 4.0); http://openedgroup.org/coup

The COUP Framework is the Open Education Group’s approach to studying the impact of open educational resources (like open textbooks) and open pedagogy in secondary and post-secondary education. COUP stands for:

Cost

Outcomes

Usage

Perceptions

Cost

The adoption of Open Educational Resources can impact a range of financial and cost metrics for students and institutions. Proponents of OER frequently claim that using these resources instead of traditional publisher textbooks or digital materials will save students money in the post-secondary context and will save organizations money in the K-12 context. There may also be other financial impacts, like changes in bookstore revenues and tuition revenues. The Cost strand of our work provides empirical evidence about the magnitude and direction of the financial impacts of OER adoption:

• Costs of textbooks previous assigned
• OER support fee models
• Changes in campus bookstore revenue
• Changes in tuition revenue due to changes in drop rates
• Changes in tuition revenue due to changes in enrollment intensity
• Changes in tuition revenue due to changes in persistence
• Changes in access to performance-based funding due to changes in drop, enrollment intensity, and persistence

Outcomes

Given the folk wisdom that “you get what you pay for,” some individuals and organizations worry that student learning will necessarily suffer when students use freely available, openly licensed resources instead of $200 textbooks. OER proponents claim that using these resources instead of traditional publisher textbooks or digital materials increases student access to critical learning materials and expands faculty’s academic freedom, consequently improving student learning outcomes. The Outcomes strand of our work provides empirical evidence about the magnitude and direction of the learning impacts of OER adoption:

• Changes in the percentage of students receiving a C or better
• Changes in rates of completion
• Changes in drop rates
• Changes in enrollment intensity
• Changes in persistence
• Changes in attainment of progress milestones (e.g., first 15 credits)
• Changes in graduation rates
• Usage

The permissions provided by open licenses allow students to use OER in a range of novel ways – for example, updating a history textbook based on recent events. Likewise, the permissions provided by open licenses allow teachers to engage in new pedagogical practices. Proponents of OER frequently claim that improvements in student learning outcomes will be highly correlated with the degree to which students and faculty exercise the permissions offered by OER. The Usage strand of our work provides empirical evidence about the ways faculty and students use OER and the the degree to which impacts on learning outcomes covary with these uses. We operationalize the idea of ‘exercising the permissions granted by open licenses’ by determining the degree to which students and faculty engage in activities described in the DIME model of OER adaptation:

• Deleting material from the OER
• Inserting other open material inside the OER
• Moving material around within the OER
• Editing material in the OER

Perceptions

What do faculty and students think about, and feel toward, Open Educational Resources? How do they judge their effectiveness relative to traditional textbooks? Their rigor and coverage? Do they find the formats, structures, and other design features easy to use? Frustrating? What about other stakeholders, like parents or policy makers – what are their thoughts and feelings toward OER? The Perceptions strand of our work provides empirical answers to these questions.

OER Adoption Impact Calculator – Lumen Learning

Another option to measure the impact of using an OER in your course is Lumen Learning’s calculator (https://impact.lumenlearning.com/). This calculator allows you measure a variety of elements related to OER adoptions, including:

• Number of Enrollments Using OER
• Average Cost of Textbook(s) Replaced ($)
• Average Spent by Students Using OER ($)
• Students Paying In-State Tuition (%)
• In-State Tuition Cost per Credit Hour ($)
• Out-of State Tuition per Credit Hour ($)
• Additional credits taken by OER Users
• Cost of New OER-related hires
• Historical C or Better Rate (%)
• Increase in C or Better Rate Among OER Users (%)
• Historical Drop Rate (%)
• Decrease in Drop Rate Among OER Users (%)
• Historical Withdraw Rate (%)
• Decrease in Withdraw Rate Among OER Users (%)
• Bookstore Textbook Sales Returned to Institution as Income (%)

This tool can help make several calculations