Learning Outcomes and Assessment

By Petra Dierkes-Thrun (@petradt) and Robin Wharton (@rswharton)

This week, I am participating in a pedagogical experiment called MOOC MOOC, organized by the folks at Hybrid Pedagogy.  Robin Wharton (Georgia Tech) and I contributed the following post, and the questions underneath the article indicate the day’s MOOC MOOC participant activities. Feel free to leave comments and questions here!


As Cathy Davidson argues in Now You See It, our current assessment methods are conditioned by the needs and values of the industrial revolution (Ch. 4, “How We Measure”). Teachers grade students in the way the USDA grades beef. We are expected to sort students into categories so prospective employers, graduate institutions, their parents, and even the students themselves can see how “well” they did in our classes. The grading model presumes the audience for the grades we assign are consumers of, not agents or participants in, the learning process. It also assumes the teacher is still the sole arbiter of wisdom and judgment when it comes to assessment.

Yet why does the system automatically assume that we, the teachers, are in a better position than our students or even a broader public to evaluate the results of their learning process? If actors beyond the classroom are interested in the outcomes of our classes, then why don’t they participate in helping students comprehend and meet those outcomes? Shouldn’t taking responsibility for one’s learning necessarily imply acquiring the ability to assess it? And shouldn’t students feel a sense of accountability for their own work that goes beyond “pleasing the teacher” and shows their awareness of a larger public arena of readers and thinkers on the internet who can see their work?

Assessment is an essential part of pedagogy. We need to assess our students in order to understand where they are in their learning process, so that we can best figure how to meet their needs as learners. We also need to provide them with feedback and with opportunities to give and receive peer feedback in order to help them understand how assessment works, model the process for them, and give them space to practice it. We question, though, whether we need to continue “grading” in the traditional sense — that is sorting students into categories for audiences beyond the classroom.

We need to reconsider our assessment tools and processes, not abandoning the vital work of assessment but updating it to meet 21st century educational goals and challenges. Our assessment methods should also take advantage of 21st century learning technology. The advent of the MOOC changes the way we consider assessment, its purposes and applications.

Reinventing Assessment

We need to invent new, creative, challenging forms of assessment that address not only individual but also group- and project-based learning. For example, given the recent experiments with rubrics training and peer grading in humanities MOOCs, it is already becoming clear that the traditional essay format doesn’t work well. This is not just because one professional teacher, a teaching team, or a handful of peer graders won’t be able to give expert individual feedback or grade large numbers of essays in a short time.

Problematically, some of the current peer grading experiments want to train peers to grade just like the teacher would, effectively assuming that producing teacher clones should actually be the goal of peer review.  As a recent New York Times article stated, in this model “students first hav[e] to show that they can match a professor’s grading of an assignment, and then grade the work of five classmates, in return for which their work is graded by five fellow students”.  But “what would happen to a student who cannot match the professor’s grading has not been determined.”

Even more important than critiquing the pedagogical shortsightedness of this particular model, however, is thinking about the traditional essay format itself in the MOOC context: it may actually not be the best tool to use in humanities MOOCs, since it does not fully capture the multi-faceted, multi-pronged, versatile nature of online learning. It is typically a teaching and assessment tool geared at individual, in-depth writing and research skills and does not measure or encourage collaborative, spread-out, project-based writing. We need a larger toolbox for assessment methods and critical thinking in MOOCs that more adequately meet the demands of the digital age.

Personalization matters: giving students choices (but not letting them drown in them); making sure they get feedback on individual as well as on group and project performance; making them feel supported and heard on multiple levels — not only the teacher, but other students and occasionally, the larger public outside the MOOC, which may become the addressee or the prompter of individual or class writings and research projects.

We need to take the challenges MOOCs present as a chance to radically question, debate, and enlarge accepted pedagogical best practices. Many things we already know about good higher ed pedagogy — we don’t need to reinvent the wheel. We can draw upon sound portfolio- and studio-based pedagogy when thinking about how to give students more agency in learning environments and designing new assessment tools. Service learning pedagogy can teach us how to involve the public beyond the classroom in productive, pedagogically sound ways. Writing and composition pedagogy provides literally decades of experience from which we can learn as we design peer-to-peer assessment methods. And there are amazing new ways to use the new technology to teach close reading of literary and other cultural texts (such as films, images, objects) online.

Wide Open Classroom

We should welcome, explore, and exploit the power of large numbers and diverse backgrounds in open discussion. Forum conversations, social media networks, blogs, and other forms of participatory media demonstrate how interactive debate thrives and amazes with the sheer variety and diversity of participants’ interests, questions, expertise. When the physical classroom is thrown open to the world, it makes itself vulnerable to some chaos, but it also explodes with creativity and new thinking from which all participants can ultimately profit. This is a huge strength, not a threat to traditional humanities learning, which values and encourages different viewpoints and the honing of rhetorical and argumentation skills (the power of words and imagery to persuade and prompt empathy in another).  At the same time, we may need to think up new ways to keep discussions productive, not to control but to creatively mine chaos.

Giving due attention to the scholarship of teaching and learning can help us to meet those challenges common to traditional classrooms and MOOCs alike. It can also help us to appreciate those attributes truly unique to the MOOC environment. For instance, MOOCs present us with the dazzling diversity and uncertainty of students’ knowledge levels, personal backgrounds, purposes, and motivations for taking the course. They also pose new, but nevertheless mundane technological challenges to the traditional teacher who has to rethink “business as usual” when preparing class. And they showcase students’ often amazing ability and motivation to get involved and help each other (for instance in debating and answering peers’ questions before the teacher or TA gets a chance to weigh in). In this, they unwittingly practice another old pedagogical truism: that one only truly knows well what one has taught another.

The Questions at Hand

  1. How might reimagining assessment prompt us to rethink not only our pedagogical processes, but also the law and policy that governs traditional academic environments?
  2. What are some major chances and challenges of MOOC-style learning when it comes to traditional (as opposed to digital) humanities classes, specifically those that focus on such seemingly elusive, long-term outcomes as the honing of close reading and critical thinking skills, as well as sustained writing and research?
  3. How would you describe the desired learning outcomes for this course, the MOOC MOOC? How would you assess and document your own or your peers’ achievement of those outcomes?
  4. Should our ability to assess certain kinds of student work in a MOOC environment determine whether or not we assign that sort of work? If we can’t assess it, can students get credit for it in other ways?
  5. Who should be allowed to enter, observe, and participate in a MOOC? How does the kind of radical openness present in some MOOCs change our pedagogy? Should it change our pedagogy?

Some Articles to Explore

And the Task

  1. Create a Storify document that reflects what you’ve learned during MOOC MOOC thus far. (If you are unfamiliar with Storify, read How to Storify. Why to Storify. for some helpful ideas.)
  2. When your document is complete, tweet about it under the hashtag #moocmooc and include a link.
  3. Follow #moocmooc throughout the day to see other participants’ Storify articles. Read carefully, and assess in the comments section of their document. You can also “like” individual components that you find particularly cool or persuasive.
  4. Join the Assessments and Outcomes discussion in the course and contribute your thoughts about your own criteria for assessing today’s assignment, as well your ideas about assessments and outcomes in MOOCs.

Leave a comment

Filed under MOOC musings

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s