by Daniel Mucinskas and Victoria Nichols
At the end of a typical university semester, professors will hand out evaluations to students with the hope of garnering insightful and constructive feedback about a course. Unfortunately, professors will typically distribute these evaluations in the final moments of class or in conjunction with an examination. As such, students will frequently dismiss the questions, providing vague or incomplete responses as they aren’t allowed the time necessary to produce thoughtful answers. Moreover, students are almost never given time to pause and reflect on their experiences before providing an evaluation; rather, the process is sudden and soon over. By requesting responses immediately upon the termination of a course, professors are limiting the scope of student surveys. If instructors were to administer surveys years after students complete a course, the answers would conceivably provide insight into the long-term personal and professional impact of the material. With this in mind, Dr. Howard Gardner, along with a team of Harvard University researchers, set out to investigate the long-term effects of a course on Good Work by surveying a wide range of former students.
At the Harvard Graduate School of Education (HGSE), Gardner has been teaching a course for 15 years on “Good Work in Education: When Excellence, Ethics, and Engagement Meet.” The course is an overview of the Good Work Project with two major components: in-class discussions of topical events through a Good Work lens and an extensive research paper on a Good Work-related topic. In the fall of 2013, a survey of eight questions was sent via email to 101 former H-175 students, soliciting their feedback on the course. The questions prompted the recipients to recall the most memorable course content and to discuss whether H-175 had changed how they approached situations in their professional and/or personal lives. A total of 47 substantive responses were received. Two graders (the authors of this blog) then scored all answers on a 0-to-3 point scale for five separate measures: assignments/readings/discussions; research paper; professional influence; personal influence; and Good Work concepts. As newly tuned social scientists, we had a crash course in achieving reliability. And, indeed, after many rounds of independent scoring, we eventually reached agreement.
Once scoring was completed, the highest-scored responses were ultimately analyzed for overarching thematic concepts. Detailed coding revealed significant and positive trends amongst former students, with 79% of responses at least mentioning, and many illustrating in anecdotes, how the course has been professionally and/or personally influential. The most rewarding part of the experience was seeing the number of former students who felt as though the course was a key part of their graduate education and that it had given them tools with which to navigate particular situations in their later lives. For example, one former student described using the Good Work framework as a part of her job while selecting recipients of grants, while another alum discussed reevaluating what it means to be personally and professionally successful and where those two realms overlap. As one student said, “My participation in the Good Work course helped me to begin to understand the complexities of ethics and ethical action.” Moreover, 45% of students stated that the final research paper was a powerful experience.
This method of assessment, however, does have limitations. While evaluating a class from the perspective of several months or years after completion appears to provide unique and helpful insights, there is the possibility that participants’ memories may be inaccurate, flawed, or incomplete. It should also be noted that not all students who have taken H-175 were emailed; moreover, not all former students who were contacted responded. In turn, there may have been a participant bias in that those who responded to the survey may have also been those who had the most profound and beneficial experiences in the course.
Nonetheless, student responses revealed that H-175 had a lasting impact on the majority of students’ lives and career. Upon reviewing the results from this survey, Gardner plans to make changes in the H175 course. Educators at all levels would undoubtedly also find it useful to engage in long-term assessment of former students in order to evaluate the effects of assignments and curricula over time. This is particularly true of a course like Good Work. Since the material is practical and adaptable across a wide variety of situations, assessment and evaluation are best done from a vantage of several years By that point, former students can assess whether, and if so, how they have employed the Good Work framework in their lives as a whole.