jossey-bass
Nonprofit Business Advisor, Strategies to Survive and Grow in Tough Times

Creating a Cadre of Assessment Gurus (at Your Institution)

By Philip I. Kramer, Rita Knuesel, Kenneth M. Jones August 15, 2012

(Article originally published in the July-August issue of Assessment Updateavailable electronically to subscribers at publication.)


This article is a case study based on the efforts of two colleges to expand and strengthen faculty and staff assessment activities; it is also a road map for you and your institution to create a cadre of assessment gurus. Although we received a grant from the Teagle Foundation that led to a series of formal on-campus “Assessment 101” workshops for faculty and staff, we suggest that your institution can create similar, highly successful assessment training workshops that are budget sensitive. At the end of this training, your newly minted assessment gurus will help themselves and their colleagues use assessment evidence to improve teaching and learning both inside and outside the classroom.

Faculty and Staff Workshops on Student Outcomes Assessment

Background. In May 2008 the College of Saint Benedict and Saint John’s University received a $149,667 grant from the Teagle Foundation for a forty-eight-month period to increase faculty expertise in assessment, create an institutionalized system of mentoring good assessment work, refine data sharing and communications structures related to assessment, and enhance oversight processes and structures to sustain the momentum for this work.

The primary vehicle for mentoring faculty and staff in the craft of assessment was to create—from scratch—four yearlong assessment workshops. Our “Assessment 101” workshops were designed to introduce faculty, staff, and student participants to the literature on student learning; help them explore issues in higher education assessment; teach them about assessment methods; and have them design, conduct, analyze, and use assessment to improve pedagogy and curricula and to provide highly visible accountability in their respective departments and programs and across the two institutions. During each workshop we employed two to three undergraduate “Student Research Scholars” who participated as peers of our faculty and staff. The students played a significant role in our class discussions while developing their skills as they worked in the Office of Academic Review and Curricular Advancement. The director of that office provided on-the-job training and professional mentoring to these students. Presently, all of the graduated Student Research Scholars are enrolled in doctoral programs or are in preparation for doctoral studies. By the end of the grant, we had trained fifty-four faculty and staff members. Of those fiftyfour, thirty-nine are faculty members from twenty-seven departments, and fifteen are staff members from twelve areas in student development or academic affairs.

These workshops have had very positive effects at our institutions. Examples include the following:

  • Our faculty governance assessment committee implemented a new reporting system for academic departments and programs that was significantly influenced by people who participated in the Assessment 101 workshops.
  • The new approach to assessing student work in our yearlong freshman seminar was substantially influenced by faculty who completed the Assessment 101 workshops.
  • Both the Teagle-funded training and subsequent assessment work have been essential in shaping our participation in the 2010 study conducted by the Center for Inquiry in the Liberal Arts at Wabash College.
  • One member of the first class, with the help of the director of academic review and curricular advancement and a student intern, developed an entirely new approach to assessing student dispositions in philosophy. He has presented this work at several national and international disciplinary and assessment conferences.
  • Teagle-trained faculty have played some role in supporting two other significant faculty-driven efforts to develop new assessment tools in intercultural understanding and liberal education.
  • Other faculty and staff have presented their work at several national and international assessment or disciplinary professional conferences.

Description. Most faculty and staff (whether during or after graduate school) never received formal training in post-secondary assessment, including being trained in typical assessment analysis. Consequently, all of the workshop training we designed used a hands-on, problem-based approach to learning about and using postsecondary student outcomes assessment, and then applying that learning to improve teaching and learning, demonstrate accountability, and improve assessment in our departments, divisions, and institutions. We designed the workshops to accomplish the following specific participant learning goals:

  • Participants became familiar with practical, real-world, postsecondary student outcomes assessment questions, challenges, and successes as a method of preparing them to work independently and collectively with their colleagues in their departments, divisions, and across the institutions.
  • Participants had opportunities to discuss, debate, and brainstorm with one another. Experiential activities were provided and allowed participants to experience planning, designing, conducting, analyzing, and reporting and implementing meaningful student outcomes in their own departments and divisions and across the institutions.
  • Participants developed a working knowledge of the relative history, trends, and purposes of postsecondary student outcomes assessment and outcomes assessment nationally and at our institutions to contextualize their assessment efforts in departmental, divisional, and institutional missions, goals, and priorities.

Each workshop participant was assigned to a “base” or home group. Membership in base groups lasted for the duration of the yearlong workshops; we did this to facilitate individual and group ideas and projects, workshop discussions, and activities. In addition, a Teagle Student Research Scholar was assigned to each base group to work as a research consultant, helping participants with workshop activities and projects. While base groups served as the focal point for several of the small-group activities and individual presentations, activities and discussions also occurred in the larger workshop groups.

Assessment Analysis (Part One and Part Two). As individuals and as a member of a base group, participants applied what they learned and analyzed assessment in their departments, programs, or divisions. Participants individually presented the strengths and weaknesses of an assessment plan meaningful to them, including the learning goals to be measured; the implementation, analysis, and reporting of assessment; and the use of assessment to improve teaching and learning, provide accountability, and refine ongoing assessment. Participants first presented their assessment analyses during meetings held at the end of the first semester. This first presentation was mostly a dry run because, although it had all the elements of careful assessment planning, participants were not asked to present their assessment evidence.

At the end of the second semester, after further development of their assessment plans (including refining learning goals, selecting appropriate measures, and using assessment to make improvements), participants were required to present assessment evidence. Participants were also asked to think carefully about providing stakeholders with evidence of accountability and to consider how to refine assessment efforts in the future.

Assessing the Workshops. The workshops were evaluated using a variety of measures including focus groups, online surveys, and individual conversations with the facilitators. In addition, involved faculty and staff were asked to help design appropriate measures of the workshops’ effectiveness. The workshops included the mandatory completion of a project.

Implementing Assessment Workshops on Your Campus

To begin your own series of Assessment 101 workshops, we first suggest that representative faculty, staff, students, and senior administrators take the time to consider the following questions. We believe that working collegially on these questions, and the implications the answers have for improving teaching and learning, is essential for success. Once this is accomplished, find the campus assessment experts willing to offer assessment workshops.

Ownership

  • Who “owns” assessment on your campus? Is the ownership the same among faculty, staff, students, and senior administrators?
  • Who has responsibility for assessment? Who has responsibilities for teaching and curricular designs? Are they the same people or units?
  • What are the origins of assessment on your campus?
  • How would assessment training be funded?
  • In general, what are the purposes of assessment? Are they systemic?
  • Is assessment supported by faculty, staff, students, and the administration?
  • Is assessment valued? Is assessment part of faculty, staff, administrative, and student cultures?

Communication

  • Do you have open and transparent communication between senior administrators and the faculty? Why or why not?
  • What would it take to have frank and transparent communication between senior administrators and the faculty?
  • What could be some of the catalysts that could change the way communication occurs?
  • What would it take to have your internal stakeholders (faculty and senior administrators) discuss some kind of formal or semiformal assessment training for faculty?
  • Would creating a culture of inquiry and evidence-informed decision making jeopardize or enhance the sense of institutional community, academic freedom, and shared governance?
  • What are some of the catalysts that could change how your institution considers assessment?

Development

  • Is assessment training “on the job” or is there some kind of training for faculty?
  • How do faculty and senior administrators “learn” assessment on your campus?
  • Who conducts the current training or who would conduct future training?
  • Would there be any faculty and staff rewards for participating in assessment training?
  • Does assessment lead to improvements in teaching and learning? Are there rewards for good results? Is there punishment for poor results?
  • Does assessment evolve? Is it evaluated and improved?
  • Is assessment woven into teaching, scholarship, and service?

Continuing the Workshops After the Grant

The provost and academic affairs have committed to sequester the funds needed to continue the Assessment 101 workshops for faculty and staff indefinitely, albeit most likely in a shorter (probably one semester in length as opposed to two semesters) and more cost-conscious way. Initial conversations suggest that we will institutionalize the workshop series by offering it as one of the faculty development program opportunities. We hope that this description of our work will encourage you to create your own assessment gurus.

Philip I. Kramer is director of The Office of Academic Review and Curricular Advancement, Rita Knuesel is provost, and Kenneth M. Jones is assistant dean of the common curriculum at the College of Saint Benedict and Saint John’s University.


Copyright © 2000-2015 by John Wiley & Sons, Inc. or related companies. All rights reserved.
Wiley