Strategies for Effective Assessment: Balancing Tradition and Technology

Chad Bebee, Director of Academic Assessment, University of Northern Colorado

Chad Bebee, Director of Academic Assessment, University of Northern Colorado

Chad has a broad spectrum of interests such as curriculum studies, literature, music, mathematics, science, and public policy and is characterized by a curiosity that drives a passion for learning and skill mastery. His research delves into the phenomenology of critical thinking instruction within foundational studies curricula.

Please tell us about the journey that you've had so far and your roles and responsibilities at University of Northern Colorado.

As a first-generation college student, I earned my Bachelor of Arts in English and later pursued a Master of Arts in English and American Literature at Indiana State University. After working for seven years, including serving as a teaching assistant during my master's program, I returned to Indiana State in 2015 to pursue a Ph.D. in curriculum instruction. My interest in helping others become effective teachers led me to focus on understanding how humans learn and the practical aspects of curriculum.

While completing my Ph.D., I worked as an English instructor at Vincennes University in Indiana and took on the role of interim assessment director during an HLC review. My prior experiences with assessment, including being the institutional assessment liaison, prepared me for this position. After completing my Ph.D., I came across an opportunity at the University of Northern Colorado as an assessment director. This allowed me to apply my skills at a larger institution with diverse academic offerings, including associate degrees, baccalaureate degrees, and graduate degrees.

In my role at the University of Northern Colorado, my primary responsibilities include overseeing all academic assessment efforts. I collaborate closely with co-curricular programs, ensuring a coordinated approach between academic and non-academic assessments. My focus has been on streamlining and enhancing assessment processes, particularly through the learning management system (LMS), Canvas. Leveraging my prior experience with Blackboard LMS, I've facilitated more efficient use of Canvas for general education and program-level assessments. This involves guiding faculty in framing program learning outcomes, embedding them in Canvas, creating performance level descriptors, and setting goals.

A significant part of my day involves working directly with faculty on various projects, especially in the realm of general education. We've dedicated time and effort to create a comprehensive assessment plan, embedding processes in Canvas, and extending these practices across the campus. I prioritize providing professional development to ensure effective utilization of assessment systems and fostering a culture of continuous improvement in student learning.

What are some of the major challenges prevailing lately in the assessment management space?

I hold the belief that technology isn't a cure-all but rather a tool, and its adoption should be cautious. There's a danger in technology companies overselling their solutions, sometimes making the cure worse than the problem. We risk becoming too reliant on systems, limiting our flexibility, especially in providing feedback to students.

While there are excellent systems, many are costly, posing challenges for cash-strapped institutions. In times of financial strain, a return on investment is crucial. My perspective is that technology is beneficial, but we should prioritize simple and effective solutions before investing in expensive options. Establishing a robust foundation and refining processes should precede exploring where technology can enhance them.

Looking ahead, I see potential transformation through generative AI, which could reshape the nature of my work significantly.

Could you provide more details on how you believe generative AI can effectively address the pain points you highlighted earlier?

We are currently in the early stages of AI development, and its potential is both frightening and inspiring. I believe that with proper policies, students should be encouraged to use generative AI as it will likely become ubiquitous in various aspects of our lives.

Rather than shielding students from AI due to concerns about academic integrity, we should focus on ethical and effective use, treating AI as a learning tool. I propose a middle ground approach. We should teach students how to use AI effectively and ethically, considering it as a learning tool. AI can offer valuable feedback on tasks like citation format and analysis, provided we frame the questions appropriately.

Understanding how to prompt the system is crucial in obtaining relevant information and avoiding errors such as parroting back input or generating non-existent sources. While caution is necessary, I've personally found AI to be a fantastic tool for instructors, particularly in developing rubrics, assignment prompts, and assessing test questions. It has the potential to streamline our work and offer creative solutions in seconds.

However, it's important to note that the responsibility of content expertise still lies with the instructor. Checking and verifying what AI provides remains a critical element, and instructors should view AI as a tool that enhances rather than replaces their role as responsible experts in education.

Is there a project initiative that you've been part of recently and have implemented one or couple of these trends to make that successful?

I am currently engaged in practical projects focused on assessing learning at the university scale, a complex task requiring fundamental processes. My work involves establishing fundamental processes for effective assessment. In addition to these efforts, I have initiated a project applying generative AI to develop learning outcomes and performance descriptors based on a clear understanding of valued student demonstrations.

I aim to encourage instructors to consider not just content, which tends to be forgotten, but also the enduring skills that students retain. I want to involve instructors in discussions about what we truly value in student learning. By leveraging artificial intelligence, we can derive performance descriptors, streamline the creation of rubrics and assessments, and free up cognitive efforts for focusing on desired student skills.

The initiative seeks to engage educators in a conversation about how AI can be practically, ethically, and effectively applied to enhance the learning experience. While AI can streamline certain processes, the goal is to maintain analog foundational processes crucial for effective learning while encouraging ethical and effective AI use by students.

Any specific piece of advice that you'd like to share with your fellow peers or other industry leaders?

Navigating the complexity of human learning, institutional structures, and bureaucratic processes is a challenging task. My guiding principle is to prioritize things that truly aid student learning and aligns with our values. In the realm of teaching, simplicity and directness are key. While we aim to support instructors effectively, teaching is both an art and a science, leaving room for creativity and innovation. Ultimately, we must focus on the core essentials and let our values guide what we want students to gain from their educational experience with us.