SoTL Scholar Ann Reimers Receives Best Paper Award for SoTL Research

Shocked—that was Engineering Assistant Professor Ann Reimers’ reaction to receiving the Best Paper Award from the American Society for Engineering Education’s (ASEE) Mechanics Division. Her paper, titled, “Modification of a Physics Rotational Kinematics Concept Inventory for use with Engineering Dynamics Students,” was the result of research she conducted as a participant in the Center for Teaching Excellence’s Scholarship of Teaching and Learning (SoTL) Scholars program.

Reimers joined the year-long program in May 2019 because she wanted to learn how to carry out academic research.

“I care deeply about the rigorous nature of doing research and I didn't want to just start doing educational research in a slapjack way. I wanted to be rigorous about it.”

Reimers’ SoTL project looked at an assessment tool published by the American Association of Physics Teachers and used by physics teachers across the country: the Rotational Kinematics Inventory. She had been using this concept inventory in her UVA courses to measure her students’ conceptual understanding of physics concepts.

“I started using it (the Rotational Kinematics Inventory) and I was noticing really weird results. Not what I would think. Rotational kinematics is a difficult subject and has traditionally been difficult for students in physics, but I was finding that even my top students, the ones who really knew everything solidly, were not getting the scores that I would have expected.”

After meeting with one of these students, she realized there might be room for improvement with the assessment tool.

“The question [the student] was asking me had everything to do with the way that the assessment was written and its graphics and its organization, and nothing to do with the concepts that the test should have been testing. So, that's where I got the idea. I thought, huh, maybe there are some changes that can be made to this so that we could make sure that what we're actually testing are the skills and the knowledge, and we're not taxing students in different ways or tricking them into getting false results because of the way that the assessment tool has been designed.”

With the support of the SoTL Scholars program and graduate student assistant Stefen Beeler-Duden, Reimers carried out her research.

“By changing the format and the content of the test in terms of its wording, its clarity, its ease of use … we were able to improve the reliability and consistency of a nationally normed, published, validated assessment tool for learning about the conceptual knowledge of physics and engineering students as related to rotational kinematics.”

Reimers’ results have far-reaching implications for physics education.

“I think I have a pretty good argument as to why they (American Association of Physics Teachers) should change the format of the test to the way that I formatted it instead of what is currently published and used as far as I know across the country by all physics teachers.”

And around the world, too, she added.

Reimers’ paper, co-authored by Beeler-Duden, was accepted to the ASEE Annual Conference and she presented her research virtually on Tuesday, June 23. She was also honored with the Mechanics Division’s Best Paper Award at a banquet.

“Even though the work itself was done on something very specific, having won Best Paper is going to allow me to hopefully broaden the conversation about the use concept inventories, their value, how to use them, and how to make sure you're assessing what you want to assess.”

After going through the SoTL Scholars program, Reimers is now confident in her ability to carry out future educational research.

“I'm no longer intimidated about things like applying for IRB; it's a confusing process, but I understand it now and I could do it again. I understand the whole branch of scholarly research into educational techniques … I feel like I can now contribute to the Scholarship of Teaching and Learning in a meaningful way.”

She encourages other faculty to make the time to join the program if they’re interested in doing research on their teaching or students’ learning.