Simulations as a Platform for Understanding and Improving Teachers’ Classroom Skills

By: Julie Cohen, Ph.D., Associate Professor of Curriculum & Instruction, University of Virginia
Vivian Wong, Ph.D., Associate Professor of Research, Statistics, & Evaluation, University of Virginia
Anandita Krishnamachari, Ph.D., Research Scientist, University of Virginia
Nathan Jones, Ph.D., Associate Professor of Special Education, Boston University

The Challenge

Beginning teachers need a lot of practice to develop classroom skills. We consistently see dramatic improvement in teachers’ impact on student outcomes during their first few years in the classroom (Atteberry et al., 2015; Kraft & Papay, 2014). However, this on-the-job learning is stressful for new teachers, and is associated with high levels of teacher burnout, attrition, and negative outcomes for students (Gavish & Friedman, 2010; Papay & Laski, 2018; Skaalvik & Skaalvik, 2017). A central question is, how can we move some of this rapid skill development into pre-service teacher education (Ball & Forzani, 2009; Grossman, Hammerness & McDonald, 2009)? Teachers who start their careers with a solid foundation in critical instructional skills would be better poised to stay in the classroom and contribute to positive student outcomes.

At the University of Virginia (UVA), we have prioritized these practice opportunities in our preparation program by having preservice teachers work on key classrooms skills in the context of mixed reality simulations. The simulations are “mixed” because pre-service teachers (called “candidates”) work in virtual classroom spaces that are remotely controlled by actors who are trained to engage in realistic classroom interactions (Diekker et al., 2013; See Figure 1). This kind of simulated practice is designed to complement —not replace—traditional forms of practice, including “student teaching” in K-12 classrooms. Though practice in real classrooms is invaluable, it also presents some limitations (Clift & Brady, 2005; Grossman et al., 2011). Candidates may not have chances to try out all the skills they will need as teachers of record. Mentors also vary in the degree to which they model strong teaching, sometimes even countering what is learned in coursework (Feiman-Nemser & Buchman, 1985; Grossman et al., 2009; Ronfeldt, 2015). In addition, not all mentors provide candidates with the feedback they need to improve (Matsko et al., 2020).

Our program of simulated practice is designed to provide candidates the flexibility to experiment, learn, and reflect on their teaching skills in an environment that is more controlled and less complex than a real classroom (Grossman et al., 2009). In addition to building a curriculum of carefully sequenced simulations, we have also focused on providing targeted feedback through coaching, and then studying the causal effects of such efforts. Although coaching can improve in-service teachers’ feelings of self-efficacy, instructional skills, and student achievement (Desimone & Pak, 2017; Kretlow & Bartholomew, 2010; Stahl et al., 2016), it is rarely used during the pre-service period when candidates’ skills are rapidly developing and ideas about “good teaching” are less well-defined (Ericsson & Pool, 2016). We have conducted nearly a dozen randomized controlled trials to understand the contexts and conditions in which coaching is more and less helpful at expediting skill development (Cohen et al., 2020; Cohen & Wiseman, 2019; Cohen, Krishnamachari, & Wong, 2021).

Simulations as a Practice Space and Assessment Platform

Over the past five years, we have created a simulation curriculum, TeachSim, that is fully integrated with coursework. The simulations play two roles. First, they provide opportunities for candidates to practice and to receive coaching on teaching skills covered in concurrent methods coursework. Second, the simulations provide a standardized assessment platform that allows us to observe candidates’ skill development over the course of their preparation program. Because simulation sessions can be delivered in similar ways across large groups of candidates, they also allow us to randomly assign and to evaluate pedagogical supports such as coaching in ways that are typically infeasible during the teacher preparation period.

Our randomized control trials follow a similar data collection procedure (see Figure 2). Candidates engage in “baseline” simulations when they start the program. Then, at set intervals, candidates practice different teaching tasks – which range from providing actionable feedback during a text-based discussion to respectfully and succinctly redirecting off-task student behaviors to facilitating a parent conference. During these sessions, candidates are randomly assigned to receive coaching or self-reflection prompts before practicing the teaching task again. We evaluate the causal effect of different kinds of coaching on standardized measures of candidates’ pedagogical performance, as well as candidates’ own assessments of student behavior (Cohen et al., 2020). Finally, candidates complete a set of “exit simulations” prior to graduation.

Figure 2: Data Collection Procedure for Individual RCT Studies

A longitudinal data system also links candidates’ performance in the simulator with scored observations of their instruction in student teaching placements, along with their background characteristics and experiences entering the program (e.g., educational history, content knowledge for teaching, attitudes towards teaching, and self-efficacy). We have also surveyed nearly 500 candidates about their perspectives on the benefits and drawbacks of simulated practice. Taken together, these data help us answer a wide range of questions about teacher learning in the pre-service period.

What We Have Learned and Implications for Research and Practice

The majority of our candidates appreciate the extra practice opportunities, saying that the simulation sessions are useful, relevant, and authentic (Sebastian & Cohen, 2021). Candidates particularly appreciate receiving immediate coaching and feedback and having the opportunity to try challenging classroom scenarios multiple times. They repeatedly mention the value in being the “lead” teacher in ways that are difficult to replicate in student teaching placements.

We also see general maturation effects across candidates. That is, candidates seem to improve meaningfully over the course of our teacher preparation program. However, some skills develop much more readily than others. Observational results from our “parent engagement” simulations suggest candidates continue to struggle to communicate effectively with adult avatars across the preparation period, making “teacher-parent communication” a high-leverage area to target in teacher education (Lightfoot, 2004; Sebastian & Datta, 2021).

In addition, we find that providing participants with directive coaching significantly improves their observed teaching skills like redirecting off task behavior or providing feedback to students in a discussion (Cohen et al., 2021; see Figure 3). Coaching effects are robust across different teaching tasks and different cohorts of teacher candidates. Moreover, coaching appears to leverage large and statistically meaningful improvements in candidates’ skills, regardless of whether coaching is provided in person or over Zoom. Practice and self-reflection alone are consistently less effective at promoting teaching skill development. In addition, candidates who self-reflect between simulations are significantly more likely to assess minor, off-task student behavior as severe and meriting exclusionary disciplinary practices, including suspension and expulsion (Cohen et al., 2020). We also find some meaningful relationships between measures of teaching in simulated classrooms and student teaching classrooms, though these associations vary depending on the particulars of the focal skill and the classroom observation measures used (Boguslav & Cohen, 2021).

Figure 3: Coaching Effects Across Replication Studies

Collectively, our studies provide encouraging evidence that teacher preparation can be an important time for rapid skill development, especially when candidates are given targeted and ongoing practice opportunities and corresponding feedback and coaching. Though we often think that practice must happen in live classrooms, our research provides evidence that practice opportunities can be effectively integrated into university settings (Ball & Forzani, 2009). Rather than waiting until candidates are in student teaching, practice during methods coursework can better prepare teacher candidates for skills with which they often report struggling (Grossman et al., 2009).

Next Steps for Research

Systematic Replication Across Diverse Samples of Teachers

An important next step in this work will be to examine whether these coaching effects in the simulation setting extend to other teacher education contexts with more diverse candidate populations. We are heartened by the findings at UVA but given the wide range of programs that prepare teachers, we need more information about the degree to which our practice and coaching supports bolster candidates’ skills more broadly.  At present, we are partnering with the University of Texas-Rio Grande Valley and Southern Methodist University/Teach For America-Dallas to examine the robustness of coaching effects across different populations of candidates, working in diverse geographic locations, and classroom settings. We also need to build a more robust evidence-base about correspondence between improved teaching in simulated classrooms and improvements in the more distal outcomes of teaching real children in real classrooms, which is also currently underway.

Supporting Students with Disabilities in Mathematics Classrooms

With support from the National Science Foundation (Grant #2009939), we are continuing to develop simulations to provide practice opportunities that candidates would not otherwise get in their programs. Our focus is better preparing elementary candidates to support students with disabilities (SWDs) in mathematics lessons. General education teachers are likely to receive minimal coverage of special education teaching methods in their coursework and have few practice opportunities focused on SWDs (Blanton et al., 2011; Blanton et al., 2018; Florian, 2012; National Center for Learning Disabilities, 2019). This concern is especially relevant in elementary mathematics; SWDs lag behind their peers in mathematics achievement as early as in kindergarten, and these gaps only grow over time (Judge & Watson, 2011; Schulte & Stevens, 2015; Wei, Lenz, & Blackorby, 2013). There are several promising mathematical teaching practices that could be leveraged to support SWDs, but general educators rarely have opportunities to develop these skills (see Fuchs et al., 2008; Gersten et al., 2009 for recent reviews).

Our current research aims to meet this challenge by integrating high-leverage practices for SWDs—and corresponding simulated practice into elementary mathematics methods courses. Specifically, we are developing a suite of portable and scalable curricular materials, simulations, and coaching supports, which we will test in randomized control trials across several teacher education programs. If the materials prove to be effective, it could provide a model for integrating special education methods and content into general education courses in mathematics. Such efforts could pay dividends for teacher preparation programs looking to enhance candidates’ skills without adding additional coursework. Given the brief duration of teacher preparation and the high number of courses required for elementary licensure in most states, this approach could be helpful in developing a wide range of needed skills, and its portability raises exciting possibilities for scale.

Acknowledgements

This research has been supported by the Bankard Fund for Political Economy at UVA, The Jefferson Trust at UVA, the Spencer Foundation/National Academy of Education, the Robertson Family Foundation, and the National Science Foundation under Grant #2009939. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of our generous funders, including the National Science Foundation.

For papers, presentations, and project details, please visit teachsim.org