By:
Jessica Krim, Ed.D., Associate Professor, Department of Teaching and Learning, Southern Illinois University Edwardsville
Laleh E. Cote, Senior Internship Coordinator; Doctoral Student, Lawrence Berkeley National Laboratory; University of California Berkeley
Elisa M. Stone, Ph.D., CalTeach Program Director, University of California Berkeley
Renee Schwartz, Ph.D., Professor, Department of Middle and Secondary Education, Georgia State University
John Keller, Ph.D., Director, Fiske Planetarium; Professor, Astrophysical and Planetary Sciences, University of Colorado Boulder

As today’s world continues to be shaped by science and technology, there is a pressing need to improve public understanding of what constitutes “science” (National Research Council [NRC], 2012). Despite decades of recommendations to involve learners in scientific activities that model authentic science, K–12 teachers still struggle to integrate scientific practices across their curricula (Capps & Crawford, 2013; Crawford, 2014). One commonly stated reason for teachers’ continued challenges is the lack of firsthand scientific research experience, and so multiple programs have engaged teachers and future teachers in teacher research experiences (TREs), (Schwartz & Crawford, 2004; Sadler, Burgin, McKinney, & Ponjuan, 2010), including the National Science Foundation (NSF) funded Research Experiences for Teachers (RET) programs.
There have been calls to action for more intentional program and course design, systematic research targeting TREs (including NSF RET programs and teacher programs funded by other sources), as well as parallel efforts such as undergraduate research experiences (UREs) and course-based undergraduate research experiences (CUREs). For more information, see comprehensive reviews of published studies by Sadler et al., 2010; Corwin, Graham, & Dolan, 2015; Linn, Palmer, Baranger, & Stone, 2015 and reports by Auchincloss et al., 2014 and NASEM, 2017.
Building on this work, the Collaborative Around Research Experiences for Teachers (CARET) has published a comprehensive literature review (Krim et al., 2019) to gain an understanding of reported TRE, URE and CURE program features, targets, and outcomes. In this blog, we will focus on our analysis related to TREs and implications for K-12 education. We examined relevant papers published between 2007 and 2017 to capture a “state of the field” with respect to types of programs, participants, program elements, assessment measures, outcomes, and theoretical frameworks. Because many undergraduate students in UREs and CUREs later become science teachers, what they learn in these research experiences is critical for their future careers as teachers. Thus, recent studies on UREs and CUREs were included in our analysis.
Methodology
Considering the frameworks of situated learning (Lave & Wenger, 1991; Wenger, 1998), and communities of practice (Crawford, 2014), and the results of prior literature reviews, we developed the CARET model. This model posits that teachers or aspiring (preservice) teachers who engage in STEM research or STEM industry experiences will demonstrate shifts in professional and pedagogical practices and identity/self-efficacy. The CARET model was developed by a collaborative research team to provide guidance in the literature search and review, and is not intended to be inclusive of all impactful features or strategies associated with TREs; rather the model serves as a starting point to articulate, challenge, and refine our understanding of TREs as reported in the literature.
An initial coding scheme was developed by building on categories used by Linn et al. (2015) on undergraduate research. We used this refined coding guide to analyze 307 relevant papers (TREs from 2007-2017, CUREs and UREs from 2014-2017, since the last published review in these areas) and reduced this to a smaller group of 177 papers published only from 2014-2017 for deeper analysis and comparison. To meet the call from Linn et al. (2015) for more empirical studies, and because the distinction between empirical and program evaluation was not always clear, we grouped the two categories that included analysis of data (empirical and program evaluation), terming the study type for these 177 papers as “data driven.” We conducted comparisons using descriptive statistics within and across program type (CURE, URE, TRE). In the “Results” section we present our analysis of the focal 177 papers, because they have the most potential to provide descriptions as well as evidence-based claims that may be translatable to other programs in the field.
Results
Our findings suggest a lack of studies explicitly targeting: 1) participation and outcomes related to learners from underrepresented populations, 2) methods for translation of research experiences into K–12 instructional practices, and 3) measurement of impact on K–12 student learning.
A number of prior studies have pointed to the importance of considering how participants from underrepresented groups experience scientific research experiences, as experiences and outcomes for undergraduate students and teachers from underrepresented groups may differ from those of other program participants (Jones, Barlow, & Villarejo, 2010; Junge, Quinones, Kakietek, Teodorescu, & Marsteller, 2010; Ovink & Veazey, 2011; Schwartz, 2012; Slovacek, Whittinghill, Flenoury, & Wiseman, 2012; Stevens & Hoskins, 2014; and Linn et al., 2015). Thus, our analysis specifically sought mention of, and disaggregated outcomes for, participants from underrepresented groups. Papers were coded by author identification of study and program populations (underrepresented groups include women, persons of color, and persons with disabilities in science and engineering). 80% of the papers reported more than 20 participants in the study, and participant numbers in studies ranged from less than 20 to greater than 1000. Of the 177 studies, 114 (64%) failed to mention the involvement of teachers or undergraduate students from underrepresented groups (Table 1). While 47 studies (26%) identified the number of participants from underrepresented groups in the program and study, they did not report findings specific to these participants. Only 17 (10%) reported an intentional focus on participants from underrepresented groups. This general trend repeated across all categories of research experiences. Notably, the majority of TRE studies did not mention underrepresented groups, and there were no TRE studies that focused explicitly on participants from underrepresented groups.
Table 1
Number of Studies by Participant Outcome Data and Program: Underrepresented (UR) Populations |
||||||
Outcome Data |
TRE
n=22 |
CURE
n=72 |
URE
n=65 |
Combination
n=9 |
Other
n=9 |
Total
n=177 |
UR focus | 0 (0%) | 1 (1%) | 15 (23%) | 0 (0%) | 1 (11%) | 17 (10%) |
UR identified | 4 (18%) | 13 (18%) | 23 (35%) | 3 (33%) | 3 (33%) | 47 (26%) |
UR not mentioned | 18 (82%) | 58 (81%) | 27 (42%) | 6 (67%) | 5 (56%) | 114 (64%) |
The three previous reviews presented well-organized lists of reported measured outcomes. Authors included similar reported outcomes in the coding scheme used here (Table 2). Across all programs, the most frequently reported outcomes were improved science practices (38%), laboratory skills (35%), disciplinary content knowledge (34%), and confidence (31%). Among the TRE studies, these four outcomes were relatively sparse, with “impacts on classroom practice” being the most targeted outcome (60%). Surprisingly, only 23% of TRE studies focus on K–12 student outcomes, a stated long-term goal for most TRE programs (see “Conclusions”).
Table 2
Number of Studies by Reported Measured Outcomes and Program (2014-2017) |
||||||
Measured outcomes* | TRE
n=22 |
CURE
n=72 |
URE
n=65 |
Combination
n=9 |
Other
n=9 |
Total
n=177 |
Not stated | 0 (0%) | 1 (1%) | 3 (5%) | 0 (0%) | 1 (11%) | 5 (3%) |
Performance† | 0 (0%) | 13 (18%) | 13 (20%) | 1 (11%) | 1 (11%) | 28 (16%) |
Content knowledge‡ | 5 (23%) | 35 (49%) | 17 (26%) | 3 (33%) | 0 (0%) | 60 (34%) |
NOS | 5 (23%) | 12 (17%) | 10 (15%) | 0 (0%) | 0 (0%) | 27 (15%) |
Persistence† | 0 (0%) | 12 (17%) | 24 (37%) | 3 (33%) | 3 (33%) | 42 (24%) |
Science practices | 5 (23%) | 33 (46%) | 25 (38%)§ | 2 (22%) | 2 (22%) | 67 (38%)§ |
Lab skills | 1 (5%) | 36 (50%)§ | 20 (31%) | 4 (44%)§ | 0 (0%) | 61 (35%) |
21st century skills | 1 (5%) | 15 (21%) | 10 (15%) | 1 (11%) | 0 (0%) | 27 (15%) |
Self-efficacy | 9 (41%) | 13 (18%) | 15 (23%) | 2 (22%) | 4 (44%)§ | 43 (25%) |
Confidence | 4 (18%) | 26 (36%) | 19 (29%) | 3 (33%) | 3 (33%) | 55 (31%) |
Attitudes/interest† | 4 (18%) | 16 (22%) | 19 (29%) | 4 (44%)§ | 1 (11%) | 44 (25%) |
Teacher identity | 1 (5%) | 2 (3%) | 0 (0%) | 1 (11%) | 0 (0%) | 4 (2%) |
Scientist identity | 2 (9%) | 8 (11%) | 9 (14%) | 1 (11%) | 1 (11%) | 21 (12%) |
Classroom practice | 13 (60%)§ | 1 (1%) | 0 (0%) | 1 (11%) | 2 (22%) | 17 (10%) |
K-12 outcomes | 5 (23%) | 0 (0%) | 0 (0%) | 0 (0%) | 0 (0%) | 5 (3%) |
Perceptions | 4 (18%) | 2 (3%) | 0 (0%) | 0 (0%) | 1 (11%) | 7 (4%) |
Awareness† | 3 (14%) | 0 (0%) | 12 (18%) | 1 (11%) | 0 (0%) | 16 (9%) |
Leadership | 0 (0%) | 0 (0%) | 3 (5%) | 1 (11%) | 0 (0%) | 4 (2%) |
*Performance includes course grades and/or grade point average. Perceptions refers to teachers and teaching. NOS, nature of science. †Pertains to STEM careers. ‡Pertains to science content discipline knowledge. §Indicates the most frequent category in the group. |
Prior studies highlighted the need to examine the impacts of research experience programs through a variety of data sources. For example, Sadler et al. (2010) and Linn et al. (2015) found that most studies examined relied solely on self-report surveys or interviews. From our analysis, we suggest that this call has begun to be answered, with 57% of the 177 data-driven reports containing measures other than, or in addition to, self-report, a significant increase across CUREs, UREs, and TREs (Table 3). The most substantial proportion were TRE studies, which more frequently used measures beyond self-report data (73%), as compared with CURE (58%) or URE (46%) studies.
Table 3
Number of Studies by Data Type and Program |
||||||
Data Type* |
TRE
n=22 |
CURE
n=72 |
URE
n=65 |
Combination
n=9 |
Other
n=9 |
Total
n=177 |
Self-report only | 6 (27%) | 30 (42%) | 35 (54%) | 4 (44%) | 1 (11%) | 76 (43%) |
More than self-report | 16 (73%) | 42 (58%) | 30 (46%) | 5 (56%) | 8 (89%) | 101 (57%) |
Institutional/extrinsic | 0 (0%) | 14 (19%) | 21 (32%) | 2 (22%) | 3 (33%) | 41 (23%) |
Quant. participant† | 12 (55%) | 42 (58%) | 37 (57%) | 5 (56%) | 5 (56%) | 101 (57%) |
Qual. participant† | 14 (64%) | 40 (56%) | 27 (42%) | 6 (67%) | 1 (11%) | 88 (50%) |
Quant. faculty/mentor† | 0 (0%) | 4 (6%) | 3 (5%) | 1 (11%) | 0 (0%) | 8 (5%) |
Qual. faculty/mentor† | 1 (5%) | 2 (3%) | 2 (3%) | 1 (11%) | 0 (0%) | 6 (3%) |
Interview‡ | 12 (55%) | 8 (11%) | 14 (22%) | 0 (0%) | 1 (11%) | 35 (20%) |
Content/practice | 8 (36%) | 27 (38%) | 5 (8%) | 1 (11%) | 1 (11%) | 42 (24%) |
Other | 8 (36%) | 10 (14%) | 6 (9%) | 1 (11%) | 1 (11%) | 26 (15%) |
*The 177 data-driven papers were independently coded for self-report only vs. more than self-report, and type of data collected (survey, interviews, content/practice assessment, other). A paper could receive multiple codes for type of data collected. †Self-report survey, Quant, quantitative data (e.g., multiple choice, checkboxes) collected via survey; Qual, qualitative data (e.g., open-ended responses, essay questions) collected via survey. ‡ Interview includes focus groups. |
Conclusions
Despite the findings of Nagda, Gregerman, Jonides, von Hippel, & Lerner (1998), Davis (1999), Sadler et al. (2010), NASEM (2017), and Linn et al. (2015) and the intentions of funding agencies to support programs that address the inclusion of underrepresented populations in STEM fields, we found that the majority of studies about CUREs, UREs, and TREs published between 2014 and 2017 fail to report demographic data that would identify the proportion of participants from underrepresented populations. They also fail to disaggregate outcomes specifically for underrepresented students and/or teachers.
Nine papers from our review stood out as not only involving data analysis for underrepresented groups but also presented findings that we consider particularly relevant for researchers of future studies and program designers (Miranda & Damico, 2015; Robnett, Chemers, & Zurbriggen, 2015; Shapiro et al., 2015; Griffeth et al., 2016; Haeger & Fresquez, 2016; Remich, Naffziger-Hirsch, Gazley, & McGee, 2016; Carpi, Ronan, Falconer, & Lents, 2017; Katz et al., 2017; Ghee, Keels, Collins, Neal-Spence, & Baker, 2018). A noteworthy example is from Carpi et al. (2017), who interviewed faculty mentors and student participants and suggested the value of an extended mentorship during URE participation (1–3 years), especially for participants from underrepresented groups, for increasing graduation rates and the number of students earning advanced degrees.
Recent work offers additional insights into impacts of science research experiences on underrepresented participants. The longitudinal study by Hernandez, Woodcock, Estrada, & Schultz (2018) found that the duration and intensity level of a research experience impacted underrepresented student persistence in STEM, stating that undergraduate research was impactful only if students engaged in research for at least 10 hours/week for two or more semesters.
A primary goal of TREs are “to equip teachers with an understanding of and a capability to conduct scientific research that will transfer to their science classrooms” (Sadler et al., 2010, p. 242). The literature also shows the importance of program elements that provide specific and supported opportunities for teachers to translate their research experiences into classroom instruction (Schwartz & Crawford, 2004; Sadler et al., 2010). We demonstrate that outcomes related to transfer of science research knowledge to classrooms and K–12 student learning are not being measured sufficiently. Also, there has been insufficient attention to outcomes such as teacher identity or perceptions of the teaching profession.
Several studies applicable to TREs outcomes are of particular note and serve as a model for future TRE studies. A study by Hanauer et al. (2017) identified measures of project ownership, scientific community values, science identity, and science networking that reflected persistence in science. Enderle et al. (2014) identified considerable impact for those teachers looking to reform their instruction and choosing programs explicitly designed for pedagogical development. Southerland et al. (2016) found that providing personal relevance and social engagement in the research context increased investment, and that TRE participation shapes science teaching beliefs, in turn influencing practice. Similarly, Miranda & Damico (2015) found that a summer TRE followed by participation in an academic year-long professional learning community can help teachers to shift their beliefs surrounding pedagogical approaches; however, documentation of classroom practices that evidence this shift is limited.
Implications
There are particular implications for researchers and program designers that can be drawn from our review:
- Researchers – We join others in reiterating the call for more program development and research studies that: 1) purposefully target and support diverse participants and 2) rigorously collect, analyze, and report data that reflect outcomes for underrepresented populations.
- Preparation Program Designers – To answer the question “Is this TRE for me?,” we recommend program designers deeply consider their goals, which are not mutually exclusive. To impact science instruction, programs need to rigorously support teachers in translating research experiences into classroom practices. To promote development of science practices and content, programs should engage participants in extended authentic research experiences. Finally, program designs must provide purposeful and supportive components that enhance science identity and sense of belonging for women and people of color.
Acknowledgements: This work was made possible by a collaboration grant from 100Kin10 to CARET to attend meetings and conferences; funding from the APLU Network of STEM Education Centers; in-kind support from the M.J. Murdock Charitable Trust; and several NSF grants, including the Robert Noyce Teacher Scholarship Program, Research Experiences for Teachers, Graduate Research Fellowship Program, and Improving Undergraduate Science Education. This material is based upon work supported by NSF Grants Nos. 1524832, 1340042, 1540826, 1542471, 1106400, and 1712001. We thank these organizations for their generous support and extend a special thank you to Bruce Johnson, College of Education, University of Arizona for his guidance and contributions.