Development of a Measure of Racial Equity-Based Social Emotional Learning Practices (REQSEL)

Summary:  View help for Summary Transformative SEL refers to the integration of an explicit equity and social justice lens into the conceptualization and implementation of social and emotional learning (SEL). As Jagers, Rivas-Drake, and Williams (2019) explain, it is an approach to SEL aimed at interrupting the reproduction of inequitable educational environments by attending to identity, power, discrimination, and social justice. Existing measures cannot be used to capture practices aligned with transformative SEL due to their focus on general competencies and SEL instructional practices. Based on our theory, prior research, experience with practitioner professional learning communities, and an ongoing research-practice partnership that includes a mixed-method study of SEL, ethnic-racial assets, and emerging civic development in early adolescence, we have begun developing a measure to assess SEL practices that are aligned with transformative SEL principles and thus can be used to promote rather than hinder racially equitable learning environments. We are engaging in a mixed-method study to further develop, refine, and disseminate this measure. We aim to: 1) iteratively develop a reliable, ecologically valid, and psychometrically sound measure of racial equity-oriented SEL practices and 2) field test the measure in large districts that serve racially marginalized youth.  

This project contains these related studies:

• Study 1. REQSEL Measure Development EFA Qualtrics Sample 1 (May 2021)
• Study 2. REQSEL Measure Development EFA Qualtrics Sample 2 (July/August 2021)
• Study 3. REQSEL Measure Development CFA-Validation Qualtrics Sample (July/August 2021)
• Study 4. REQSEL Measure Development CFA Field Sample Wave 1 (Fall 2021)  

PREVIOUS PROJECT WORK This measure development project follows Boateng et al.’s (2018) nine-step protocol for scale development and validation. Prior to Study 1, steps 1 (item generation) and 2 (content validity) have been completed. Concurrent with Study 1, step 3 (pre-testing) was conducted. Below is a summary of those steps:

1.Initial item generation: A team of four researchers drew on knowledge from SEL practitioners and our own preliminary knowledge of theory, research, and prior experience as K-12 classroom teachers to brainstorm a pool of 248 potential survey items. Over nine meetings and through a consensual decision-making process, the research team revised the brainstormed items to arrive at 150 draft items across six themes: Engaging with Ethnic-Racial Identity, Navigating Intergroup Relations, Racial Injustice, Xenophobia, Student Agency, Voice, and Power; and Racially Equitable Discipline. We also conducted two focus groups with 5th-8th grade teachers to identify relevant content and confirm the relevance of the themes. Over an additional seven meetings, we further refined the items and arrived at 139 items across the six themes.

2a. Content validity with expert advisors: We solicited and received individual written feedback from six field experts (selected based on their familiarity with the content) who commented on the content relevance, representativeness, and technical quality of the items. Through this process, we further reduced the number of items to 80 across the six themes.  

2b. Content validity with target population (i.e., teachers): We conducted a survey of 60 5th-8th grade teachers familiar with SEL to collect individual written feedback on the set of 80 items to determine face and ecological validity and clarity. Through this process, we further reduced the set of items to 73 across 7 domains: Engagement with Ethnic/Racial Identity (7 items), Language (3), Navigating Intergroup Relations (8), Racial Injustice (24), Racially Equitable Discipline (6), Student Agency, Voice, & Power (14), and Xenophobia (11).  

3. Pre-testing. Concurrent with Steps 4, 5, and 6 (Survey Administration, Item Reduction, and Extraction of Factors), which is described in our Study 1, we are also conducting Step 3 (Pre-testing). Step 3 consisted of conducting cognitive interviews with five middle school teachers who had taught SEL in the past 3 years at an urban school and taught primarily students of color (at least 50% of their students were non-white). Participants were recruited by sending an email and flier advertising the study to research team members’ contacts. Seven people responded to the request. One was eliminated because they indicated their students were more than 50% white, and another was eliminated because they did not teach in an urban setting. The five participants were 3 men and 2 women, 3 white and 2 Asian, 2 masters degrees and 3 bachelors, 4 public and 1 charter school, 4 with between 2 and 5 years teaching experience and 1 with 20 years.

Interviews were conducted by one research team member and followed the following format: Participants were read each survey item aloud. They answered the survey item using the provided response scale. Then the interviewer asked two follow-up questions: 1) What do you think this question is asking? (To gauge their comprehension of what the question was meant to ask.), and 2) How did you arrive at your answer? (To assess what experiences they were drawing on in response to the question.) After all of the individual questions had been discussed, participants were asked to share any general feedback on the measure. Using proactive, standardized probes in this manner reflects a combination of the think-aloud and the probing techniques common to cognitive interviews (Beatty & Willis, 2007).

All research team members listened to all five interviews and recorded their notes for each item about whether or not the participant understood the question as it was intended and whether they drew on reasonable evidence to arrive at their response. The research team then met to discuss their impressions from the interviews and to revise the items based on the participants’ responses. Based on this feedback, we dropped 7 items, and revised the wording of 11 items. The resulting survey had 51 items in 4 domains: Awareness of Diverse Ethnic/Racial Identities and Experiences (8 items), Racial Injustice (19), Student Agency, Voice, & Power (12), and Xenophobia (12).

Funding Sources:  View help for Funding Sources William T. Grant Foundation (188925); Spencer Foundation

Subject Terms:  View help for Subject Terms social and emotional learning; racial equity; teaching practice

Geographic Coverage:  View help for Geographic Coverage United States

Time Period(s):  View help for Time Period(s) 5/20/2021 – 5/27/2021 (Study 1. REQSEL Measure Development Exploratory Factor Analysis)

Collection Date(s):  View help for Collection Date(s) 5/20/2021 – 5/27/2021 (Study 1. REQSEL Measure Development Exploratory Factor Analysis)

Universe:  View help for Universe Study 1. REQSEL Measure Development Exploratory Factor Analysis: Fifth-eighth grade teachers in the United States who are familiar with social and emotional learning (SEL), recruited through a Qualtrics panel. 

Study 2. REQSEL Measure Development Test-Retest Reliability, Confirmatory Factor Analysis, and Construct Validity: Fifth-eighth grade teachers in the United States who are familiar with social and emotional learning (SEL), recruited through a Qualtrics panel. 

Study 3. REQSEL Measure Development Field Test: Fifth-eighth grade teachers in the United States who are familiar with social and emotional learning (SEL), recruited through CASEL's partner districts, National Equity Project partner districts, and the BELE Network. 

Data Type(s):  View help for Data Type(s) survey data

Response Rate:  View help for Response Rate Study 1. REQSEL Measure Development Exploratory Factor Analysis: 385 individuals were contacted through a Qualtrics panel for the survey. Of these contacts, 129 were ineligible to participate because they reported having no familiarity with social and emotional learning (59), did not teach 5th-8th grade (23), or did not have between 1 and 50 years of teaching experience (2). Additionally, 34 responses were removed for quality control because they were completed in less than 260 seconds, and 16 were incomplete.  Eleven respondents were rejected because the quota for the survey had already been met at the time they began the survey. There were a total of 240 completed surveys, a response rate of 84 percent. 

Sampling:  View help for Sampling Study 1. REQSEL Measure Development Exploratory Factor Analysis: Used a Qualtrics Teacher Panel.

Collection Mode(s):  View help for Collection Mode(s) web-based survey

Scales:  View help for Scales Study 1. REQSEL Measure Development Exploratory Factor Analysis: Because the aim of this project is to develop new scales to measure teaching practices for racial equity-based social and emotional learning (SEL), we did not include existing scales. All of the items in the REQSEL measure are Likert-style items requesting the frequency of the use of the teaching practice during SEL instruction. 

Unit(s) of Observation:  View help for Unit(s) of Observation Individuals