OUR GOALS
Assessing the Cultural Validity of Science and Mathematics Assessments
A Project funded by the National Science Foundation
Dr. Sharon Nelson-Barber (Principal Investigator)
Dr. Guillermo Solano-Flores (Co-Principal Investigator and Project Director)
Summary
This project is funded by the National Science Foundation and is conducted by staff of the Culture and Language in Education Program, WestEd (formerly Far West Laboratory). Our goal is to contribute to attaining equitable testing in science and mathematics. We think that testing in science and mathematics may be not properly taking into consideration the fact that culture and society shape an individual's mind and thinking. We will investigate how students' everyday experience influences the way in which they interpret and solve science and mathematics test items.
Students will be asked to respond in writing to items from science and mathematics tests. The students will also be asked to complete a short demographic questionnaire. In addition, we will work individually with some students for no longer than thirty minutes. We will give them some science and mathematics exercises and ask them to "think aloud" as they engage in solving them. After completing each exercise, we will ask them to explain how they interpreted it and how they related its content to their personal experience.
From the results of this investigation, we expect to gain knowledge on how cultural diversity should be considered systematically in assessment development and testing practices.
Purpose and Use of Standardized Student Records
As a part of the activities of the project on cultural validity, we are asking schools to share with us the individual scores obtained by the students in standardized reading, mathematics, and science tests. We need to compare the individual students' scores in math and science tests, so as to be able to examine how each individual record differs within each subject area, depending on the method of the test used. This will help us to identify the types of tests that are sensitive to the different ways in which students form different cultural backgrounds can demonstrate their knowledge.
The information will be used only with research purposes. No decision or judgment about the students' performance will be made. All the information will be treated confidentially and the name of the students will never be associated to their scores.
What is the purpose of this project?
This project is funded by the National Science Foundation and is conducted by staff of the Culture and Language in Education Program at WestEd in Menlo Park, California. It is a study on the cultural validity of assessments in science and mathematics. The goals of this project are:
Why is this project important?
Linguistic and cultural minority students tend to score lower than white, mainstream students in state and national tests. We suspect that one reason for this gap is that each cultural group has intrinsic values, learning styles, and ways of demonstrating skills that are not properly considered in testing. State and national testing may be penalizing minority students because these cultural differences are not recognized. The results of our investigation will allow us to determine how culture can be taken into account in order to test all students more equitably.
Why is our participation important?
The population in your program has been identified as representative of one community among ten diverse cultural communities proposed in our national study. This investigation will explore the possibility that cultural influences that shape the way students interpret and solve science and mathematics problems are not being properly considered. These cultural influences may be one reason current tests do not provide accurate information about the academic achievement of cultural minorities.
We realize that school staff are very busy with their current responsibilities. We would like to emphasize the relevance of this study, its importance for policy making, and its potential impact on the instruction and assessment of all students, especially those from diverse cultural backgrounds. Your participation and collaboration are key in our efforts to raise awareness about the sensitivity of assessment instruments to cultural differences.
What does our participation involve?
We will ask the school administration to provide information on a socioeconomic questionnaire in areas such as availability of instructional resources, teacher annual income, teacher turnover and free lunch programs. We will also need to collect information on student demographics and participating students’ standardized test scores. In addition, we will ask teachers of participant students to provide information on the science and mathematics instruction they offer (e.g., number of hours of instruction per week, formal instruction as science and mathematics instructors, etc.). During our visit, we will ask fourth and fifth-grade teachers to set aside a 45-minute period for us to give a 12-item science and math test to their students.
Your role will involve working with us to identify the student participants and their schools; providing background information such as student demographic information and standardized test scores for student participants; and designating a person to serve as a community liaison.
Ideally, the community liaison will be a person from the same community as the students who is familiar with both the students and schools in your program. This person will help us to coordinate the testing and interviews and may be present during the interviews if needed. The community liaison will receive an honorarium for his or her participation.
What will the students be asked to do?
We will give 120 or more of your fourth and fifth-grade students a 6-item science and mathematics test and a questionnaire on activities they engage in at school and outside school. We will also interview at least 16 of your fourth and fifth-graders individually. Each of these 16 students will be given one item from the test and asked to "think aloud" as they solve it. They will also be asked to describe how they relate their science knowledge to their everyday personal experience. The community liaison you have designated may be present during these interviews. Each interview will take approximately thirty minutes.
How much time is involved?
Preliminary arrangements will be made with each institution and participating fourth and fifth grade teachers, so as to minimize interference with school activities during the collection of data. One class period will need to be designated for the students to respond in writing to items from science and mathematics tests and to complete a short demographic questionnaire. In addition, we will need about a half hour with each of the students interviewed individually.
Are the student responses confidential?
All the methods of data collection and scoring will ensure the privacy of students, parents and teachers. We will also ensure the confidentiality of the school and the school district. Students will not be identified by name in reporting the results of the study. Also, we will be sure to comply with state and school regulations that require written permission of the students’ parents or guardians prior to our testing and interviewing.
What will be done with the study results?
From the knowledge gained in this investigation, we will be able to inform decision makers in educational policy and testing about more equitable ways to develop assessments. Final report findings will be shared with your program.