EDC Progress Report June 2007

The research carried out by Education Development Center, Inc. (EDC) focused on helping middle-grades students improve reading comprehension in social studies by building word knowledge through the use of a web-based tool, Visual Thesaurus (VT). VT allows the reader to look up a word directly from a piece of electronic text. It does not directly define words; rather it presents an array of concepts that capture the varied word meanings. The features of VT align with the following e-text resources: translational, illustrative, enrichment, and collaborative. From August, 2006 to May 2007, EDC carried out a series of four brief design studies, with each one building on findings of the previous study. Read the executive summary and download the June 2007 progress report of research on the Visual Thesaurus:  Progress Report June 2007

Download the June 2007 progress report of research on the Visual Thesaurus:  Progress Report June 2007

 

The research carried out by Education Development Center, Inc. (EDC) focused on helping middle-grades students improve reading comprehension in social studies by building word knowledge through the use of a web-based tool, Visual Thesaurus (VT). VT allows the reader to look up a word directly from a piece of electronic text. It does not directly define words; rather it presents an array of concepts that capture the varied word meanings. The features of VT align with the following e-text resources: translational, illustrative, enrichment, and collaborative.

From August, 2006 to May 2007, EDC carried out a series of four brief design studies, with each one building on findings of the previous study. All of the studies addressed variations on the following four questions:

1. What characteristics of VT text facilitate or impede access to and learning of social studies content?
2. Does the use of VT improve learning of academic content in actual educational settings with typical resources and levels of teacher support?
3. What student characteristics (e.g., disability, technology skills) and components of the tool influence the effectiveness of use of VT?
4. What contextual factors (e.g., teacher training, hardware resources, and student groupings) influence the effectiveness of supported e-text?

The studies were all carried out at a large (893 students) middle-grades school (grades 6 to 8) with a diverse student population. Newly renovated, the school was well-equipped: SMART Boards in every classroom, several rolling laptop carts (1:6 student to computer ratio), and wireless Internet access. Below we describe each study.

STUDY 1 (AUGUST, 2006)
Two researchers each worked individually with four summer school students (three 6th grade students, one 7th grade student) with cognitive delays who were participating in a remedial summer-school program led by a reading specialist. They worked with the students across two days, with two hour sessions each day. The goal of this design study was to determine whether the students could use VT and find it appealing, whether they could use it to understand words in a passage, and which features of the tool were most and least useful for this population of students.  We gathered data by writing field notes of observations and transcribing audio taped sessions.
The findings indicated that low-level readers with cognitive difficulties found the tool difficult to use without a tremendous amount of adult input. The very features that make it appealing as an interactive tool seemed to make it overwhelming for low-level readers.

STUDY 2 (OCTOBER, 2006)
To continue to explore the use of the tool with older students who were better readers (although still struggling with comprehension), three researchers each worked with two 8th grade students who could decode but who had trouble with comprehension. We used an electronic version of content from their general education social studies text. 
In working with these students, we found that:

• All six were highly motivated to use VT.
• All quickly learned how to use the tool after the first demonstration.
• Setting the bar to have prerequisite decoding skills was critical.
• It was critical to put the words and meanings back in context to support comprehension.
• Students needed direct instruction and support (e.g., pointing out contextual information, encouraging the students to use the definitions in context, asking prompting questions, and helping students to link pieces of information).

STUDY 3 (NOVEMBER, 2006 – JANUARY, 2007) 
Based on the previous set of findings, Study 3 involved one 8th grade social studies class of 20 students, 11 of whom were identified as struggling comprehenders by the social studies and special education teachers who co-taught the class. The instructional strategies and supporting materials focused on understanding vocabulary within the context of the social studies text. For this study, the researchers taught the lessons to the whole class using VT on the classroom's SMART Board, both to model use of the tool for the teacher and to create a "habit" of using VT with students.
We gathered data by administering pre- and post-tests of selected words before and after the unit, observing the teacher and the class, collecting student work (from struggling and non-struggling students), and tape recording student responses during lessons.
We found that as students became more familiar with VT, they became more engaged with the word development process. Students identified the following features of VT as being valuable: having the word read aloud, expanding the word's web, reading the definition of the word, and moving a different word to the center of the web.
All students made statistically significant gains in their scores from pre-test to post-test.

STUDY 4 (MARCH – MAY, 2007)
Study 4 was carried out in the same 8th grade classroom as Study 3 with the same electronic social studies text, but with a new chapter. We had planned to include a comparison classroom, but teacher-based circumstances prevented this planned design from occurring. In this study, the teachers, rather than the researchers, were responsible for teaching the lessons. Students (individually or in pairs) used VT on their own with laptop computers. This shift in approach required that the researchers provided VT training for the students. Teachers initially provided significant support in using the tool and completing the assignments, and then gradually phased out their involvement.
We used the following methods to collect data related to individual students' use of VT: a pre- and post-test for each section of chapter 17, collection of student worksheets, classroom observations, and post-study survey of students and teachers.
We carried out qualitative and quantitative analysis of our data in Study 4. Below we discuss some of our emerging findings in relation to the research questions.

• What characteristics of VT facilitate or impede access to and learning of social studies content?  The results of a student survey showed that students used the following features most frequently: expanding the word, reading the word's definition, hearing the word read aloud, and placing a new word at the center. These features corroborate the features that were identified by teachers and students in Study 3 as being most helpful.
• Does use of the VT improve learning of academic content in actual educational settings with typical resources and levels of teacher support?  The scores of all the students (including struggling students) improved from the pre-test to the post-test. The effect size was statistically significant. We recognized that without a control group we could not with certainty claim that students improved due to the use of VT.  We also collected additional comparison data by designing one final pre/post test for the subsequent chapter, without using VT. The results of these post-study pre- and post-tests show that students' scores increased more for the chapter that used VT than for the one without its use. We understand that this comparison could suffer from challenges associated with internal validity, but we believe that students' score increases from one section to another show clearly that VT was instrumental in helping them make contextual meaning of their social studies vocabulary.
• What student characteristics (e.g., disability, technology skills) and components of the tool influence the effectiveness of use of VT? We found that searching for images, expanding the word, and hearing the word read aloud were instrumental in improving student understanding of the concept, particularly for those words that have more than one meaning. This was especially helpful for students who struggled with reading comprehension.

 
FUTURE RESEARCH DIRECTION
We identified the following areas for future research:

1. Focus on two features: "search for images" and "search the Internet" could have the potential to differentiate learning for students who are visual learners or who are not strong linguistic learners.
2. Explore the effects of a collaborative approach using VT versus students working independently with the tool.
3. Add a comparison class to address some of the challenges associated with internal validity that we had with these design studies.
4. Design a structured curriculum to directly enhance students' understanding of domain-specific vocabulary.