July 21, 2006 Dear XXXXXXXXX Thank you for the information you have shared with us in regards to the Adult ESL Explicit Literacy Impact Study, a groundbreaking study designed to test the effect of a high quality, innovative curriculum on the English literacy and language development of low-literate adult ESL learners. The study is sponsored by the U.S. Department of Education’s Institute of Education Sciences, and is scheduled to begin in fall, 2007. This letter is to apprise you of the status of our program selection process, and to invite you to participate in the next phase of this process. The selection of programs that can implement the study is key to the study’s success and involves a multi-phase process to ensure that programs that are being considered meet study criteria. As you may recall, a project staff member interviewed you requesting information on whether your program has: a. Site enrollments of low-literate adult ESL literacy learners large enough to support the evaluation (i.e., enrollment of at least 90 low-literate adult ESL students per semester or instructional period). b. At least 4 classes of about 20–25 low-literate adult ESL learners that provide 120 hours of instruction per semester or other term, and meet at least two hours at a time. c. At least 5 trained adult ESL literacy instructors who teach the target classes. d. Two pairs (or all 4) of the classes meet at the same time and location. e. Managed enrollment or an enrollment process where a majority of learners enter during the beginning of a course. Based on our prior discussions, we believe that your program meets these criteria. We now would like you to participate in a conference call, which will be conducted by project staff, including one of our research partners. During this call, we will explain the study’s objectives and the benefits of your participation to your program. We will also review each criterion for inclusion in the study with you to confirm the information we collected earlier and to fill information gaps related to enrollment policies, class schedules, teacher qualifications, and the nature of your current curriculum. In addition, we will explain the study’s methodology and request from you names of any other people that we need to contact to secure your participation. As we proceed to phase two, please remember that the study is specifically about low literacy adult learners who span a range of low-level literacy abilities. In phase two of the selection process, we want to focus our discussion more on the specific sites and classes which will participate in the study. We will want to learn more about the students, including their enrollment patterns, attendance and retention in these specific classes and sites. We also want to know more about the teachers of these classes who will potentially participate in the study, including their willingness to participate. To prepare for the conference call, we would like you to gather more information about these topics, including any data you may have. Please also consider which staff members are appropriate to join in the conference with us (i.e. ESL specialists, other site or program directors, teachers who might want to be involved to help us determine the site’s eligibility for the study). I am including more information about the study and the curriculum, including the benefits of participation, the site selection criteria, and the random assignment process. One of the project staff members will call you in a few days to confirm your interest in the study and to schedule the date and time of the conference call. If you have any questions, please don’t hesitate to contact me or email Anestine Hector-Mason at ahector-mason@air.org; 202-403-6913. Thanks for your kind consideration and we look forward to speaking with you. Sincerely, Larry Condelli, PhD Project Director American Institutes for Research 1000 Thomas Jefferson Street Washington, DC 20007 Tel: 202-403-5331 Email: lcondelli@air.org Site Selection Criteria for Explicit Literacy Impact Study Selecting sites that meet the study’s criteria is very important to the study’s proper implementation. Sites in different geographic regions in the US that meet the following criteria will be considered for the study: • Operate large ESL programs, serving lowliterate adult ESL students. To meet the sample size requirements, we need programs in the study that offer at least four beginning-level ESL classes per semester with a minimum of 20–25 low- literate students in each class. Over two semesters, this will result in about 180 low-literacy students receiving instruction in either the explicit literacy or the existing ESL group. By low-literacy we mean students who have only limited literacy skills in their native language. We verify literacy level through a brief native language literacy screener. • Spanish speakers. While we believe that speakers of all languages will benefit from the curriculum, some instructional activities build on students’ Spanish literacy. Therefore, we want a large majority of students in the classes to be native Spanish speakers to allow us to conduct a full evaluation of all activities in the curriculum. • Managed enrollment policy. To implement the explicit literacy curriculum effectively, it is necessary that most students receive the curriculum from start to finish. This means that the students are only allowed to enroll at a limited time at the beginning of the class term. Programs with open- enrollment policies can only be considered if the large majority of learners enroll during a limited time period at the beginning of the session. • Sufficient class time. The curriculum is designed in 60 one-hour modules of instruction that are to be followed by one hour of instruction normally provided (e.g., conversational, life skills-type instruction). Therefore, in a 2 hour class, 1 hour would be devoted to the explicit literacy instruction. This suggests that, ideally, a class would need to meet at least four times per week for 2 hours each day for 15 weeks, for a total of 120+ instructional hours. Other class arrangements are possible as long as they can implement the full curriculum within 15 weeks. • Student retention. Student retention is key to evaluating the curriculum because students must receive a sufficient amount of instruction. Therefore, it is important that the number of students who fail to complete the entire scheduled instruction is minimal. We need classes in the study that have a record of high student retention. • Do not already offer explicit literacy instruction. Explicit instruction is the systematic, sustained, and deliberate teaching of language and literacy skills and of strategies for comprehension. In ESL literacy, three essential features of this type of instruction are (a) to help students see language and literacy connections, (b) to make the learning objectives transparent, and (c) to provide a cumulative and sequential approach to literacy learning. This is a new approach in adult ESL and we believe that no programs currently provide this type of instruction. •Teachers. Selected sites must have at least 5teacherswhoarewillingtoparticipateinthestudy. Two of the teachers will be assigned randomly to teach the explicit literacy curriculum and one teacherwillbeaback-upwhowillonlyteachif either of the two assigned teachers cannot participate. All teachers should have some experience teaching low-literate adult ESL students. Curriculum for Explicit Literacy Impact Study The goal of the curriculum is to increase the English literacy skills of students who speak no or little English and have limited literacy skills in their native language. Explicit Literacy Principles The curriculum uses an approach to teaching known as “explicit teaching” and will focus on the development of literacy skills, including skills associated with learning to read in another language, vocabulary development, listening/reading and comprehension. The following are some key principles of explicit literacy instruction: • Curriculum, instruction, learning experiences and assessment must be aligned. • Teaching goals and curriculum must be clearly defined. • Lessons must involve instructional modeling or demonstration, guided practice, corrective feedback, and practice. • Instruction must promote the development of skills and strategies for making meaning. • Teaching of oral language and literacy must be systematic and purposeful. • Curriculum and instruction must entail transparent connection between language and literacy. Materials The curriculum reflects research in second language acquisition, reading, and adult literacy development. A fully developed set of teaching materials includes: • illustrated and detailed lesson plans, • demonstration materials on PowerPoint slides or overheads, • student handouts and manipulatives, • a student workbook, • a student reader with picture dictionaries, and • a cd-rom of recorded reading in two speeds. Lessons Lessons are organized by themes of interest to adult learners and learning activities are integrated and contextualized. The lessons are structured in a way that allows teachers to move through a series of activities that include providing comprehensible input and guided practice for pairs or small groups; demonstrating the skill or strategy to be learned, and conducting a short teacher-led review. Each lesson will include teacher observations and strategies for dealing with a multi-level classroom. Lessons reflect an explicit approach to teaching that emphasizes the deliberate teaching of skills and strategies associated with literacy development, a systematic effort to help students understand how English works, ample opportunities for students to engage in the content and alignment of the key skills through demonstration, guided practice, and ongoing informal assessment. The curriculum is rich in illustrations and photographs and uses multiple modes of learning (visual, text-based, auditory). It seeks to promote transfer of knowledge and builds on both the linguistic knowledge and the real world knowledge that adults bring to the learning process. Teacher Training and Ongoing Support The curriculum developers will conduct a week-long training session for teachers that will familiarize them with the instructional strategies that are part of explicit teaching and to offer them opportunities to engage in, and reflect on, teaching demonstrations. Teachers will be supported during implementation through on-line and on-call coaching, listserv conversations with fellow teachers, web resources, and a classroom visit. Benefits of Participating in the Explicit Literacy Impact Study Selected programs will play an important part in a large-scale study that has the potential to make very important contributions to the Adult English as a Second Language (ESL) field specifically, and Adult Education in general. In addition, the programs will receive the following direct benefits: • The Explicit Literacy curriculum. Each teacher assigned to deliver the explicit literacy curriculum will receive a free copy of the curriculum, including a copy of all corresponding supplemental materials such as detailed lesson plans, PowerPoint slides and other demonstration materials, handouts and manipulatives, student workbooksandreaderwithpicturedictionary,and a CD-ROM of recorded readings. This curriculum is based upon the best research available. Programs will also receive one additional copy of the curriculum at the conclusion of the study, at no charge. • Training. Each teacher assigned to deliver the explicit literacy curriculum will receive free training in a one-week course offered prior to the start of class. These teachers will receive training in using the new explicit literacy curriculum and materials designed to improve the literacy skills of low-literate adult ESL learners. AIR teamed with a curriculum developer, Literacy Work, to create a training program built around principles of adult learning, effective ESL teaching, and explicit literacyinstruction.Inadditiontothefreetraining, all corresponding training materials, travel, and accommodations during the training are free of charge to teachers assigned to deliver the explicit literacy curriculum. • Technical assistance. The curriculum developer will be available to provide ongoing technical assistance to the explicit literacy teachers through a project web site and regular contact by telephone and e-mail. The web site will provide useful instructional modeling video clips and a listserv forum, as well as other useful implementation tips and materials. The developer will also be available for questions by phone, and will hold weekly phone calls with teachers. In addition, Literacy Work and AIR staff will conduct at least one class observation during the instructional period to assess whether teachers are implementing the curriculum faithfully. Literacy Work staff will also provide feedback and additional training to teachers as needed. • Cost-free program evaluation. The Explicit Literacy Impact study will use a random assignment design to measure whether the explicit literacy curriculum makes a difference in improving the English literacy skills of low-literate adults compared to the instruction usually offered. We will provide your program with the results of this evaluation for the students in your program. In addition to the results of the impact study, we will provide you with descriptive analyses on the characteristics of the students in the program, their test performance, patterns of attendance and other data. There will be no burden on program staff to obtain this information because research staff will collect all post-test assessment data, conduct interviews and track students. Random Assignment in the Explicit Literacy Impact Study The random assignment of students and teachers is a key component of the study because it enables us to assure that changes over time in student performance are due to the new curriculum rather than to differences in teacher and student characteristics and/or naturally occurring changes in the students and their social and educational environments. What is random assignment? Researchers use random assignment in impact studies to form two statistically equivalent groups of participants in the most objective way possible. For this study, the term “participants” refers to teachers and students who are eligible for and agree to participate in the study. A list of participating students will be created and those students on the list will be randomly assigned to classrooms. Some of the students will be assigned a classroom in which the explicit literacy curriculum will be delivered and some will be assigned to classrooms in which the program’s existing curriculum is delivered. Participating teachers will also be randomly assigned to teach either the explicit literacy curriculum or the existing curriculum. How does random assignment work? The random assignment process works much the same as a lottery, so each participant—whether a student or a teacher—has the same chance of ending up in the class using the explicit literacy instruction or to the one using the program’s existing ESL curriculum. In this study, a computer will be used to randomly assign both the teachers and the students. • Teachers. We will work with the programs to identify teachers who agree to be in the study. Among this group, we will randomly assign the teachers to the explicit literacy group or the existing ESL curriculum group. Only those assigned to the explicit literacy group, along with one back-up instructor, will be trained in the curriculum. The back-up instructor will only be used for this study if he/she is needed as a substitute for a teacher that has been randomly assigned to the explicit literacy class but is unable to participate for unforeseen reasons.. • Students. Students who register for ESL instruction will be screened at intake to determine their literacy level. Those who agree to participate in the study and who are at a low level of literacy (as determined through a brief native language literacy screener) will be randomly assigned, using a web-based computer program, to either the explicit literacy group or the existing ESL group. The attached chart illustrates our planned process. Why randomly assign teachers? Teachers will be randomly assigned to either group to ensure we are specifically testing the approach to teaching ESL and not other factors that are tied to their willingness to teach the new curriculum or other teacher characteristics. Teachers who volunteer to teach this new curriculum may differ, in key ways that affect the outcome of the study, from teachers who are not interested in teaching the new curriculum. To control for this selection bias we will select from a pool of teachers who are willing to teach the explicit literacy curriculum and randomly assign them to one group or the other. Why randomly assign students? This study will be measuring the effect of the explicit literacy instruction relative to the existing ESL instruction. To do so, we will need to compare the outcomes of the students assigned to each group after program completion. Similar to our reason for randomly assigning teachers, if we were to allow students to self-select into each group, we may find that students who seek out explicit literacy instruction are more motivated or differ in other important ways to those who choose the existing program. By randomly assigning students, the two groups are virtually identical in all ways except that one group receives the explicit literacy instruction, while the other receives instruction as it is currently being offered. This way, any differences in outcomes among students in both groups can be attributed to the explicit literacy curriculum. Is random assignment a fair way to select students who participate in the explicit literacy curriculum? Random assignment is fair because it gives all participating students an equal chance of being selected for the explicit literacy curriculum. Personal factors play no role in whether a student is selected. No students will be denied services as a result of this study. If students are not assigned to the explicit literacy instruction, they are assigned to the class that they would have participated in had this study not been implemented. What is required of the sites to do random assignment? The evaluation team will work with each site to develop a random assignment process that is not unduly burdensome. We are developing a simple web-based system that will record information regarding the students at intake and will automatically assign students to classes using a random process. We will work with the sites to determine the most appropriate time to collect the intake information, conduct the screening, and determine the random assignment group. It is likely that the screening process will occur during the intake process and the random assignment will occur when students show up for the first few days of class. We also will work closely with sites to ensure that teachers assigned to the explicit literacy classes avoid disclosing information about features and techniques in the explicit literacy curriculum. Will participants assigned to the existing ESL curriculum group ever have an opportunity to receive instruction from the explicit literacy curriculum? Assuming the new curriculum proves effective, which we do not know prior to an evaluation of the curriculum’s impact, a copy of such curriculum will be provided to all programs after the study’s completion, and programs are free to use it with students. What happens if a participant “drops out” of the program after being randomly assigned to either group? Students who drop out of the class assigned to them will be viewed as if they were still part of the study. Researchers will continue to collect follow-up data. Students who return to the school during the study period will be assigned to the group to which they were originally assigned. Random Assignment Flowchart Student Application Program Intake Initial Screening of Students (program administers usual screening and assessment to identify beginning ESL students who would normally attend study classes) Student Literacy Screening (Native language assessment) Explicit Literacy Instruction Existing ESL Instruction Assessment Assessment Analysis of Impact Students with Low or Advanced Native Language Literacy Levels Students with Limited Native Language Literacy (Informed consent obtained) Included in study classes First Two Days of Class ..Students check in at site ..Random assignment occurs ..Initial assessment and interview conducted ..Students informed of class assignment ..Students begin class Random Assignment (Students) Excluded from study, but students will receive normal instruction provided by the program in classes not in the research study