Study of the Effect of the Talent Search Program on Secondary and Postsecondary Outcomes in Florida, Indiana and Texas - PDF

Description
U. S. DEPARTMENT OF EDUCATION Study of the Effect of the Talent Search Program on Secondary and Postsecondary Outcomes in Florida, Indiana and Texas A Study of the Effect of the Talent Search Program On

Please download to get full document.

View again

of 24
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Information
Category:

Resumes & CVs

Publish on:

Views: 55 | Pages: 24

Extension: PDF | Download: 0

Share
Transcript
U. S. DEPARTMENT OF EDUCATION Study of the Effect of the Talent Search Program on Secondary and Postsecondary Outcomes in Florida, Indiana and Texas A Study of the Effect of the Talent Search Program On Secondary and Postsecondary Outcomes in Florida, Indiana and Texas Final Report From Phase II Of the National Evaluation By: Jill M Constantine Neil S. Seftor Emily Sama Martin Tim Silva David Myers Prepared for: U.S. Department of Education Office of Planning, Evaluation and Policy Development Policy and Program Studies Service 2006 This report was prepared for the U.S. Department of Education under Contract Number ED98-CO-0073 with Mathematica Policy Research Inc. The project monitor was Sandra Furey in the Policy and Program Studies Service. The views expressed herein do not necessarily represent the positions or policies of the Department of Education. No official endorsement by the U.S. Department of Education is intended or should be inferred. U.S. Department of Education Margaret Spellings Secretary Office of Planning, Evaluation and Policy Development Tom Luce Assistant Secretary Policy and Program Studies Service Alan Ginsburg Director Program and Analytic Studies David Goodwin Director June 2006 This report is in the public domain. Authorization to reproduce it in whole or in part is granted. While permission to reprint this publication is not necessary, the suggested citation is: U.S. Department of Education, Office of Planning, Evaluation and Policy Development, Policy and Program Studies Service, A Study of the Effect of the Talent Search Program on Secondary and Postsecondary Outcomes in Florida, Indiana and Texas: Final Report From Phase II of the National Evaluation, Washington, D.C., To order copies of this report, write: ED Pubs Education Publications Center U.S. Department of Education P.O. Box 1398 Jessup, MD ; via fax, dial (301) ; or via electronic mail, send your request to: You may also call toll-free: ( ED-PUBS). If 877 service is not yet available in your area, call (1-800-USA-LEARN). Those who use a telecommunications device for the deaf (TDD) or a teletypewriter (TTY) should call To order online, point your Internet browser to: This report is also available on the Department s Web site at On request, this publication is available in alternate formats, such as Braille, large print or computer diskette. For more information, please contact the Department s Alternate Format Center at or ACKNOWLEDGMENTS This report completes Phase II of the National Evaluation of Talent Search and reflects the contributions of many people. The authors would like to thank Project Officer Sandra Furey of the Policy and Program Studies Service (PPSS) in the U.S. Department of Education (ED) for her technical guidance, support, and encouragement throughout all phases of the study. Other PPSS staff providing guidance and support at key times were David Goodwin and Maggie Cahalan. The study also benefited from feedback and assistance from staff members in the Office of Federal TRIO programs including Larry Oxendine and Frances Bergeron. The compilation of the data described in this report would not have been possible without assistance from individuals across the country. Talent Search project directors and staff members in Florida, Indiana, Minnesota, Texas, and Washington provided volumes of data on the students they served. Data on applications for federal financial aid were generously and expertly provided by Joe Mike from the Office of Postsecondary Education in ED and Elise Brand from Computer Business Methods Inc. Finally, we owe special thanks to dedicated staff and researchers in the study states, including Glenda Droogsma Musoba, Edward St. John, and Jack Schmit at Indiana University, Darlene Gouge and Shannon Housson of the Texas Education Agency, Kathy Benson of the Texas Higher Education Coordinating Board, and Jay Pfeiffer, Murray Cooper, Kathy Peck, Jeff Sellers, and Duane Whitfield at the Florida Department of Education, whose heroics in battling through the four-hurricane summer of 2004 to provide data were greatly appreciated. The study was performed under contract to Mathematica Policy Research Inc. The successful compilation of the data for the study was due to the efforts and skills of several Mathematica project team members: Justin Humphrey and Missy Thomas led the effort to collect data from Talent Search projects, Jackie Agufa provided technical supervision in processing the data, Carol Razafindrakoto and Miriam Loewenberg provided expert programming to compile millions of records across all of the states, and Annette Luyegu conducted phone interviews with all Talent Search project directors providing data for this report. Patricia Ciaccio and Roy Grisham edited numerous drafts of the report, and Bill Garrett was responsible for word processing and the layout of the report. Paul Decker and Mark Dynarski provided a constructive quality assurance review at Mathematica. iii CONTENTS Chapter Page Acknowledgments.. iii List of Tables.. ix List of Figures xiii Executive Summary.xv I Introduction...1 A. Program Description Context for Studying Talent Search Services Offered by Talent Search Study Description and Report Overview...4 II Study Design and Methodology...7 A. Study Design Research Questions State and Project Selection...7 B. Analytic Approach The Preferred Comparison Group Using Propensity Score Models to Identify the Comparison Group...11 C. Estimating the Effect of Talent Search on Secondary and Postsecondary Outcomes Empirical Specification Standard Errors Reliability of Propensity Score Matching Methods...15 v CONTENTS Chapter Page III Texas...17 A. Introduction...17 B. Data Strengths and Weaknesses of the Data Characteristics of Talent Search Participants and Potential Comparison Students...22 C. Comparison Groups...25 D. Results High School Completion Application for Financial Aid Postsecondary Enrollment Postsecondary Persistence...37 E. Discussion of Results...43 IV Indiana...49 A. Introduction...49 B. Data Strengths and Weaknesses of the Data Characteristics of Talent Search Participants and Potential Comparison Students...54 C. Comparison Groups...57 D. Results Application for Financial Aid Postsecondary Enrollment...64 E. Discussion of Results...68 vi CONTENTS Chapter Page V Florida...71 A. Introduction...71 B. Data Strengths and Weaknesses of the Data Characteristics of Talent Search Participants and Potential Comparison Students...75 C. Comparison Groups...77 D. Results High School Completion, Application for Financial Aid, and College Entrance Test Taking Postsecondary Enrollment Postsecondary Persistence and Completion...87 E. Discussion of Results...90 VI Comparing Findings Across the States...95 A. Implications of the Data for Studying Talent Search...95 B. Findings...96 C. Implications References Appendix A: Chapter Tables Appendix B: Compilation of Data Sources and Feasibility of Evaluations Based on Administrative Records vii TABLES Table I Page Number of Talent Search Projects, Talent Search Participants, and Comparison Students, by State... xvi I.1 Services Most Commonly Offered by Talent Search Projects in III.1 III.2 III.3 III.4 III.5 III.6 III.7 III.8 III.9 III.10 III.11 Texas Data Sources...19 Baseline Characteristics of Talent Search Participants and All Other Students, All of Texas...23 Below Grade and Persistence of Talent Search Participants and All Other Students in Texas...25 Assessing Balance Between Talent Search Participants and Nonparticipants in the Same High Schools in Texas...28 Assessing Balance Between Talent Search Participants and Nonparticipants at Different High Schools in the Same Districts in Texas...30 Percentage of Talent Search Participants and Comparison Students Who Completed High School in Texas, by Project...33 Percentage of Talent Search Participants and Comparison Students Who Were First-Time Applicants for Federal Financial Aid in 1999 or 2000 in Texas, by Project...35 Percentage of Talent Search Participants and Comparison Students Who Enrolled in Any Public Postsecondary Institution in Texas, by Project...38 Percentage of Talent Search Participants and Comparison Students Who Enrolled in a Four-Year Public Postsecondary Institution in Texas, by Project...39 Percentage of Talent Search Participants and Comparison Students Who Enrolled in a Two-Year Public Postsecondary Institution in Texas, by Project...40 Continuous Enrollment in Four-Year Institutions and Total Credits Earned by Talent Search Participants and Comparison Students in Texas...42 IV.1 Indiana Data Sources...51 ix TABLES Table IV.2 IV.3 IV.4 Page Baseline Characteristics of Talent Search Participants and All Other Students in Ninth Grade in Fall 1995, All of Indiana...55 Assessing Balance Between Talent Search Participants and Nonparticipants in the Same High Schools in Indiana...60 Percentage of Talent Search Participants and Comparison Students Who Applied for Financial Aid or Enrolled in a Postsecondary Institution in Indiana, by Project Group...67 V.1 Florida Data Sources...73 V.2 Baseline Characteristics of Talent Search Participants and All Other Students, All of Florida...76 V.3 Below Grade and Persistence of Talent Search Participants and All Other Students in Florida...77 V.4 Assessing Balance Between Talent Search Participants and Nonparticipants in the Same High Schools in Florida...78 V.5 Assessing Balance Between Talent Search Participants and Nonparticipants at Different High Schools in the Same Districts in Florida...80 V.6 Percentage of Talent Search Participants and Comparison Students Who Completed High School and Applied for Financial Aid in Florida, by Project...83 V.7 Percentage of Talent Search Participants and Comparison Students Who Took College Entrance Exams in Florida, by Project...84 V.8 Percentage of Talent Search Participants and Comparison Students Who Enrolled in Any Public Postsecondary Institution in Florida, by Project...86 V.9 Percentage of Talent Search Participants and Comparison Students Who Enrolled in a Four-Year Public Postsecondary Institution in Florida, by Project...88 V.10 Percentage of Talent Search Participants and Comparison Students Who Enrolled in a Two-Year Public Postsecondary Institution in Florida, by Project...89 V.11 Percentage of Talent Search Participants and Comparison Students Who Persisted in Public Postsecondary Institutions in Florida, by Project...91 x TABLES Table Page V.12 Percentage of Talent Search Participants and Comparison Students Who Completed a Two-Year Degree in Florida, by Project...92 A.III.1 A.III.2 A.III.3 A.IV.1 A.IV.2 A.V.1 A.V.2 A.V.3 Variable Descriptions Texas Baseline Characteristics of Talent Search Participants and Nonparticipants in Texas, by Project Below Grade and Persistence of Talent Search Participants and Nonparticipants in Texas, by Project Variable Descriptions Indiana Baseline Characteristics of Talent Search Participants and Nonparticipants in Ninth Grade in Fall 1995, in Indiana, by Project Group Variable Descriptions Florida Baseline Characteristics of Talent Search Participants and Nonparticipants in Florida, by Project Below Grade and Persistence of Talent Search Participants and Nonparticipants in Florida, by Project B.1 Summary of Talent Search Project Record Data Collection xi FIGURES Figure Page 1 Talent Search Participants and Comparison Students Who Were First-Time Applicants for Federal Financial Aid, , by State... xviii 2 Talent Search Participants and Comparison Students Who Enrolled in a Public Postsecondary Institution, , by State... xix 3 Talent Search Participants and Comparison Students Who Enrolled in a Public Two-Year or Four-Year Institution, , by State...xx III.1 III.2 III.3 IV.1 IV.2 IV.3 Talent Search Participants and Comparison Students Who Completed High School and Were First-Time Applicants for Financial Aid from Texas in 1999 or Talent Search Participants and Comparison Students Who Enrolled in a Public Postsecondary Institution in Texas in 1999, 2000, or Talent Search Participants and Comparison Students Who Enrolled in a Public Postsecondary Institution in Texas in 1999, 2000, or 2001, by Institution Type...37 Talent Search Participants and Comparison Students Who Applied for Financial Aid in Indiana...64 Talent Search Participants and Comparison Students Who Enrolled in College in Indiana...65 Talent Search Participants and Comparison Students Enrolled in College in Indiana, by Degree Program...66 V.1 Talent Search Participants and Comparison Students Who Completed High School, Applied for Financial Aid, and Took College Entrance Exams in Florida...82 V.2 Talent Search Participants and Comparison Students Who Enrolled in a Public Postsecondary Institution in Florida...85 V.3 Talent Search Participants and Comparison Students Who Persisted in Public Postsecondary Institutions in Florida...90 VI.1 Talent Search Participants and Comparison Students Who Were First-Time Applicants for Federal Financial Aid, , by State...97 xiii FIGURES Figure VI.2 VI.3 Page Talent Search Participants and Comparison Students Who Enrolled in a Public Postsecondary Institution, , by State...98 Talent Search Participants and Comparison Students Who Enrolled in a Public Two-Year or Four-Year Institution, , by State...99 xiv EXECUTIVE SUMMARY Many studies have documented the fact that college graduates earn significantly more money during their lifetimes than high school graduates (U.S. Department of Education 2004). Research also shows that college graduates demonstrate greater civic involvement and are more likely to vote and assume leadership roles in their communities (Astin 1993; Bowen and Bok 1998). Low-income students and students whose parents have not attended college typically are less likely than middle- and upper-income students to complete high school and attend college, and are thus less likely to reap the benefits of attending college. Lack of information, resources, and exposure to others who have navigated the college process may be substantial hurdles for these students. Federal financial aid is available through Pell Grants, college tuition tax credits, and student loan programs, but low-income students may not be taking full advantage of these sources. Even low-income students with high educational aspirations may find the financial aid and college application processes overwhelming and discouraging. In 1965, Congress established the Talent Search Program as one of the original federal TRIO programs. The others include Upward Bound, created in 1964, and Special Services (later renamed Student Support Services), established in Today, TRIO includes five other programs. Collectively, these programs help low-income, potentially first-generation college students prepare for and gain access to college. The Talent Search program primarily provides information on the types of high school courses students should take to prepare for college and on the financial aid available to pay for college. The program also helps students access financial aid through applications for grants, loans, and scholarships, and orients students to different types of colleges and the college application process. In fiscal year (FY) 2004, the Talent Search program received approximately $144 million to serve 382,500 students in 470 projects across the country; on average, each project spent approximately $375 per participant served. The program has grown since FY 2000, the period relevant to this study, when Talent Search served 320,000 students and spent approximately $313 per student. After a two-year implementation study, the U.S. Department of Education s Policy and Program Studies Service selected Mathematica Policy Research Inc. (MPR) in 2000 to assess the effect of Talent Search in selected states. A variety of designs were considered and ultimately the study team opted to compile data from administrative records from many sources, including program, state, and federal records, to evaluate the effectiveness of federal education programs, partly as a test of whether such an evaluation was feasible. The study also yielded useful information about the effectiveness of the Talent Search program. METHODOLOGY The study included an analysis of the effectiveness of the Talent Search program in Florida, Indiana, and Texas. We based our analysis on administrative data compiled in these three states and a quasi-experimental design to create matched comparison groups (Rosenbaum and Rubin xv 1985). Outcomes of students who participated in Talent Search were compared with outcomes of similar students at the same schools or other schools who did not participate. We restricted our analysis to the cohort of students who were in ninth grade in to allow us to collect information on high school completion and postsecondary enrollment, which occurred as late as Although all students were in ninth grade in , Talent Search participants may have received services through the program at any point from grades six through twelve. We compared secondary and postsecondary outcomes between Talent Search participants and comparison groups within each state. We received data on Talent Search participants from at least 60 percent of all Talent Search programs operating in Florida, Indiana, and Texas in As shown in Table 1, samples included large numbers of Talent Search participants and matched comparison students in each state. TABLE 1 NUMBER OF TALENT SEARCH PROJECTS, TALENT SEARCH PARTICIPANTS, AND COMPARISON STUDENTS, BY STATE State Total Number of Talent Search Projects Operating in Number of Projects Providing Data Number of Talent Search Participants in Study Cohort Number of Matched Comparison Students in Study Cohort Florida ,843 Indiana 8 7 1,166 9,844 Texas ,112 30,842 KEY FINDINGS The main research questions addressed in this report are as follows. 1. Is it possible to rely on administrative records to compile a retrospective record of participation in Talent Search, characteristics of students in secondary school, and secondary and postsecondary outcomes? 2. Can administrative data and quasi-experimental techniques be used to identify students who are not participating in the program but have characteristics similar to the Talent Search students in order to estimate the relationships between participation in Talent Search and secondary and postsecondary outcomes? xvi The Efficacy of Using Large State Databases to Inform Policy The compilation of data from administrative sources to study the effect of Talent Search on participants was successful in Florida, Indiana, and Texas. We could not compile a suitable data file for analysis in Minnesota (due to a lack of access to state secondary school records) and Washington (due to missing or poor-quality Talent Search project data). Obtaining student level data which included information identifying students to facilitate merging records across data sources was challenging to obtain for the years of interest, Data from recent years should be easier to attain as more states develop systems for compiling secondary and postsecondary school records, and federal programs are more consistent in reporting information on the participants served and maintain records electronically. The data files compiled in Florida, Indiana, and Texas contained a wealth of information on students in Talent Search. This included important demographic information such as age, race, and gender; the school the student was enrolled in for ninth grade; and postsecondary outcomes, such as first-time application for financial aid and postsecondary enrollment. Because we compiled a large amount of data in each state, both in terms of the number of data elements available and the size of the student samples, we were able to use complex propensity score matching models to identify nonparticipating students who were most similar to Talent Search participants. We were also able
Related Search
Similar documents
View more...
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks