"PRRAC's Federal & State Data Reconnaissance Project"July/August 1998 issue of Poverty & Race
As far back as the early 90s, when PRRAC got started, advocates (those on our Board and others) were complaining that the absence of good data on program impacts - triggered in part by governmental "paperwork reduction" mandates - was making it difficult to carry out effective advocacy work.
We therefore undertook our first Board-mandated research/advocacy project (as opposed to our program of funding proposals by others for research tied to advocacy): a reconnaissance of federal-level data collection and dissemination practices and results with respect to the impact of federal housing, health, education and income maintenance programs on poor and minority beneficiaries. We also wanted to report on what legal mandates exist to collect and report such data. We commissioned studies in these four areas by, respectively: Ann Shlay of Temple; Carol Korenbrot/Ayesha Gill/Dana Hughes of the Univ. of Calif-SF Inst. for Health Policy Studies; James McPartland/Nettie Legters of the Johns Hopkins Center for the Social Organization of Schools; and Brett Brown/Kristin Moore of Child Trends. (We have available copies of these studies, as well as a summary report on the four; contact us for prices.)
Our intention was to then to use these studies - completed in 1993 and 1994 - in a multi-pronged effort to improve the quantity, quality, relevance and dissemination of such key data. We had in mind Congressional hearings and legislation, administrative reform, aid from friends and colleagues appointed to high-level posi-tions in Clinton Administration, possibly litigation. We also began conversations about collaborative work with the Leadership Conference on Civil Rights and other national organizations. Then the 1994 Congressional elections hit, and that (plus the somewhat disappointing performance of the Administration) put a hold on the project.
Fortuitously, we moved the project to the state level at just about the time the devolution revolution was upon us. The Irvine Foundation agreed to fund a similar project for California, and we contracted with the California Budget Project to carry out a parallel set of reconnaissance studies. These too were carried out (and are available from us, individually and in summary form): the housing report by the California Coalition for Rural Housing Project; health by Gale Berkowitz of the UCSF Inst. for Health Policy Studies; education by Susan Conklin/Julia Koppich of UC-Berkeley's Policy Analysis for California Education; income maintenance by Henry Brady/Ilona Einowski of UC-Berkeley's Data Archive & Technical Assistance.
Next, we added four more states to the project, assisted by funding by the CS Mott Foundation: North Carolina, Texas, Illinois and Alabama. In each case, we partnered with a state group participating in the State Fiscal Analysis Project, a project, funded by a consortium of foundations, to create in a dozen states entities and capacities similar to what is done at the national level by the Center on Budget & Policy Priorities. We selected those states that might, with California, give us sufficient variety in size, population composition and geography so that we might credibly hold this sample of five out as depicting deficiencies in state data collection/dissemination efforts all over the country, in order to persuade Congress and the federal administrative agencies of the need to require more from the states in the way of evaluation capacity and accountability. These new state reports, now virtually all complete (contact us or our state partner organizations listed below regarding ordering copies), will be used, both within the five states as well as nationally, to undertake a range of advocacy activities similar to what had been earlier planned. Once again, we hope to work collaboratively with a range of national groups.
Our five state partners (some of whom did the reconnaissance studies with their own staff, some contracted the work out to consultants) are:
California: California Budget Project (Jean Ross), 921 11th St., #701, Sacramento, CA 95814, 916/444-0500.
North Carolina: N.C. Budget & Tax Center (Dan Gerlach), P0 Box 27343, Raleigh, NC 27611, 919/856-2158.
Texas: Center for Public Policy Priorities (Dianne Stewart/Pam Hormuth), 900 Lydia St., Austin, TX 78702, 512/320-0222.
Illinois: Voices for Illinois Children (Jerry Stermer/Brian Matikias), 208 S. LaSalle, Chicago, IL 60604, 312/456-0600.
Alabama: Alabama Arise (Dave Dawson), P0 Box 612, Montgomery, AL 36101, 334/832-9060.
Results of the State Reconnaissance Studies
While the picture varies somewhat from state to state, the overall, and non-surprising, bottom line is that in none of the states is there anywhere near an adequate system of data collection and dissemination so that advocates can access information sufficient to evaluate the impact of these key safety-net programs on poor and minority recipients. And absent such data, observed program defects and those brought to the attention of advocacy groups via anecdotes cannot be adequately and forcefully documented, the prerequisite for effective efforts to improve defective programs, jettison truly poor programs and propose alternatives.
The California report indicates the harm budget cuts have done to data collection and publication. In the education area, there is no consistent source of data assessing pupil performance, due to frequent state policy changes regarding pupil testing and assessment. The state lacks a comprehensive statewide data system for tracking social service programs. The four-county sample used to detail participant characteristics in income support programs is not geographically reflective of the state's diversity (e.g., none of the four counties sampled is small or rural): as the state moves towards a decentralized welfare system, maintaining consistency and availability of data will become ever more important, but difficult to attain. In the health area, one notable weakness is the lack of consistency between data sets with respect to race and ethnicity reporting categories and lack of information on socioeconomic status. Housing data are very weak: the state imposes only minimal requirements on its many housing programs and there is little standardization of data collection. Categories for reporting race/ethnicity of housing program participants is imprecise, particularly with regard to the state's large Hispanic population.
The North Carolina report showed that while there is a rich set of data, collected by race, there is no evaluative or data research component. Since individual counties are permitted to construct their own TANF eligibility criteria and benefit levels, data are needed to determine if race, disability or other demographic factors impact decisions to sanction families, access certain types of assistance or allow time limit extensions. In the education area, emphasis on testing has produced a situation whereby data are inadequate to determine whether children can advance to the next grade: only end-of-grade tests are used, which results in tremendous disparities among low- and high-income and white and nonwhite children.
The Texas project observed that in so large (254 counties) and regionally diverse a state, data collection based on sample surveys does not permit usable generalizations, or gives false pictures. It also found inconsistent and uncoordinated data systems among various agencies that make comparison and inter-agency communication difficult. Agency data are often not user-friendly and display inconsistent and unclear formatting. Some data crucial for planning vital health and human services are completely unavailable (e.g., it is not possible to get child poverty rates by ethnic group for each county). There is no consistent standard for defining many important variables that all agencies use. Data definitions are inconsistent, making comparisons among different data sources difficult (e.g.,income level definitions used in allocating certain health care services are not consistent with Census or IRS designations); comparison between states, agencies and other databases are problematic. Finally, because of processing delays, data often are old and no longer relevant; relatedly, websites are sometimes out of date.
The Illinois report shows that a great deal of government data is available, not all of it useful, but that many of the data are difficult to access; data kept in one government agency cannot be linked to data in another agency; and that data do not track how people's circumstances change over time, how different services interact in their lives, and how these services interact with family and community networks. Education data are notably lacking in outcomes of school programs. Much of the data on poverty is in case files and thus tends to be more useful for deciding program eligibility than for program evaluation and research.
The Alabama report observed that access to information is greatly dependent on the policies of the leaders of each department and agency, and such policies vary greatly. The state has few laws regarding the type of data to be made available and the public's access to it. Acquiring state data is often a very time-consuming and cumbersome task. Most staff are unaware of information outside their own department. Personal contacts and relationships are often essential for acquiring data in a timely manner. Some independent (non-governmental, mainly university) data sources exist. Education data on demographic characteristics are only available at the school system level. Health data omit income and quality of care variables.
PRRAC is now moving this project into the advocacy phase. Each of our state partners has drafted a preliminary advocacy strategy, and we will be working closely with them to generate and implement specific steps appropriate to the defects and political/legal situation in each state. Simultaneously, we will be bringing together appropriate national-level partners to deal with this important issue in Congress and with the Administration. Our approach is that even conservatives in Congress ought to be concerned with how the states are carrying out their newly mandated responsibilities; that evaluating how government programs are carried out, how tax money is spent is, or should be, a universal concern, and that such accountability requires good, relevant data to be widely and easily available. Taking full advantage of new communications technologies will be a central goal.
We'll keep P&R readers regularly informed as to the progress of this effort.
|Poverty & Race Research Action Council | 740 15th St. NW, Suite 300, Washington, DC 20005|
ęCopyright 1992-2018 Poverty & Race Research Action Council