Grading the States 2009: A Brief Overview of Methodology

In August 2008, NAMI surveyed state mental health agencies in preparation for this 2009 report. While covering similar topic areas to the 2006 Grading the States report, this latest edition of the survey has evolved in several ways:

More Detail-Oriented Questions: Questions have been structured to draw out clearer and more detailed information. States were also encouraged to offer clarifications and additional comments to their responses.
Supplemental Information Requested: NAMI asked states to provide a variety of supporting materials and planning documents, including those covering cultural competence, housing, and workforce development.
Most of the data for assessing states in this report came from NAMI's survey of state mental health agencies. However, three secondary sources of information were used for state estimates on these measures:

the number of adults living with serious mental illnesses (based on work by Charles E. Holzer, III, Ph.D., of the University of Texas Medical Branch in Galveston, Texas, and Hoang T. Nguyen, Ph.D., of LifeStat LLC10);
the extent of shortages in the mental health workforce (based on work by Joseph P. Morrissey, Ph.D., Thomas R. Konrad, Ph.D., Kathleen C. Thomas, Ph.D., and Alan R. Ellis, M.S.W., of the Cecil G. Sheps Center for Health Services Research at the University of North Carolina at Chapel Hill);
hospital-based inpatient psychiatric bed capacity (based on annual survey data from the American Hospital Association).

Other information sources were used to identify states with ongoing federal investigations and lawsuits involving public sector programs’ treatment of adults living with serious mental illnesses.

State Scorecards Scoring

All states except South Dakota responded to NAMI’s survey for this 2009 Grading the States report. The information was scored and weighted in four broad categories:

Health Promotion and Measurement - 25 percent of grade
Financing and Core Treatment/Recovery Services - 45 percent of grade
Consumer and Family Empowerment - 15 percent of grade
Community Integration and Social Inclusion - 15 percent of grade

Challenges in Assessing a Complex System

Our nation’s public mental health system is complex, bridging inpatient and community-based health services, housing and economic support programs, vocational and social supports, and the criminal justice system, among others. Because of this complexity, it is extraordinarily challenging to accurately assess not only its overall quality, but also the effectiveness of each component and the extent to which the components successfully interact.

The lack of reliable outcome data generally limits the ability to measure the effectiveness of state services. Plans and policies may exist, but they do not necessarily translate to implementation. Evidencebased practices may be intended, but fall short of fidelity standards.

With those caveats in mind, this report provides the best comprehensive, comparative assessment of state mental healthcare systems to date. State-by-state narratives go beyond existing state data and shed light on each state’s qualitative performance.

download full methodology (PDF)

"[Recovery means] functioning normally within society without mental health issues proving to be a barrier."