UNITING PHYSICIANS & PATIENTS AS A VOICE IN HEALTH CARE
Process measures (e.g., did the surgeon prescribe a beta blocker for the heart attack patient?) often do not require risk adjustment, and, for diagnoses for which a standard of care has been reached, are therefore likely to be less expensive than outcome measures. This is true, however, only where the patient population examined is limited to patients who actually saw a doctor. For example, a process measure that measured whether a doctor advised a patient to quit smoking would not need to be risk adjusted as long as the study was limited to patients who visited the doctor. But a process measure that used all patients assigned to a clinic by an HMO as the denominator would have to be risk adjusted for both health and socioeconomic factors that influence patients' ability and inclination to see a doctor. These factors are partially or wholly outside the doctor's control and, if uncontrolled, could bias physician "scores." If patients refuse or are unable to see their doctor, or refuse or are unable to comply with physician recommendations, that is not the doctor's fault, and it is irrational to punish doctors for conditions beyond their control. For example, a PFP scheme that paid doctors more if a higher proportion of their female patients got mammograms would have to risk adjust physician "scores" so that the report card would not be confounded by the following factors: the woman's insurance status (is she insured, if so, does her policy cover mammograms, and if so, how big is the deductible?), her income, her education, and the presence or absence of language and transportation barriers. (17)
However, even process measures that do not require risk adjustment present a hurdle that is almost as daunting as risk adjustment, namely, the need for an agreed-upon standard of care that applies to all patients with a given diagnosis. Relative to the thousands of medical services rendered in America today, evidence-based standards are few. The proportion of medical services for which a science-based consensus on standard of care exists is apparently no more than15 to 20 percent (18). According to Landon et al., "[F]ew medical specialties have an evidence base that is robust and comprehensive enough to support PCPA [physician clinical performance assessment]." (19)
The main advantage of process measures – the fact that many do not have to be risk adjusted – is, of course, their main disadvantage – they may bear little relation to patient health. A physician could score high, for example, on a report card that measured how often the physician's diabetic patients had their cholesterol measured (a process measure), but that score may say little about how well the patients have kept their cholesterol levels within the normal range (an outcome measure).
Many advocates of report cards have expressed the hope that patient surveys will prove to be an inexpensive alternative to report cards based on risk-adjusted outcome measures or process measures. It is no doubt true that most of the survey-based report cards being published today are relatively inexpensive, but, unfortunately, they are not accurate and should not be used by patients. Surveys face the same problems outcome- and process-based report cards face. If the survey question seeks information on outcomes, the "grade" has to be risk adjusted, that is, the respondents' health status must be taken into account. It is well established that sicker patients are more critical of their caregivers. (20) If the survey question seeks information about processes of care, the process being measured must be shown to have a robust relationship with high-quality outcomes. It is not clear that questions commonly asked in patient surveys meet these criteria. Consumer surveys do not, in other words, provide a low-cost method of deriving accurate report cards on physician services.
The typical consumer survey suffers from a defect that is closely related to the problem of inadequate risk adjustment – the "bundled product" problem. Whereas physician outcome and process measures typically measure the quality of a single, discrete medical service (e.g., bypass surgery or cholesterol checks for diabetics), publishers of consumer surveys typically make no effort to limit their sample to patients receiving a single medical service. The Buyers Health Care Action Group in Minnesota, for example, makes no effort to limit its surveys of patients who visit BHCAG "care systems" to patients receiving specific services. Instead, BHCAG bundles all patient responses into one score. Thus, readers of BHCAG "satisfaction" surveys have no idea what services patients sought, and, therefore, no way to determine whether one care system's ostensibly superior score was caused by factors within or without the system's control. To take an extreme example, consider how easy it would be for Care System A to outscore Care System B if the former saw only children who needed immunization shots while the latter saw only elderly cancer patients with numerous comorbidities.
The most obvious problem with "satisfaction" surveys is that they may bear little relation to technical quality of care. Unfortunately, like so many other managed-care tools, little research has been done on whether the "satisfaction" survey tool actually works. As recently as 2003, Edlund et al. noted, "[G]iven the widespread use of satisfaction surveys, surprisingly little work has been done to investigate the relationship between subjective patient satisfaction and objective measure of quality of care." (21) But after comparing a process-measure-based report card on mental health providers with the results of a risk-adjusted survey of those providers' patients with addictions or mental disorders, Edlund et al. concluded that there is a modest correlation between the scores based on a well-designed survey and those derived from process measures. The crucial phrase here is "well-designed survey." The survey used by Edlund et al. was unusually well adjusted for risk, and the bundled-product problem was greatly diminished by examining only patients needing mental health services. Very few, perhaps none, of the "consumer satisfaction" surveys published by plans, magazines, and entrepreneurs with Web sites are risk adjusted and limited to a single service, or to services provided by a single specialty. The reason for that is obvious: Accurate risk adjustment is expensive, and survey-based report cards would have to be far more numerous if they were restricted to a single service or specialty.