| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
ORIGINAL ARTICLES |
From the Departments of Psychiatry (M.B.S., J.R.M., C.L.) and Family and Preventive Medicine (M.E.M), University of California, San Diego, La Jolla, California; Department of Psychiatry, University of Washington, Seattle, Washington (P.P.R.-B., J.R., W.K.); University of California, Los Angeles, Los Angeles, California (M.C., A.B.); and RAND, Santa Monica, California (C.D.S.).
Address reprint requests to: Murray B. Stein, MD, Department of Psychiatry (0985), University of California, San Diego, 9500 Gilman Dr., La Jolla, CA 92093-0985. Email: mstein{at}ucsd.edu
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMETHODSRESULTSDISCUSSIONAPPENDIXACKNOWLEDGMENTSREFERENCES |
|---|
METHODS: A total of 1476 primary care outpatients in three primary care medical clinics on the West Coast of the United States were studied. Patients completed a brief self-report measure, the five-item Autonomic Nervous System Questionnaire (ANS), while in the waiting room. The presence of DSM-IV panic disorder was subsequently determined in groups of "screen-positive" and "screen-negative" subjects using the Composite International Diagnostic Interview. A subset of patients (N = 511) also completed the 21-item Beck Anxiety Inventory. Indices of diagnostic utility were calculated using receiving operating characteristic analyses to guide the selection of optimal cutoff levels.
RESULTS: The two-question version of the ANS had excellent sensitivity (range = 0.94–1.00 across the three clinic sites) and negative predictive value (0.94–1.00) but low specificity (0.25–0.59) and positive predictive value (range 0.18–0.40). The three- and five-question versions of the ANS had only modestly improved specificity, and this was achieved at the cost of reduced sensitivity and increased respondent burden to complete the questionnaire. The 21-item Beck Anxiety Inventory had maximal clinical utility at a cutoff level of 
20, but sensitivity was lower than desirable for a screening instrument (0.67).
CONCLUSIONS: The two-question version of the ANS shows promise as a screening instrument for panic disorder in the primary care setting.
Key Words: panic disorder • primary care • general medicine • screening • diagnostic interview • anxiety disorders
Abbreviations: GIM = University of Washington General Internal MedicineClinic; HMC = University of Washington Harborview Medical CenterAdult Medical Clinic; UCSD = University of California, San Diego,Mira Mesa Family Medicine Clinic; NPV = negative predictive value; Se = sensitivity; Sp = specificity; NPV = negativepredictive value; PPV = positive predictive value; ROC =receiver operating characteristic; SDDS-PC = Symptom-DrivenDiagnostic System for Primary Care; PRIME-MD = Primary CareEvaluation of Mental Disorders.
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONAPPENDIXACKNOWLEDGMENTSREFERENCES |
|---|
Panic disorder is commonly encountered in the primary care setting, with prevalence ranging from 2 to 13% (7, 20–22). Despite its high prevalence in primary care, accurate diagnosis is rare, and appropriate treatment is even rarer (22–24). This failure to recognize and adequately treat panic disorder likely contributes to inordinately high rates of medical utilization, medical costs, and excessive disability in these patients (3, 8, 25, 26). Primary care physicians are busy and, moreover, are being asked to recognize and treat an ever-growing list of ailments. This has led to interest in the development of brief screening measures for mental disorders for primary care, in the hope that their use might facilitate detection and diagnosis.
Screening measures for depression in medical settings have been used for some time (27–29), but screening measures for anxiety disorders have been much less extensively studied (30). In this report, we describe the development of a brief new measure to detect panic disorder in the primary care medical setting. Our goal was to develop an instrument that would identify patients with panic disorder who could then be approached to participate in an intervention research study. We evaluated the utility of the instrument by comparing it with a diagnostic confirmatory interview (the Composite International Diagnostic Interview) (31), as well as with a longer and more generic anxiety measure (the Beck Anxiety Inventory) (32), in a subset of patients.
|
METHODS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONAPPENDIXACKNOWLEDGMENTSREFERENCES |
|---|
Research assistants were stationed in the waiting rooms of the primary care clinics for periods of 3 to 4 hours. Times were chosen on the basis of convenience, usually to coincide with periods of maximal patient flow. Research assistants approached all English-speaking adult patients after they had checked in and asked them if they would be willing to participate in the study. At GIM and HMC, subjects were asked to participate in a study looking at "characteristics of a common autonomic nervous system disorder that causes sudden physical symptoms in patients." At UCSD, the study was described as looking at "the relationship between physical and mental health." Subjects were informed that the study consisted of two parts, the first being a short set of questionnaires and the second being a phone interview for a subset of participants. Subjects gave informed written consent to participate in the study, which was approved by the human subjects committees of the three institutions.
Reasons for refusal to participate were systematically recorded only at the UCSD site. At UCSD, 1024 potentially eligible participants were approached to take part in the study. Of those, 566 (55%) agreed to fill out the self-report measures, 239 (23%) refused to participate because they were not interested, 145 (14%) refused because they were too busy, and 74 (7%) did not participate because they were too sick. Of the 566 participants, 55 were unable to complete the screening measures before being called in for their appointment, leaving 511 participants with complete data.
Patients (N = 1614; 431 at HMC, 534 at GIM, and 511 at UCSD) completed a brief self-report questionnaire that assessed panic symptoms (the ANS; see below) and demographic variables. Patients at UCSD also completed the 21-item Beck Anxiety Inventory (32). The research assistant reviewed the information immediately and then approached patients who were positive according to the ANS ("screen-positive"; see below) and a convenience (ie, nonrandom) sample of subjects who were negative ("screen-negative") and invited them to participate in a follow-up diagnostic interview. At UCSD, we chose to interview approximately two screen-negative subjects for every screen-positive subject (2:1 ratio), whereas at GIM and HMC, the ratio was approximately 1:4. These sampling differences across sites, which were dictated by practical constraints such as research assistant availability, preclude us from combining the samples (see Data Analysis). Patients who agreed were contacted by phone within 10 days of the screening assessment and completed the diagnostic interview and disability and utilization questionnaires (see below) over the phone. Patients were paid $20 for the follow-up interview.
Panic Screening Measure
We adapted the panic screening measure used in the DSM-IV field trial (33) for use in this study. This instrument (Appendix), which was deliberately given a "medical" name ("Autonomic Nervous System Questionnaire") to be consonant with the expectancy set of patients in this setting, consisted of two "gating" and three subsequent questions. The two gating questions asked about the occurrence in the prior 6 months of anxiety attacks or unexplained paroxysms of physical symptoms (eg, tachycardia, dizziness, and shortness of breath). Only those persons who responded affirmatively were instructed to complete the next three questions. These questions asked about the occurrence of attacks outside dangerous or performance situations, the frequency of attacks in the prior month, and the extent of worry in the prior month about the possible recurrence of attacks. For purposes of this study, individuals who gave an affirmative response to either of the two gating questions were considered screen-positive.
Confirmatory Diagnostic Interview
The confirmatory diagnostic interview consisted of the panic attack/panic disorder module of the World Health Organization’s Composite International Diagnostic Interview, Version 2.1 (31). This module has a past-12-months time frame, whereas the ANS has a 6-month time frame; this may have inflated the number of false-negative results on the screener. This interview was administered by telephone to all accessible ANS screen-positive subjects (N = 309) and to a convenience sample of screen-negative subjects (N = 160).
Data Analysis
Standard indices of diagnostic utility were calculated for various cutoff levels on the ANS, including Se, Sp, PPV, and NPV. The Se of a screener is the proportion of true-positive cases that it correctly detects. Generally, it is desirable to minimize the number of true-positive cases missed in screening. A good screener therefore has high Se. The Sp of a screener is the proportion of true-negative cases that it correctly detects. High numbers of incorrectly classified negative cases are often the cost of maintaining high screener Se (27).
For descriptive purposes, we report the results of each site separately and do not report composite values for the three sites combined. The latter value would be misleading, given that one site (UCSD) deliberately oversampled screen-negative subjects. As a result of this sampling inequity, test characteristics that are strongly influenced by relatively small changes in the base rate of screen-negative (ie, Sp) or disease-positive (ie, PPV) individuals are noticeably different at the UCSD site. By varying sampling strategies at the three sites, one can derive information about how widely the test performance characteristics might vary at different base rates of disease prevalence. It is reasonable, then, to assume that the range of results reported here will "bracket" the results likely to be obtained in a variety of clinical settings.
ROC curves were used to determine an optimal cutoff level for the Beck Anxiety Inventory, and these values were then compared with those for the ANS (in the UCSD sample only). For all analyses, diagnostic results of the Composite International Diagnostic Interview were considered the reference standard for the test. Because ROC analysis does not address the effect of prevalence on test performance (34), Bayesian statistics were applied. Specifically, an equation developed by Metz et al. (35) measures in bits the reduction of diagnostic uncertainty that one obtains by performing a test. This characteristic, known as information content (I), can be applied to any point on an ROC graph and varies depending on the prevalence of the condition being tested for. To examine the utility of the test (and its alternate scoring options) at various prevalences, we calculated I for prevalence rates varying from 1 to 13%, which brackets the range seen in studies of panic disorder in the primary care setting (7, 20–22). This enabled us to compare the information content of the tests across a relevant range of prevalences and to choose as the best test the one that provided maximal I across this range. For reasons outlined above, these analyses were calculated separately for each site.
|
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONAPPENDIXACKNOWLEDGMENTSREFERENCES |
|---|
To examine the possibility of a "nonresponse" bias to the interview, we compared sociodemographic characteristics of ANS screen-positive subjects at the UCSD site who were interviewed (N = 96) with those of subjects who were not interviewed (N = 84). The two groups did not differ significantly in gender (79 vs. 67% female; 
2 = 0.08, p = NS), marital status (56 vs. 44% married or common law; 2 = 0.16, p = NS), education (54 vs. 46% with more than high school; 2 = 0.82, p = NS), or age (39.0 (SD 16.5) vs. 37.3 (SD 14.1) years; t = 0.79, df = 179, p = .43). The only difference between the groups was in ethnicity: interviewed subjects were more likely to be white (72%) than those not interviewed (58%; 2 = 3.23, p < .08).
The Se of our screener ranged from 1.00 at HMC to 0.97 at GIM to 0.94 at UCSD. Sp ranged from 0.23 at GIM to 0.26 at HMC to 0.59 at UCSD (where we deliberately oversampled screen-negative patients in our interviewed sample). The PPV of the screener ranged from 0.40 at GIM to 0.38 at HMC to 0.18 at UCSD. The NPV of the screener ranged from 1.00 at HMC to 0.99 at UCSD to 0.94 at GIM. These data are summarized in Table 1 , Option A.
|
As shown in Table 1, relatively modest increases in Sp and PPV accompany the more stringent definitions of positivity as one moves from Options A through C. This comes, though, at the cost of a reduction in Se as well as an increased burden for respondents to complete the additional questions required to ascertain whether these additional criteria are met (ie, filling out five questions instead of two).
Comparative Performance of the Beck Anxiety Inventory
At the UCSD site, subjects also completed the 21-item Beck Anxiety Inventory. Using cutoff scores from 5 to 40 in five-point increments, Se of the Beck Anxiety Inventory ranged from 0.94 to 0.06, Sp ranged from 0.45 to 0.98, PPV ranged from 0.14 to 0.57, and NPV ranged from 0.92 to 0.99. An ROC plot of these data are shown in Figure 1 . As indicated, if one were to use a Beck Anxiety Inventory cutoff score of 20 as optimal in terms of minimizing errors, values would be as follows: Se = 0.67, Sp = 0.93, PPV = 0.46, and NPV = 0.97. To achieve comparable Se to Option A of the ANS (ie, Se 0.94), one would choose a cutoff level of 5 on the Beck Anxiety Inventory. At this Beck Anxiety Inventory level, Se = 0.94, Sp = 0.45, PPV = 0.14, and NPV = 0.99, all of which are similar to the values obtained with Option A of the ANS. Using the Beck Anxiety Inventory, however, this comes at the expense of patient burden to complete the questionnaire (21 questions on the Beck Anxiety Inventory compared with two on Option A of the ANS).
|
|
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONAPPENDIXACKNOWLEDGMENTSREFERENCES |
|---|
Two extensively studied instruments in this regard are the SDDS-PC (44, 45) and PRIME-MD (40, 43). Both of these are intended to cover a wide range of psychiatric disorders, including major depression, generalized anxiety disorder, panic disorder, and alcohol and drug dependence. Both make use of a brief initial self-report screen and then use the results of this screen to draw the physician’s attention to potential diagnoses, which are covered in a systematized interview. It is thus difficult to compare the operating characteristics of these instruments to the very brief screener we developed, the ANS, which was designed specifically with panic disorder in mind. Still, it should be noted that the most recent (DSM-IV) version of the SDDS-PC has a somewhat superior PPV (0.43) for panic disorder compared with our instrument and similar NPV (0.97) (43). In future studies, it will be important to compare directly the utility of the ANS to other established diagnostic screens, such as the SDDS-PC and PRIME-MD. When these studies are conducted, it will be of interest to compare the instruments with regard to patient acceptability. Furthermore, it is unclear at this time whether a highly specialized instrument such as the ANS would find a place in general physicians’ offices or whether they would find one of the more comprehensive instruments (ie, one that screens for multiple psychiatric disorders, such as PRIME-MD or SDSS-PC) more useful.
How did the ANS fare as a diagnostic screening instrument for panic disorder in primary care? With two questions, the ANS achieves a level of sensitivity for panic disorder equivalent to any other screening instrument, including some that are considerably longer (30, 32, 43–45). The extremely high NPV indicates that our screener is highly unlikely to miss patients with panic disorder. On the other hand, the relatively low PPV indicates that a large proportion of patients will unnecessarily require a confirmatory test (eg, further interview with the physician), placing a heavier burden on clinician time than would be desirable. Thus, we presently advocate use of the ANS in settings where sensitivity is paramount and the extra costs imposed by the need for confirmatory interviews can be tolerated. An example of one such setting would be screening for panic disorder patients in primary care to include them in a clinical trial or other research study, where the costs of screening are borne by the study. A compelling argument could be made that the costs of untreated panic disorder are high enough to warrant the added expense of screening even under routine clinical circumstances (3, 8, 25, 26). At present, this remains a matter of conjecture, but one that merits further study.
|
APPENDIX |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONAPPENDIXACKNOWLEDGMENTSREFERENCES |
|---|
2. In the past 6 months, did you ever have a spell or attack when for no reason your heart suddenly began to race, you felt faint, or you couldn’t catch your breath? yes___ no___
IF YOU ANSWERED YES TO QUESTION #1 OR TO QUESTION #2, THEN CONTINUE WITH THE QUESTIONNAIRE, OTHERWISE, STOP.
3. Did any of these spells or attacks ever happen in a situation when you were not in danger or not the center of attention? yes___ no___
4. How many times have you had a spell or attacks in the past month? (Check one.) Hasn’t happened at all in the past month___ Once___ 2 to 3 times___ 4 to 10 times___ More than 10 times___
5. In the past month, how worried have you been that spells or attacks might happen again? (Check one.) Not at all worried___ Somewhat worried___ Very worried___
|
ACKNOWLEDGMENTS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONAPPENDIXACKNOWLEDGMENTSREFERENCES |
|---|
Received for publication May 6, 1998.
Revision received December 21, 1998.
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONAPPENDIXACKNOWLEDGMENTSREFERENCES |
|---|
This article has been cited by other articles:
 |
|
 |
|
  J. W. Smoller, M. H. Pollack, S. Wassertheil-Smoller, R. D. Jackson, A. Oberman, N. D. Wong, and D. Sheps Panic Attacks and Risk of Incident Cardiovascular Events Among Postmenopausal Women in the Women's Health Initiative Observational Study Arch Gen Psychiatry, October 1, 2007; 64(10): 1153 - 1160. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 W. J. Katon Clinical practice. Panic disorder. N. Engl. J. Med., June 1, 2006; 354(22): 2360 - 2367. [Full Text] [PDF]
|
|
|
|
 |
|
 M. G. Craske, M. J. Edlund, G. Sullivan, P. Roy-Byrne, C. Sherbourne, A. Bystritsky, and M. B. Stein Perceived Unmet Need for Mental Health Treatment and Barriers to Care Among Patients With Panic Disorder Psychiatr Serv, August 1, 2005; 56(8): 988 - 994. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. P. Roy-Byrne, M. G. Craske, M. B. Stein, G. Sullivan, A. Bystritsky, W. Katon, D. Golinelli, and C. D. Sherbourne A Randomized Effectiveness Trial of Cognitive-Behavioral Therapy and Medication for Primary Care Panic Disorder Arch Gen Psychiatry, March 1, 2005; 62(3): 290 - 298. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. B. Stein, C. D. Sherbourne, M. G. Craske, A. Means-Christensen, A. Bystritsky, W. Katon, G. Sullivan, and P. P. Roy-Byrne Quality of Care for Primary Care Patients With Anxiety Disorders Am J Psychiatry, December 1, 2004; 161(12): 2230 - 2237. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. P. Roy-Byrne, C. D. Sherbourne, M. G. Craske, M. B. Stein, W. Katon, G. Sullivan, A. Means-Christensen, and A. Bystritsky Moving Treatment Research From Clinical Trials to the Real World Focus, July 1, 2004; 2(3): 410 - 415. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. W. Smoller, M. H. Pollack, S. Wassertheil-Smoller, B. Barton, S. L. Hendrix, R. D. Jackson, T. Dicken, A. Oberman, and D. S. Sheps Prevalence and Correlates of Panic Attacks in Postmenopausal Women: Results From an Ancillary Study to the Women's Health Initiative Arch Intern Med, September 22, 2003; 163(17): 2041 - 2050. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. P. Roy-Byrne, C. D. Sherbourne, M. G. Craske, M. B. Stein, W. Katon, G. Sullivan, A. Means-Christensen, and A. Bystritsky Moving Treatment Research From Clinical Trials to the Real World Psychiatr Serv, March 1, 2003; 54(3): 327 - 332. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
W. J. Katon, P. Roy-Byrne, J. Russo, and D. Cowley Cost-effectiveness and Cost Offset of a Collaborative Care Intervention for Primary Care Patients With Panic Disorder Arch Gen Psychiatry, December 1, 2002; 59(12): 1098 - 1104. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. P. Roy-Byrne, W. Katon, D. S. Cowley, and J. Russo A Randomized Effectiveness Trial of Collaborative Care for Patients With Panic Disorder in Primary Care Arch Gen Psychiatry, September 1, 2001; 58(9): 869 - 876. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
