| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH |
(See below for detailed descriptions)
Updated: 04/13/2006
Psychosomatic Medicine adopted statistical guidelines in 2005. The initiative was introduced in an editorial, in which the editors outlined their goals for the guidelines:
First, they will advance the journal’s mission of promoting methodologically rigorous psychosomatic research. Second, they will enable our contributors to avoid some of the most common statistical criticisms when they submit manuscripts for review, and they will spare our reviewers from having to consider these problems. Third, they will reduce uncertainty about the acceptability of certain statistical methods.The editors went on to note that the guidelines are not requirements but requested that authors who wished to deviate from them should make a convincing case. For further guidance on statistical matters, please refer to the journal’s Statistical Corner series and to the references listed below.
Psychosomatic Medicine encourages contributors to follow the statistical reporting recommendations presented in two standard manuals, How to Report Statistics in Medicine (Lang and Secic, 1997) and the Publication Manual of the American Psychological Association, 5th edition (2001). We also recommend Cleveland’s 1994 book, The Elements of Graphing Data, as a general guide to the effective graphical presentation of data.
Contributors are asked to consult these manuals as needed when deciding how to describe their statistical methods and how to present their results. Please cite supporting material from these sources in your response if you disagree with a reviewer’s position on a statistical reporting issue. Since these manuals offer little guidance on how to report the results of some of the newer, more advanced methods, it may be necessary to consult other authoritative sources as well.
References
APA Publication manual. 5th ed. Washington, D.C.: American Psychological Association; 2001.
Cleveland WS. The elements of graphing data. Summit, New Jersey: Hobart Press; 1994.
Lang TA, Secic M. How to report statistics in medicine: annotated guidelines for authors, editors, and reviewers. Philadelphia: American College of Physicians; 1997.
2. One-Tailed (Directional) Hypothesis Tests
One-tailed (directional) hypothesis tests have been controversial for decades. Some authorities recommend them, others discourage them. We discourage them because they are less conservative than two-tailed tests, and because they tend to catch readers off guard. This guideline is intended to prevent the controversy and confusion that can arise when a one-tailed test produces a significant result that would have been nonsignificant had a two-tailed test been used instead.
References
Hays WL. Statistics. 5th ed. Forth Worth, Texas: Harcourt Brace; 1994. p. 293-299.
Moyé LA. Statistical reasoning in medicine: the intuitive p-value primer. New York, NY: Springer-Verlag; 2000. Chapters 7 and 8.
3. Artificial Categorization of Variables
Whenever possible, artificial dichotomies or categorizations should be avoided. The reasons for this are well described in a number of excellent reviews, such as MacCullum et al. (2002). A possible exception to this guideline arises when it is difficult to operationalize a construct as a unitary, continuous variable (e.g., hypertension). Even in this case, however, adjunctive modeling of the continuous scores (e.g., systolic and diastolic blood pressure) can be very informative.
The availability of widely used clinical cutpoints for a scale or measure is not an acceptable rationale for creating artificial categories for the primary or sole analysis. If one of the aims of an analysis is to identify cutpoints, it is far better to use fitted values from a model that uses the continuous form of the predictor variable as the basis for the cutpoints than to make the cuts before modeling. In other words, model first, cut later.
Risk limits and effect sizes for continuous variables should be estimated using clinically or theoretically interesting predictor values, e.g., the 25th and 75th percentile value of the predictor. The 25/75 scaling is preferred in that it will always have a meaningful interpretation—it compares an individual in the middle of the upper half of the predictor’s distribution with an individual in the middle of the lower half. Scaling to standard deviations is also acceptable, but if the predictor is not symmetrically distributed, the meaning of a standard deviation can be dubious. Alternatively, the continuous variable can be scaled to a meaningful difference on the predictor. For example, if a 6-point difference on the Beck Depression Inventory is considered “clinically meaningful,” it would be reasonable to scale this score in 6-point increments. This would be done by dividing the original scores by 6. The model value (e.g., the hazard ratio) would then represent the effect of a 6-point change on the BDI, rather than of a one-point change.
References
Cohen, J. (1983) The cost of dichotomization. Applied Psychological Measurement, 7, 249-253.
Maxwell, SE, & Delaney, HD (1993). Bivariate median splits and spurious statistical significance. Psychological Bulletin, 113, 181-190
Frank Harrell's page on dichotomization of continuous variables
4. Automated Stepwise Selection Procedures
Automated stepwise techniques often produce wildly unreliable results. This includes not only forward and backward automated selection, but also “best subset” approaches. Manuscripts that employ these techniques will not be considered for review unless the model is supported by a validation procedure, such as external replication, or an internal penalization or bootstrap shrinkage procedure as described in the references below.
References
Harrell FE. Regression modeling strategies. New York: Springer; 2000.
5. Covariables and Covariate Adjustment
The traditional practice of pre-screening covariates by choosing ones with significant univariate p values is well known to bias the results of multiple regression models. The choice of covariates should be based as much as possible on external information such as previous research and clinical knowledge. For example, it is often necessary to control for the severity of medical illness when modeling the effects of psychological variables on medical outcomes. The decision to include a particular index of medical illness severity as a covariate should based primarily on such a priori considerations as whether previous research suggests that it is a potential confounder of the relationship of interest. It should not be based on whether the index was significantly associated with the psychological variable and/or the medical outcome in preliminary univariate analyses.
In addition, exceeding the recommended ratio of observations to predictors (or of events per predictor in the case of logistic or survival models) yields unreliable estimates. When confronted with too many predictors for the data at hand, we strongly recommend eliminating redundant predictors or combining ones that are highly correlated. This applies not only to the predictor variables that are of primary interest to the investigator, but also to a) covariates that are included to control for potential confounders and b) covariates that are included because they are well-established predictors and the investigators are trying to determine whether the variable of interest has independent predictive value.
More generally, we recommend consulting the NIH guidelines for prognostic modeling (Steyerberg and Harrell, 2003).
References
Harrell FE. Regression modeling strategies. New York: Springer; 2000.
Model validation enables the scientific community to determine whether a reported model is likely to replicate in future samples. Even if external validation on a new set of data is not possible, it usually is possible to assess a model’s internal validity. This can be achieved a number of ways, including the shrunken R-square statistic that is available in most general linear model/regression packages, or more sophisticated approaches, such as penalized likelihood or bootstrap shrinkage. For structural equation models, indices of expected cross validation can be reported. Simpler methods, such as randomly splitting the data into a training and validation set, can be used, but these techniques have been shown to be too conservative.
Consequently, we recommend the use of state-of-the-art model validation techniques. However, we recognize that the more advanced approaches to model validation can be difficult to implement, and that the necessary software may not be accessible to some investigators. We are therefore willing to consider papers in which the approach to model validation is falls short of the current state of the art. Authors should recognize, however, that this may make it harder to convince the reviewers that the study’s findings are likely to be replicable.
References
Harrell FE. Regression modeling strategies. New York: Springer; 2000.