Non-significant in univariate but significant in multivariate analysis: a discussion with examples

Perhaps as a result of higher research standard and advancement in computer technology, the amount and level of statistical analysis required by medical journals become more and more demanding. It is now realized by researchers that univariate analysis alone may not be sufficient, especially for complex data sets. Additional, and sometimes even contradictory, results may be found using multivariate analysis. During the course of data analysis, a common practice is to include in multivariate analysis only those variables that are statistically significant in univariate analysis. Such a habit is risky as some variables not significant in univariate analysis may become significant in multivariate analysis. In this study, we identify, with examples, four possible scenarios in which the above situation could occur: (1) the effect of unbalanced sample size; (2) the influence of missing data; (3) an extremely large within group variation, relative to between group variation; and (4) the presence of interaction. In addition to detailed analysis steps, raw data sets are also available for readers to verify all the results presented. Although we only used the log-rank test and Cox regression for illustration purposes, the underlying concepts can be applied to other multivariate procedures such as the logistic regression and multiple linear regression.