Development and Validation of a Scale for Searching Skills in Evidence-Based Medicine

The aim of evidence-based medicine (EBM) is to use the best evidence to resolve clinical uncertainty and efficiently provide effective health care. The practice of EBM is composed of five steps (‘5A’): (1) asking a question; (2) acquiring relevant evidence; (3) appraisal of the evidence obtained; (4) applying the evidence to clinical care; and (5) auditing of the evidence-based performance. However, in the real world, busy healthcare professionals usually lack adequate capability and time to complete the whole 5A process. Only EBM researchers have enough ability and time in practicing the 5A and producing the best evidence for clinical application. Therefore the goal of EBM education is no longer expecting all healthcare professionals have the ability to go through the 5A process, but enabling them to acquire the best evidence efficiently. The current available assessment tools, for example Berlin questionnaire and Fresno test, were not designed for examining the ability to efficiently acquire the best evidence, which do not meet the current goal of EBM education. Moreover, the two tools were designed for residents or medical students and include essay and arithmetic questions, which are unsuitable for allied health professionals. Therefore we wish to develop and validate a scale for evaluating the EBM searching capability which can be applied to all kinds of healthcare professionals. We will design clinical scenarios that are common to all healthcare professionals and use structure performance test in assessing EBM searching capability. Our scale is composed to three components. The first component is characteristics of the examinee. The second component consists of three dimensions using analytic rating score: the first dimension measures the capability to propose key words for the EBM PICO (Patient /Problem、Intervention、Comparator、Outcome) question; the second dimension assesses the search strategy and skills; the third dimension assesses the search results. The third component is a global rating. We will organize a workforce of 7 experts and use modified Delphi technique to achieve consensus for revision of the scale. Two examiners will view standard videotapes of examinees with experiences of less than and at least 5 years. We will use Pearson’s correlation coefficient to evaluate the inter-rater reliability. The examiners will repeat the scoring 2 weeks later and calculate the intraclass correlation coefficient for the analysis of test-retest reliability. We will enroll 8 examinees from various healthcare professions in conducting the pilot study, and use the Cronbach’s α coefficient to evaluate the internal consistency. We will recruit 80 examinees from various healthcare professions for the formal study of our scale. We will apply the generalizability theory and use the analysis of variance to evaluate the reliability and validity (G-study). Then we will employ the Decision Study (D-study), which emphasizes the application and explanation of variance. The data from the G-study will be used to adjust the levels of various dimensions to reduce errors or increase the generalizability coefficient. Two reliability coefficients in the D-study, generalizability coefficient (Ep2 or G) and dependability index (φ) will be used for application and explanation. The application of the generalizability theory will be used in deciding the appropriate number of items and constructing the quality of the scale.

Project ID:PF10406-3008
External Project ID:MOST104-2511-S182-004