Testing the revised barriers scale
Testing the revised barriers to research utilization scale for use in the UK G.W. Marsh, M. Nolan and S. Hopkins
Gene W. Marsh RN, PhD, Professor of Acute and Critical Care Nursing Mike Nolan RGN, PhD, Professor of Gerontological Nursing, University of Sheffield, School of Nursing and Midwifery,Winter Street, Sheffield 53 7ND, UK.
The drive towards evidence-based practice and clinical effectiveness is a dominant theme within the NHS in the UK (Newman et al. 1998) and figures prominently in several recent policy documents (DoH 1997, 1998). However, despite the expenditure of large sums of money on research and development in the NHS, commissioned studies often have limited impact on practice (Dalziel 1999). Exactly why there is so little implementation of the available evidence is unclear but a formidable array of barriers have been identified in a number of domains (White et al. 1995). The vexed issue of the relative failure of research to inform nursing practice has been the subject of debate and study for several decades (Funk et al. 1995), but with no clear resolution. In an attempt to better identify and understand those factors that inhibit the use of research, the Barriers to Research Utilization Scale (Funk et al. 1991a, b) has been used to measure the magnitude and scope of perceived barriers that prevent nurses from using research findings to guide clinical decision making and practice (Walsh 1997a,b, Dunn et al. 1998, Kajermo et al. 1998, Kirshbaum 1999, Retsas & Nolan 1999). The rationale exists that if barriers are adequately identified and measured, strategies to overcome them can be implemented in practice settings, thus improving patient care by assuring that practice is evidence based. One question that arises, however, is whether or not the BARRIERS Scale, developed and tested in the USA, can adequately reflect the perceptions of nurses in the UK when the language prompting item responses may have different or unfamiliar colloquial meanings or interpretations.
Sue Hopkins Head of Student Support, Central Sheffield University Hospitals NHS Trust
Correspondence to: Gene W. Marsh,Tel.: +44 (0) 114 222 9775; E-mail: [email protected]
The main purpose of this study on which this paper is based was to identify and describe the perceived
Clinical Effectiveness in Nursing (2001) 5, 66–72 © 2001 Harcourt Publishers Ltd doi: 10.1054/cein.2001.0192, available online at http://www.idealibrary.com.on
barriers to research utilization perceived by nurses and midwives in two large NHS Trusts in Sheffield, to compare and contrast findings across trusts, and to develop and implement strategies to positively influence the research culture and evidence-based practice within each Trust. However, important secondary aims were to test the revised version of the Barriers to Research Utilization Scale (Funk et al. 1991a) that reflected UK-friendly language, determine the adequacy of the revised scale psychometric properties and make recommendations for using the scale in the UK.
CHARACTERISTICS OF THE BARRIERS SCALE The 29-item Barriers to Research Utilization Scale was originally developed in the United States by Funk, Champagne, Wiese, and Tornquist at the University of North Carolina (1991a,b). The Scale content was identified and developed from the available literature, the Conduct and Utilization of Research in Nursing Questionnaire (CURN) and informal data collected from nurses (Crane et al. 1977, Funk, et al. 1991b). Each item describes a potential barrier to research utilization such as, ‘the nurse does not feel she/he has enough authority to change patient care’. Respondents are asked to rate the extent to which they think each item is a barrier to nurses’ use of research to change or enhance practice on a four-point scale, with higher scores representing greater perceived barriers (Funk et al. 1991a,b). A fifth option, ‘no opinion’, is also provided but this response should be analysed separately (Sandra Funk, personal communication, August, 1998).
Subscale Dimensions The scale comprises four subscales that reflect perceived barriers that correspond to major concepts in
Testing the revised barriers scale 67
Rogers’ (1983) model of innovation diffusion. The concepts identified by Rogers are the innovation, the communication channel, time, and the social system. Rogers (1995) defines time as being related to the ‘adopter’ of an innovation. Specifically it is the amount of time necessary for a potential adopter of an innovation to become knowledgeable, be persuaded to accept or reject an innovation, and if accepted to implement the innovation and confirm its usefulness. Characteristics of the research, such as methodological rigor and appropriateness of conclusions, correspond to Rogers’ (1995) innovation. Funk et al. (1991a) represents Rogers’ communication channels as the presentation of the research including its accessibility. Time, or characteristics of the adopter of the research are represented by the nurse. Characteristics such as the nurses’ research values, skills, awareness, and the time involved to move through the process of adopting the research innovation are included. The fourth subscale dimension, organizational setting, corresponds to Rogers’ (1995) social system.
Face and content validity of the scale were established by a panel of judges (Funk et al. 1991a) and the scale was originally tested on a random sample of 5000 USA nurses, stratified by educational level, with a 40% return rate (Funk et al. 1991a). Principal components and principal axis factor analyses supported four independent factors consistent with Roger’s theory. All items but one loaded on factors that were theoretically congruent with Rogers’ (1995) critical concepts.
community health trust and then compared the perceptions of nurses across the directorates of the same trust. Marsh et al. (1999) compared findings across two large acute-care trusts. In contrast, other researchers have studied barriers perceived by nurses within specific areas of practice. Kirshbaum (1999) studied perceived barriers among breast care nurses from England, Scotland and Wales, and Walsh (1997b) compared differences and similarities among community and acute-care nurses. Researchers generally analyze BARRIERS Scale data by ranking the frequency and percentage of respondents reporting a scale item as a moderate or great barrier (Walsh 1997b, Nolan et al. 1998, CHS 1999, Marsh et al. 1999, Retsas 2000). Subscale dimensions are then examined and compared to determine the dimension/s (setting, presentation, research, nurse) that reflects the greatest barrier/s to research utilization. Across studies, the organisational setting followed by the presentation of and access to findings pose the greatest barriers (Walsh 1997b, Kajermo et al. 1998, Nolan et al. 1998, CHS 1999, Marsh et al. 1999, Retsas 2000). Frequently, researchers compare both individual items and subscales across samples (Kirshbaum 1999) across intra-organizational settings (CHS 1999), across organisations (Walsh 1997b, Kajermo et al. 1998, Marsh et al. 1999), and internationally (Walsh 1997b, Dunn et al. 1998, Nolan et al. 1998, Retsas 2000). Expected differences in item rankings and subscale scores across settings support the Scale’s ability to descriminate between alternative organizational settings and study populations.
Internal consistency reliability estimates revealed Cronbach’s alphas for the subscales of: Nurse, .80; Setting, .80; and Research, .72 with adequate item to toal correlations (Funk et al. 1991a). The alpha for the Presentation subscale was lower (.65), but deletion of one item would have substantially decreased the alpha. Therefore, all items plus the factor were retained (Funk et al. 1991a). One-week test-retest stability estimates ranged from .68 to .83 on a sample of 17 nurses (Funk et al. 1991a).
Among the studies for which BARRIERS Scale psychometric properties are reestimated. Reliability is generally above .70 for the Nurse, Setting, and Research subscales (Dunn et al. 1998, Kajermo et al. 1998, CHS 1999). However, Dunn et al. report a Research subscale alpha of .67. Alphas for the presentation subscale have been lower and range from .47 (Dunn et al. 1998) to .65 (Funk et al. 1991a). Kajermo et al. (1998) reported subscale alphas ranging from .81 to .96. However, the study was conducted in Sweden and the BARRIERS Scale underwent translation into Swedish and back translation to English, illustrating that culturally sensitive language may yield a more reliable instrument. With the exception of Kajermo et al.’s. study, the reviewed studies utilised the USA version of the BARRIERS Scale. However, the researchers who factor analyzed the data to estimate construct validity, were unsuccessful in replicating Funk et al.’s (1991a) factor structure (Dunn et al. 1998, Kirshbaum 1999, CHS 1999, Retsas & Nolan 1999, Retsas 2000). Based on data from Austrialian
LITERATURE REVIEW The literature and recent conference presentations reveal that researchers in the UK, Sweden, and Australia use the USA developed BARRIERS Scale to identify and describe barriers to research utilization within specific organizational settings (Kajermo et al. 1998, Nolan et al. 1998, Community Health Sheffield 1999, Marsh et al. 1999, Retsas 2000). For example, Community Health Sheffield (1999) examined the barriers to research utilization within a large
68 Clinical Effectiveness in Nursing
nurses, Retsas and Nolan (1999) found that three factors utilising 26 items best described the data whilst Retsas (2000) confirmed an alternative four factor structure. One study was particularly relevant to this investigation. Dunn et al. (1998) examined the adequacy of the USA BARRIERS Scale’s psychometric properties when it was administered to UK nurses. Dunn et al. (1998) utilized the original 29-item scale to collect data on a convenience sample of 316 UK nurses including: 139 clinical nurse specialists, 132 nurses involved in elderly care, and 45 nurses enrolled in a one-day critical appraisal course. Results revealed subscale internal consistency reliability ranging from .48 to .78, with a total scale alpha of .85 suggesting that when used in the UK this scale is less reliable when compared to the findings that Funk et al. (1991a,b) reported. The factor model proposed by Funk et al. (1991a), depicting four subscale dimensions, was not supported by UK data. This later finding raises the question of whether or not the language of the USA form is less adequate for UK use and if minor item revisions to reflect UK friendly language would better support the original factor model. The results of a study exploring the use of a revised Barriers scale form the substance of this paper.
METHODOLOGY Setting and Sample This methodological research was conducted at The Northern General Hospital Trust (NGH) and The Central Sheffield University Hospitals Trust (CSUHT) both in Sheffield. Both settings were university teaching hospitals, with about 1000 beds each, and serve the Trent Region. Although demographically similar, the provision of services across Trusts naturally varies to minimise replication. For example, the NGH provides the majority of accident, emergency and trauma care to Sheffield. The CSUHT provides Obstetric and Gynaecological services. Thus, the CSUHT sample consisted of both nurses and midwives whilst the NGH sample consisted of nurses only. All nursing and midwifery staff at the two study sites were invited to participate in the study.
Instrument Revision Prior to distributing the scale to the UK sample, a small pilot study was conducted to assess scale terminology and comprehensibility (Nolan et al. 1998). Minor changes in wording were made to 18 of the 29 items. For example, ‘physician’ was changed to ‘doctor’ and ‘administrator’ to ‘manager’, and the item ‘The research has not been replicated’ was changed to ‘The research findings are only based on a one off study’.
Data Collection Approximately 1400 Revised Barriers Scales were distributed to all qualified nursing staff at the NGH with their salary advice and 382 were returned complete, representing a 27% response rate. At the CSUHT 1509 scales were similarly distributed; 563 were returned representing a 37.3% response rate. However, only 549 (36.4%) were suitable for analysis. Data were collected from the NGH in 1996 and from the CSUHT in 1998. During this time gap, CSUHT employed a new Director of Nursing. The extent to which this may have biased CSUHT responses is unknown. Based on characteristics with the highest proportion of responses, the typical respondent from NGH and CSUHT respectively was a full time (65.5%, 66.8%), E Grade (33.5%, 39.5%), staff nurse (67.3%, 53.6%), who had no post-registration degree (81.2%, 91%), and who claimed to have last read a research based article within the past week (43.5%, 54.1%). Most nurses at NGH (50.4%) had received their initial registration between 1990 and 1994, at the CSUHT most registration (38.2%) occurred between 1985 and 1989. A comparison of the samples with the broad composition of each Trust’s nursing work force suggested no obvious bias in respondents and the data set was large enough for factor analysis (13 subjects per item at NGH and 19 subjects per item at CSUHT). Nunnally (1978) recommends at least 10 subjects per item for factor analysis. Data were analyzed separately for the two samples. Replication was more consistent with the research purposes than the expected results from aggregating the two data sets. Because of the similar demographic and geographic sample characteristics, major differences across settings were not anticipated.
RESULTS The main purpose of this paper was to describe the psychometric properties of the revised scale. Prior to this discussion, however, attention is briefly directed toward the major perceived barriers identified. Descriptive analysis yielded separate rankings of those items perceived as a great or moderate barrier as well as subscale descriptive statistics (see Table 1). Both samples ranked the same top 10 items as moderate or great barriers to research utilization. Sixty-three to 82% of nurses at CSUHT reported that each of the top ten items were moderate or great barriers to research utilization, and 62–84% of nurses at NGH similarly reported the top 10 items as moderate or great barriers. However, the position of the items within the rankings varied across samples. Eighty-four per cent of the NGH nurses and 82% of the CSUHT respondents ranked ‘time to
Testing the revised barriers scale 69 Table 1 barrier
Ranking of Barriers by percentage of respondents reporting item as a great or moderate
1 2 3 4
1 2 4 3
5 6 7 8 9 10
10 6 9 5 8 9
There is insufficient time at work to implement new ideas Resources are inadequate for implementation The statistics are difficult to understand The nurse does not feel that she/he has enough authority to change patient care procedures The relevant research literature is not available in one place Doctors will not cooperate with implementation The nurse does not have time to read research The research is not easy to read and understand Other staff are not supportive of implementation The nurse does not know what research is available
implement new ideas’ as the number one barrier to research utilization and ‘inadequate resources for implementation’ ranked second with 81% of the NGH nurses and 80% of the CSUHT respondents reporting that inadequate resources were a moderate or great barrier.
Subscale item means The two Trusts were very similar in their overall perceptions. Comparing the item means of each subscale initially revealed that nurses in both Trusts viewed the organizational setting as the greatest barrier to research utilization. The validity of comparing subscale means and recommended cautions are explored later.
Revised scale properties Content validity Content validity was re-examined to determine the adequacy of the sampled content domain and to determine if the scale adequately represented the world of content about existing barriers. CSUHT participants also had provided qualitative responses to the question: ‘What are the things that you think facilitate research utilization?’ The question yielded 549 responses. These data underwent content analysis performed by six nursing lecturers at the University of Sheffield who had no prior knowledge of the Scale’s structure or of the quantitative results. Thirteen categories emerged from the data and were defined. The four subscale dimensions of Research, Presentation, Nurse, Setting (Funk et al. 1991a, b) were then presented to the content analysis research team. The four subscale dimensions are displayed in Table 2 with the emergent categories from the qualitative data. Twelve categories were judged to be congruent with one of the four subscale dimensions. However, one category, information technology (IT), failed to fit a single concept as it appeared to provide new information that overlapped several dimensions. IT infrastructure is congruent with the organizational setting subscale whilst confidence in using IT is consistent with characteristics of nurses, and internet
% CSUHT (n = 549)
NGH (n = 382)
82% 80% 76%
84% 81% 73%
74% 71% 71% 71% 66% 63% 63%
74% 62% 70% 69% 72% 67% 66%
access theoretically fits with the presentation/ access subscale. Use of the original BARRIERS scale was first reported in the late 1980s. The prominence of IT for obtaining research information in clinical settings has grown substantially since that time. Therefore, additional items may need to be constructed to index the adequacy of nurses’ access to electronic information, databases, and on-line journals.
Construct validity Funk et al.’s (1991a) factor analysis scheme was employed to estimate construct validity. The principal components factor analysis identified four factors underlying both data sets, suggesting that Funk’s factor model possibly existed. The principal components analysis was followed by a confirmatory factor analysis that specified the extraction of four factors using the principal axis method that facilitated examining the shared variance and communalities of items as well as determining which items of the Revised Barriers Scale had the greatest association with one of the four factors. The association of an item with a factor is represented by the item’s factor loading score. Higher scores indicate a stronger relationship between item and factor. The criterion for an item loading on one factor was a factor loading score of .40 with a .20 margin of difference between the primary factor loading score and all loading scores with alternative factors. In the CSUHT this resulted in seven items loading on factor one, seven items on factor two, and three items on factors three and four. Two items loaded on multiple factors and seven items failed to load. For the NGH data seven items loaded on factor one, four items on factor two, six items loaded on factor three and three items loaded on factor four. Five items loaded on multiple factors and four items failed to meet the loading criterion. Unlike Funk et al.’s (1991a) data that resulted in all but one item loading on one of the four factors, the data generated by the two samples in this study were difficult to interpret. Items from all four of the subscales loaded inconsistently across the extracted factors for both of the data sets creating a factor structure that was impossible to interpret (see Table 3).
70 Clinical Effectiveness in Nursing Table 2 Emergent categories from qualitative responses sorted by subscale dimension Presentation/Accessibility
Patient benefits Quality of research
Education Confidence, ‘fear factor’ Knowledge base
Time Staff levels Support from colleagues Support from medical staff Managerial support and leadership Culture
Table 3 Number of items loading on Factors for CSUHT (n = 549) and NGH (n = 382)
Factor 1 Factor 2 Factor 3 Factor 4 ≤ .40 loading score
7 7 3 3 7
7 4 6 3 4
* Factor loading criteria ≥ .40; ≥ .20 margin with other factors
On the basis of data generated from this study the model proposed by Funk et al. (1991a) was not supported. Therefore, the scale has limited subscale validity when either the original or revised version is administered to UK nurses. The finding was evident in both samples and was similar to the findings reported by Dunn et al. (1998). One must question if Rogers’ (1995) complex theory of innovation diffusion has been oversimplified, under-represented, or misspecified by the original BARRIERS Scale. The overarching construct of barriers to research utilization, however, is well referenced by the total 29 items, and this is further supported by the content analysis of qualitative data that demonstrates a fit with the total scale. Overall, the scale is measuring what is intended, that is barriers to research utilization in nursing practice. Another way to demonstrate construct validity is to examine if the scale or construct performs as expected (Nunnally 1978). One can hypothesize that some items will demonstrate similar rankings when comparing studies that were conducted in the UK, and different rankings when comparing UK results with USA studies. When comparing the rankings of individual barriers items across the two sites in this investigation, Dunn et al.’s (1998) sample of UK nurses, and Funk et al.’s (1991a) sample of USA nurses, a pattern of similarity emerges between the three UK sites. Yet, these findings diverge from the USA findings.
Revised Scale Reliability Internal consistency was estimated using Cronbach’s alpha. Subscale alphas were adequate except for the presentation subscale (CSUHT, alpha = .67; NGH,
alpha = .61). However, in light of the failure to support subscales in the UK data, this is of minor concern and the total scale alpha then is the best estimate of reliability. Total scale alphas were well above the criterion of .80. (CSUHT, .89; NGH, .87). The reliability estimates are higher than those reported by Dunn et al. (1998), and are more similar to Funk et al’s findings. All item-total correlations for both sites ranged from .31 to .60, and deleting any items would have deflated the alpha. Neither stability nor sensitivity were assessed in this investigation. However, future research should focus on both analyses as it is important to know the extent to which the Revised Barriers to Research Utilization Scale can detect changes in perceived barriers over time.
DISCUSSION The purpose of this investigation was to test the Revised Barriers Scale and estimate the Scale psychometric properties. Recommendations for using the Scale within the UK are then derived from these findings.
Effect of Item Revision Item revision most likely had no effect on the instrument content validity as the researchers made every effort to preserve the original scale content. The content analysis of the qualitative data also indicated a well-referenced content domain. Therefore, the effect on content validity was most likely neutral. The effect of item revision on construct validity is a more difficult assessment. Rewriting items possibly may have changed the meanings of intended constructs and could have accounted for the murky factor analysis results. However, the lack of factor clarity in this study was similar to Dunn et al.’s (1998) experience with the original USA BARRIERS Scale. These results suggest that the factor structure of the original scale was not supported in the international setting. Therefore, it is difficult to detect and hence unknown if item revision had a major effect on the construct validity. Additional research is necessary to clarify this question.
Testing the revised barriers scale 71
Dunn et al’s (1998) results with the USA scale on an UK sample demonstrated lower reliability than when the scale was administered in the USA. Rewriting items may have resulted in improved clarity, more consistent responses and therefore less random measurement error. Therefore, the revised scale appears to have improved reliability over the original scale when administered in the UK. A similar result is evident in Kajermo et al.’s study of Swedish nurses. These findings suggest that culturally sensitive language does appear to be associated with higher scale reliability.
Recommendations for Use in UK Based on the current research findings, several recommendations are made regarding the use of the Revised Barriers to Research Utilization Scale with UK subjects. The revised scale is suitable for use as a diagnostic tool for ranking barriers by item mean scores as well as frequency and percentage of respondents assessing items as a great or moderate barrier. The revised scale is acceptable for investigating perceived barriers in the acute care setting. However, it has not been tested in other settings. The original scale has performed adequately in a variety of settings (Walsh 1997b, CHS 1999) and presumably the revised scale may also be transferable across clinical sites. The original BARRIERS Scale needs updating to reflect changes in the way nurses access research findings. To improve the scale content validity, new items are needed to reference accessibility, knowledge, and use of electronic information sources. The total scale reliability is adequate, and in this study the revised scale demonstrated higher reliability than did the original USA form when administered to UK respondents. However, much caution is advised in interpreting the findings by subscale mean scores because the underlying subscale dimensions were not upheld with UK samples (Dunn et al. 1998, Marsh et al. 1999). In future research both stability and sensitivity should be studied. Cautious interpretation is also advised when making international comparisons with the original USA data that are now more than a decade old. In the past decade, USA nurses have witnessed advances in technology, education, research funding, autonomy of nursing roles, and multidisciplinary collaboration. These forces may have altered nurses’ use of research findings in practice, and the original data (Funk et al. 1991a, b) may be too outdated to support valid comparisons.
CONCLUSION The findings should contribute to the rapidly growing understanding of barriers research and help to clarify how the Revised Barriers Scale may be
used to identify and rank perceived barriers in practice settings within the UK. The results from our two samples show reasonable similarities with the top 10 barriers being identified, albeit in slightly differing order (see Table 1). The majority of these barriers are described by items related to the organizational context and are consistent with the conclusions of Newman et al. (1998) who cite a formidable list of organisational impediments to the use of research. Further confirming the importance of organizational influences, our qualitative data indicate that the greatest facilitators of research are also to be found within a supportive culture. Therefore, while strategies to improve the use of research need to be targeted at multiple levels, the central importance of local contact cannot be ignored (Hutchinson 1998). Although our data do not support the use of the Barriers Scale as proposed by Funk et al. (1991a, b) they do support its value as a general diagnostic tool that provides valuable data to help identify local barriers and devise strategies to overcome them. REFERENCES Community Health Sheffield 1999 Results form the ‘barriers questionnaire’ executive summary. Community Health Sheffield NHS Trust (CHS), Research and Development, unpublished report Crane J, Peltz D, Horsley JA 1977 CURN project research utilization questionnaire. Ann Arbor, MI: Conduct and Utilization of Research in Nursing Project. School of Nursing, The University of Michigan Dalziel M 1999 R, D in the NHS, an overview. National Co-ordinating Centre for NHS Service Delivery and Organisation, London School of Hygiene and Tropical Medicine Department of Health 1997 The new NHS modern, dependable. London, Department of Health Department of Health 1998 A first class service. London, Department of Health Dunn V, Crichton N, Williams K, Roe B, Seers K 1998 Using research for practice: a UK experience of BARRIERS Scale. Journal of Advanced Nursing 26: 1203–1210 Funk SG, Champagne MT, Wiese RA, Tornquist EM 1991a BARRIERS: The Barriers to Research Utilization Scale. Applied Nursing Research 4: 39–45 Funk SG, Champagne MT, Wiese RA, Tornquist EM 1991b Barriers to Using Research Findings in Practice: The Clinician’s Perspective. Applied Nursing Research 4: 90–95 Funk SG, Tornquist EM, Champagne MT 1995 Barriers and facilitators of research utilization: an integrative review. Nursing Clinics of North America 3: 395–408 Hutchinson A 1998 The philosophy of clinical practice guidelines, purposes, problems, practicalities and implementation. Journal of Quality & Clinical Practice 18: 63–73 Kajermo KN, Nordstrom G, Krusebrant A, Bjorvell H 1998 Barriers to and facilitators of research utilization, as perceived by a group of registered nurses in Sweden. Journal of Advanced Nursing 27: 798–807 Kirshbaum M 1999 Barriers to research utilization for breast care nurses: Proceedings of the 1999 Royal College of Nursing Annual Research Conference held at Keele University. Keele, Staffordshire, England, Keele University
72 Clinical Effectiveness in Nursing Marsh G, Nolan M, Hopkins S 1999 Examining the barriers to nursing and midwifery research utilization: Proceedings of the 1999 Sheffield Nursing & Midwifery Research Partnership Group held at Ranmoor House. Sheffield, England Newman M, Papadopoulos I, Sigsworth J 1998 Barriers to evidence-based practice. Clinical Effectiveness in Nursing 2: 11–20 Nolan M, Morgan L, Curran M, Clayton J, Gerrish K, Parker K 1998 Evidence-based care: can we overcome the barriers? British Journal of Nursing 7: 1273–1278 Nunnally JC 1978 Psychometric theory, 2nd edn. New York, McGraw-Hill Retsas A 2000 Barriers to using research evidence in nursing practice. Journal of Advanced Nursing 31: 599–606
Retsas AP, Nolan M 1999 Barriers to nurses’ use of research: an Australian hospital study. International Journal of Nursing Studies 36: 335–343 Rogers EM 1983 Diffusion of Innovations, 3rd edn. New York, The Free Press Rogers EM 1995 Diffusion of innovations, 4th edn. New York, The Free Press Walsh M 1997a Perceptions of Barriers to implementing research. Nursing Standard 11: 34–37 Walsh M 1997b How nurses perceive barriers to research implementation. Nursing Standard 11: 34–39 White JM, Leske JS, Pearcy JM 1995 Models and processes of research utilization. Nursing Clinics of North America 30: 409–420