Home About Journal AHEAD OF PRINT Current Issue Back Issues Instructions Submission Search Subscribe Blog    

Users Online: 2515 
Print this page  Email this page Small font sizeDefault font sizeIncrease font size 

SYMPOSIUM Table of Contents   
Year : 2008  |  Volume : 42  |  Issue : 2  |  Page : 144-149
How good is the orthopaedic literature?

1 Department of Surgery, McMaster University, Hamilton, Ontario, Canada
2 Division of Orthopedic Surgery, Millcreek Community Hospital, Erie, PA, Canada
3 Department of Orthopaedic Surgery, Boston University, Boston, MA, USA

Click here for correspondence address and email


Randomized trials constitute approximately 3% of the orthopaedic literature Concerns regarding quality of the orthopaedic literature stem from a widespread notion that the overall quality of the surgical literature is in need of improvement. Limitations in surgical research arises primarily from two pervasive issues: 1) A reliance on low levels of evidence to advance surgical knowledge, and 2) Poor reporting quality among the high level surgical evidence that is available. The scarcity of randomized trials may be largely attributable to several unique challenges which make them difficult to conduct. We present characteristics of the orthopaedic literature and address the challenges of conducting randomized trials in surgery.

Keywords: Critical appraisal, evidence-based medicine, quality, randomized trials

How to cite this article:
Chaudhry H, Mundi R, Singh I, Einhorn TA, Bhandari M. How good is the orthopaedic literature?. Indian J Orthop 2008;42:144-9

How to cite this URL:
Chaudhry H, Mundi R, Singh I, Einhorn TA, Bhandari M. How good is the orthopaedic literature?. Indian J Orthop [serial online] 2008 [cited 2020 Feb 25];42:144-9. Available from:

   Evidence-based Medicine in Practice Top

In everyday clinical practice, evidence-based decision making depends on the integrated assessment of three critical factors-availability of clinical resources, patient values and the best-available evidence. [1],[2] As the need for orthopaedic care rises globally, [3],[4],[5],[6],[7],[8] it is imperative that these three factors guide orthopaedic care providers in their treatment decisions in order to ensure that patients receive optimal standards of care. Although judgments regarding clinical resources and patient values can be made on the physician's own accord, successful incorporation of evidence into the decision-making process depends directly on the existence of high quality literature. The following review will: 1) briefly outline the reporting practices that define high quality reporting and 2) assess the current quality of reporting in the orthopaedic literature.

   Characteristics of High Quality Evidence Top

The hierarchy

With the numerous studies of varying credibility being published constantly, it is essential that physicians have a sound understanding of both study design and study quality. This will enable them to 'critically appraise' the studies they read and safely adopt changes to their practice.

Hierarchies of evidence have been established based on study design to aid physicians in evaluating the credibility of the studies they read. Different hierarchies of evidence exist for the various classes of studies, including those focusing on therapy, prognosis, harm, economic analysis and overviews. Studies focusing on therapy are the most common class of study found in the orthopaedic literature. [9] A hierarchy of evidence for such studies is presented in [Table - 1]. However, it must be noted that the quality of studies within each hierarchical category may vary based on the extent to which methodological safeguards are employed by study authors. As discussed below for randomized controlled trials, high quality reporting entails the adequate reporting of such methodological safeguards.

Characteristics of high quality RCTs

The randomized controlled trial (RCT) has been established as the most secure study design for arriving at accurate estimates of treatment effects. This accuracy is contingent upon the process of randomization, which ensures that treatment and control groups are balanced for both known and unknown prognostic factors (characteristics of patients that influence outcomes). Although randomization can circumvent the introduction of bias by balancing groups for prognostic factors, there are several other measures-in addition to randomization-that must be taken to protect randomized trials from biased results.

One such measure referred to as allocation concealment entails that those recruiting patients into the study are unaware of which treatment arm the next patient will be delegated to. Failure to adequately implement allocation concealment in the design of an RCT has been shown to produce biased results that overestimate the effect of treatment. [10] Other important safeguards that should be reported include blinding, loss to follow-up, sample size calculations and adherence to the intention-to-treat principle (i.e. analyzing patients in the groups they were originally randomized to regardless of which treatment they actually received).

Surgical randomized trials also have unique and special challenges that, if not addressed, can act as potential sources of bias. Firstly, surgical procedures consist of several components including preoperative care, the anesthetic procedure, the main intervention and post-operative care. Thus, only adequate reporting can allow readers to decipher which components of care were kept consistent or altered between the treatment arms. Secondly, in surgical procedures the skill and experience of the surgeon directly influences the patient's final outcome. Having treatment arms with imbalances in surgeon expertise would lead to the potential introduction of biased results. Finally, the type of setting (i.e. primary, secondary, tertiary or academic) and the volume of procedures performed at the study center prior to the study may also introduce bias if not balanced. [11] Thus, adequate reporting of the treatment interventions, surgeon expertise and the study center are imperative.

Characteristics of high quality systematic reviews [Table - 1]

In contrast to unsystematic literature reviews, the systematic review attempts to collect, appraise, integrate and report results through a systematic process which limits bias. [12] There are eight steps which are crucial to this process: 1) formulating a review question, 2) defining inclusion and exclusion criteria, 3) locating studies, 4) selecting studies, 5) assessing study quality, 6) extracting data, 7) analyzing and presenting results and 8) interpreting results. [13] One of the most beneficial aspects of a systematic review is the meta-analysis. A meta-analysis is a quantitative analysis of results across many studies to arrive at the single best estimate of a treatment effect. The implications of the ability to provide a single best treatment effect while limiting bias is that the systematic review is a valuable tool for clinicians when making treatment decisions. Recognizing the potential of systematic reviews to positively impact clinical practice, the Quality of Reporting of Meta-Analyses (QUOROM) statement was developed to provide authors and readers with a general guide to methodological considerations in reporting and appraising a systematic review. [14] In addition to familiarizing themselves with the QUOROM Statement, clinicians should ensure that systematic reviews are up-to-date, relevant (i.e. to the issue being considered), comprehensive (i.e. all relevant studies are included) and methodologically sound before implementing results into their clinical decision-making process. [13],[15]

   Why are we Concerned? The Surgical Literature Top

Overall quality of the surgical literature

Concerns regarding quality of the orthopaedic literature stem from a widespread notion that the overall quality of the surgical literature is in need of improvement. Limitations in surgical research arises primarily from two pervasive issues: 1) A reliance on low levels of evidence to advance surgical knowledge and 2) Poor reporting quality among the high level surgical evidence that is available.

The first issue was brought to the forefront of debate in a controversial paper published in the Lancet, criticizing the surgical literature for its reliance on case studies and case series-which constitute the lowest level of evidence available [Table - 1]-as the primary source of evidence to advance surgical knowledge. [16] Since then, researchers have largely supported the use of higher levels of evidence, noting that although case studies and case series do have an invaluable role in surgical research, these sources of evidence have limitations which simply prevent them from answering certain types of questions. [17] For instance, Petrisor and Bhandari have outlined that these study designs are limited by their often retrospective nature, lack of a control group, incomplete data collection and generalizability. [9] Therefore, as mentioned earlier, questions of effectiveness and efficacy are best answered using high quality RCTs and systematic reviews.

Fortunately, since criticism of the quality of surgical literature first surfaced, the number of published RCTs has grown rapidly. [18] However, the publication of these high level trials is not sufficient. Recent evidence has criticized the reporting quality of available surgical RCTs, including those evaluating cardiothoracic, general, urological, gastrointestinal and vascular interventions. [19],[20],[21] Jacquier and colleagues have made the pressing assertion that the quality of surgical RCTs in "in need of immediate improvement" after demonstrating that the reporting of methodological factors, treatment descriptions, care providers and treatment centers was poor across trials of various surgical subspecialities-30% of which were orthopaedic. [11]

The aforementioned evidence provides us with a brief overview which indicates that orthopaedic research is experiencing similar challenges and may need improvement. However, this evidence is marginal and cannot necessarily be extrapolated to the orthopaedic literature with certainty. Therefore, the remainder of this report will review the available evidence to determine whether the orthopaedic literature is experiencing similar quality 'pitfalls' and, if so, discuss the nature of these 'pitfalls'.

   Quality of the Orthopaedic Literature Top

Randomized controlled trials

There is evidence that suggests the quality of reporting in orthopaedic RCTs needs of improvement. [18],[19],[22],[23],[24],[25] Several different quality checklists available to evaluate reporting-including the Detsky Quality Index, the CLEAR NPT and the CONSORT statement-have all been applied to the orthopaedic literature to demonstrate this poor state of affairs. One of the earlier studies to do so, carried out by Bhandari et al. , applied the Detsky scale to 72 randomized trials published in the Journal of Bone and Joint Surgery (American Volume) between 1988 to 2000. Briefly, the Detsky scale is comprised of fourteen items that assess reporting quality among five categories: 1) randomization, 2) outcome measures, 3) eligibility criteria and reasons for patient exclusion, 4) interventions and 5) statistical issues. A score of greater than 75% on this scale was deemed high quality. The study found that on average, only 68% of the Detsky items were reported, with 43 of the trials scoring below 75%. [18] In addition to these findings, Bhandari and colleagues also highlighted some other interesting observations.

First and foremost, drug trials scored significantly better than surgical trials (73% vs. 64%, respectively) and an overwhelmingly greater number of drug trials were also reported as double-blinded compared to surgical trials (46% vs. 10%, respectively). [18] This is particularly alarming, as the risk of introducing bias as a result of non-blinding is higher in non-pharmacological trials compared to pharmacological trials. [23] This problem of inadequate blinding among nonpharmacolgical trials in the orthopaedic literature has been corroborated by a study that evaluated both non-pharmacological and pharmacological randomized trials on knee and hip osteoarthritis. The findings indicated that 65% of surgical trials that did not employ blinding could have feasibly done so. [23]

Secondly, Bhandari et al. , also demonstrated that cited statistical support and cited funding were correlated with higher quality reporting. Although funding may improve reporting practices, the source of funding always deserves a critical assessment, as industry-funded studies are significantly more likely to be "positive" studies with pro-industry outcomes. This holds true even when studies are controlled for their quality. [26] Thus, it is imperative that trial authors 1) adequately report all methodological parameters utilized in their trials and 2) disclose the sources of funding, so that readers can sufficiently evaluate the validity of their trial results.

As the true clinical benefits of randomized trials can only be realized if manuscripts are thoroughly and accurately reported, several checklists have been developed to guide authors in writing their reports and in helping readers evaluate them. First published in the Journal of the American Medical Association in 1996, the Consolidated Standards of Reporting Trials (CONSORT) statement is considered as one of the more useful and popular of such guides. It is comprised of a comprehensive 22-item checklist and a flow diagram which collectively focus on the reporting of trial design, analysis, interpretation and participant progress. [27] Several investigations applying the CONSORT statement to orthopaedic randomized trials have reinforced the poor quality of reporting. [21],[24],[25] For instance, Bhandari et al. , [24] carried out a comprehensive investigation in which they evaluated the extent to which reports of RCTs in orthopaedic trauma met the CONSORT criteria. Of the 196 reports (published across 32 journals) evaluated, it was found that the reports adhered to an average of only 32% ± 29% of the CONSORT criteria. Surprisingly, over 70% of the studies failed to meet even half of the criteria outlined in the CONSORT statement. [24]

To assist clinicians in evaluating scientific articles, several orthopaedic journals have implemented the Levels of Evidence rating system to their publication process, which pre-evaluates study reports and provides a quality rating for readers. Not only does this rating system enable readers to approach the "pre-appraised" trials with confidence, but it also allows journals to monitor the quality of the orthopaedic literature. [28] In this rating system, clinical studies are rated on a scale of one to five (level one being the highest quality) based on quality and design. RCTs are designated a rating of level I or II, depending on the extent to which methodological safeguards are used to protect the study from bias. A recent evaluation of the orthopaedic literature found that only 11.3% of published papers are considered level I evidence according to this system. [29] Even more alarming than the scarcity of level I evidence is that the reporting quality among level I studies has been called into question. In particular, Poolman and colleagues have found that level I studies published in the Journal of Bone and Joint Surgery (American Volume) have severe limitations in the reporting of methodological safeguards. They also found that the reporting quality in level I studies is not necessarily superior to level II studies. [28]

   Systematic Reviews Top

The number of published systematic reviews in both the medical literature and the orthopaedic literature has grown rapidly over the past decade. [30],[31] Researchers and clinicians are increasingly coming to recognize the value of systematic reviews. For instance, a study of the orthopaedic literature by Bhandari and colleagues demonstrated that systematic reviews are more likely to be cited as evidence as compared to their unsystematic counterparts. [32] Unfortunately, shortcomings continue to plague systematic reviews published in the orthopaedic literature and compromise the validity of their conclusions. An evaluation of the reporting quality of orthopaedic systematic reviews revealed that only 15% could be classified as methodologically rigorous. [31] Systematic reviews that are identified as methodologically poor are also more likely to report positive outcomes when compared to high quality systematic reviews. [31] Taken together, this suggests that nearly 85% of orthopaedic systematic reviews may assert biased conclusions.

One of the major shortcomings apparent in orthopaedic systematic reviews is the quality of the reviewed articles. The external validity of even a high quality systematic review is ultimately contingent on the quality of the included articles. Therefore, addressing the shortcomings of RCTs, as discussed in the previous section, is the first step to ensuring that systematic reviews produce the most valid and unbiased conclusions possible. Currently, however, the orthopaedic literature is experiencing a trend in which not only methodologically compromised RCTs but also non-randomized trials are being included in systematic reviews and meta-analyses. For instance, between 1996 and 2001, less than 15% of systematic reviews that analyzed only RCTs were published outside the Cochrane library. The majority of orthopaedic systematic reviews published in peer-reviewed journals included non-randomized trials. [33] The most often cited reason for including non-randomized trials was lack of available RCTs in the orthopaedic literature. Perhaps most disconcerting is the fact that most systematic reviews and especially systematic reviews evaluating non-randomized trials, fail to assess the quality of included studies, thereby further compromising the validity of reported results. [33]

The orthopaedic literature has recently been scrutinized for the presence of publication bias. Publication bias occurs when positive trials are published more frequently than trials that demonstrate neutral results (i.e. those which did not find a statistically significant effect). This form of bias, also referred to as the 'positive outcome bias', is particularly elusive because it can corrupt even the most methodologically rigorous systematic reviews and bias results towards the direction of the positive effect. [34] In a follow-up of 318 abstracts presented at the 1999 American Academy of Orthopaedic Surgeons annual meeting, Harris and colleagues demonstrated that trials with positive outcomes were far more likely to be published than trials with neutral outcomes. [35] More recently, a landmark trial by Hasenboehler and colleagues reviewed 16,397 original surgical studies from 12 journals between 2000 and 2006, demonstrating that less than 10% were neutral studies whereas over 70% were positive studies. [36] Subgroup analysis revealed identical trends of publication bias for orthopaedic journals. As systematic reviews are increasingly being referred to in clinical decision-making, this often unrecognized introduction of bias has been referred to as a "severe challenge to patient safety". [36]

   Future Directions for Improvement Top

Many of the issues discussed in this report can be addressed by conducting 'higher level' studies-in particular RCTs-and ensuring that trial manuscripts are reported thoroughly.

Need for higher level orthopaedic evidence

Only 11.3% of studies published in the orthopaedic literature are considered level I. It has been further suggested that RCTs in particular constitute approximately 3% of the orthopaedic literature. [18],[28],[29] To assume that orthopaedic researchers have not considered RCTs because of a lack of awareness is, perhaps, an overstatement. The scarcity of RCTs may be largely attributable to several unique challenges which make surgical RCTs difficult to conduct. We would like to address some of these challenges.

The first challenge has to do with inter-surgeon variability in skill and the associated ethical and practical concerns of randomization that this poses. From an ethical standpoint orthopaedic surgeons may not feel comfortable performing a surgical procedure which they are not particularly adept at. Some may feel it is unethical to provide a patient with a randomly assigned procedure which they do not feel is the 'superior' approach. From a practical standpoint, different surgeons will have different learning curves for a new procedure, which may serve as a source of bias. Devereux and colleagues have suggested the use of "expertise based" RCTs, in which patients are not randomized to a procedure per se , but rather to a surgeon with expertise in a particular procedure. [37] It has been suggested that, among other things, expertise based RCTs will increase surgeon recruitment for trials, reduce bias associated with learning curves, prevent procedural crossovers and present a more ethical alternative to the traditional RCT. [37]

The second challenge involves the difficulty of blinding in a surgical trial. It has been suggested that lack of blinding in surgical trials may be attributable, at least in part, to a lack of awareness of available blinding methods. [38] Researchers and readers must become thoroughly acquainted with various blinding methods. It should be noted that blinding methods may be categorized into three categories based on the group of individuals that may potentially be blinded: participants, health care providers and outcome assessors. There are extensive methods of blinding which further vary depending on the type of data being collected (i.e. participant-reported, physician-driven, paraclinical data or clinical event). [38] Both researchers and readers are encouraged to become acquainted with the methods that Boutron and colleagues [38] outline as a starting point for further investigation.

Another challenge involves the difficulty obtaining funding for surgical trials. Surgical researchers typically receive fewer grants, which are often of lesser value, than those received by their non-surgical counterparts. [39] Surgeons need to work together and participate in clinical research, thus being better able to persuade funding sources as an organized lobby. [17] Often surgeons will simply not conduct trials with new implants because of lenient regulatory practices and the ease with which implants can be introduced into clinical practice (as compared to pharmaceuticals). [39] We hope that surgeons will recognize the value of RCTs in disseminating important innovations to their colleagues and, more importantly, to improving patient care and safety.

Need for improved reporting of orthopaedic evidence

There is an immense need for improved reporting of orthopaedic RCTs. Chan and Bhandari have reinforced this notion through their recent assessment of 87 orthopaedic RCTs across eight medical journals. [40] They found, as expected, reporting to be highly variable, with poor reporting for several methodological issues. However, after contacting the authors of these studies they concluded that "not reported" does not necessarily mean "not conducted". The implication is that there is hope for a substantial improvement in the apparent quality of the orthopaedic literature through a mere improvement in reporting practices. Several checklists exist for various study designs that can improve reporting if adopted, endorsed and enforced by journals [Table - 2]. We suggest that the submission of these quality checklists alongside manuscripts should become a strict part of the submission process. This would standardize reporting and enable clinicians to confidently interpret and incorporate study results into clinical practice.

   References Top

1.Bhandari M, Giannoudis P. Evidence-based medicine: What it is and what it is not. Injury 2006;37:302-6.  Back to cited text no. 1    
2.Guyatt GH, Haynes RB, Jaeschke R, Cook D, Greenhalgh T, Meade M, et al. Introduction: The philosophy of evidence-based medicine. In : Guyatt GH, Rennie D, editors. Users' guides to the medical literature: A manual for evidence-based clinical practice/The Evidence-Based Medicine Working Group. Chicago: AMA Press; 2002. p. 3-12.  Back to cited text no. 2    
3.Kurtz S, Ong K, Lau E, Mowat F, Halpern M. Projections of primary and revision hip and knee arthroplasty in the United States from 2005 to 2030. J Bone Joint Surg Am 2007;89:780-5.  Back to cited text no. 3  [PUBMED]  [FULLTEXT]
4.Haentjens P, Autier P, Barette M, Boonen S; for the Belgian Hip Fracture Study Group. The economic cost of hip fractures among elderly women: A one year, prospective, observational cohort study with matched-pair analysis. J Bone Joint Surg Am 2001;83:493-500.  Back to cited text no. 4    
5.Johnell O, Kanis JA. An estimate of the worldwide prevalence, mortality and disability associated with hip fracture. Osteoporos Int 2004;15:897-902.  Back to cited text no. 5  [PUBMED]  [FULLTEXT]
6.Gullberg B, Johnell O, Kanis JA. World-wide projections for hip fracture. Osteoporos Int 1997;7:407-13.  Back to cited text no. 6  [PUBMED]  [FULLTEXT]
7.Dixon T, Shaw M, Ebrahim S, Dieppe P. Trends in hip and knee joint replacement: Socioeconomic inequalities and projections of need. Ann Rheum Dis 2004;63:825-30.  Back to cited text no. 7  [PUBMED]  [FULLTEXT]
8.Papadimitropoulos EA, Coyte P, Josse RG, Greenwood CE. Current and projected rates of hip fracture in Canada. CMAJ 1997;157:1357-63.  Back to cited text no. 8    
9.Petrisor BA, Bhandari M. The hierarchy of evidence: Levels and grades of recommendation. Indian J Orthop 2007;41:11-5.  Back to cited text no. 9    
10.Mills E, Wu P, Gagnier J, Heels-Ansdell D, Montori V. An analysis of general medical and specialist journals that endorse CONSORT found that reporting was not enforced consistently. J Clin Epidemiol 2005;58:662-7.  Back to cited text no. 10    
11.Jacquier I, Boutron I, Moher D, Roy C, Ravaud P. The reporting of randomized clinical trials using a surgical intervention is in need of immediate improvement: A systematic review. Ann Surg 2006;244:677-83.  Back to cited text no. 11  [PUBMED]  [FULLTEXT]
12.Oxman A, Guyatt GH, Cook D, Montori V. Summarizing the evidence. In : Guyatt GH, Rennie D, editors. Users' guides to the medical literature: A manual for evidence-based clinical practice/The Evidence-Based Medicine Working Group. Chicago: AMA Press; 2002. p. 155-73.  Back to cited text no. 12    
13.Egger M, Smith GD. Principles and procedures for systematic reviews. In : Egger M, Smith GD, Altman DG, editors. Systematic reviews in health Care-meta-analysis in context. London: BMJ Publishing; 2002. p. 23-42.  Back to cited text no. 13    
14.Moher D, Cook DJ, Eastwood S, Ingram O, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomized controlled trials: The QUOROM statement. Lancet 1999;354:1896-900.  Back to cited text no. 14    
15.Moher D, Tsertsvadze A. Systematic reviews: When is an update an update? Lancet 2006;18:881-2.  Back to cited text no. 15    
16.Horton R. Surgical research or comic opera: Questions, but few answers. Lancet 1996;347:984-5.  Back to cited text no. 16  [PUBMED]  
17.Carr AJ. Evidence-based orthopaedic surgery: What type of research will best improve clinical practice? J Bone Joint Surg Br 2005;87:1593-4.  Back to cited text no. 17  [PUBMED]  [FULLTEXT]
18.Bhandari M, Richards RR, Sprague S, Schemitsch EH. The quality of reporting randomized trials in the Journal of Bone and Joint Surgery from 1988 through 2000. J Bone Joint Surg Am 2002;84:388-96.  Back to cited text no. 18  [PUBMED]  [FULLTEXT]
19.Tiruvoipati R, Balasubramanian SP, Atturu G, Peek GJ, Elbourne D. Improving the quality of reporting randomized controlled trials in cardiothoracic surgery: The way forward. J Thorac Cardiovasc Surg 2006;132:233-40.  Back to cited text no. 19  [PUBMED]  [FULLTEXT]
20.Balasubramanian SP, Wiener M, Alshameeri Z, Tiruvoipati R, Elbourne D, Reed MW. Standards of reporting of randomized controlled trials in general surgery: Can we do better? Ann Surg 2006;244:663-7.  Back to cited text no. 20  [PUBMED]  [FULLTEXT]
21.Agha R, Cooper D, Muir G. The reporting quality of randomised controlled trials in surgery: A systematic review. Int J Surg 2007;5:413-22.  Back to cited text no. 21  [PUBMED]  
22.Boutron I, Tubach F, Giraudeau B, Ravaud P. Methodological differences in clinical trials evaluating nonpharmacological treatments of hip and knee osteoarthritis. JAMA 2003;290:1062-70.  Back to cited text no. 22  [PUBMED]  [FULLTEXT]
23.Boutron I, Tubach F, Giraudeau B, Ravaud P. Blinding was judged more difficult to achieve and maintain in nonpharmacologic than pharmacologic trials. J Clin Epidemiol 2004;57:543-50.  Back to cited text no. 23  [PUBMED]  [FULLTEXT]
24.Bhandari M, Guyatt GH, Lochner H, Sprague S, Tornetta P 3rd. Application of the consolidated standards of reporting trials (CONSORT) in the fracture care literature. J Bone Joint Surg Am 2002;84:485-9.  Back to cited text no. 24  [PUBMED]  [FULLTEXT]
25.Cowan J, Lozano-Calderon S, Ring D. Quality of prospective controlled randomized trials: Analysis of trials of treatment for lateral epicondylitis as an example. J Bone Joint Surg Am 2007;89:1693-9.  Back to cited text no. 25    
26.Bhandari M, Busse JW, Jackowski D, Montori VM, Schunemann H, Sprague S, et al. Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials. CMAJ 2004;170:477-80.  Back to cited text no. 26    
27.The CONSORT group. CONSORT 2001 checklist and CONSORT 2001 flow diagram. [cited on 2008 Feb 27]. Available from: = 1031.  Back to cited text no. 27    
28.Poolman R, Struijs PA, Krips R, Sierevelt IN, Lutz KH, Bhandari M. Does a "Level I Evidence" rating imply high quality of reporting in orthopaedic randomised controlled trials? BMC Med Res Methodol 2006;6:44-51.  Back to cited text no. 28    
29.Obremskey WT, Pappas N, Attallah-Wasif E, Tornetta P 3rd, Bhandari M. Level of evidence in orthopaedic journals. J Bone Joint Surg Am 2005;87:2632-8.  Back to cited text no. 29  [PUBMED]  [FULLTEXT]
30.Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemiology and reporting characteristics of systematic reviews. PLoS Med 2007;4:447-55.  Back to cited text no. 30    
31.Bhandari M, Morrow F, Kulkarni AV, Tornetta P 3rd. Meta-analyses in orthopaedic surgery: A systematic review of their methodologies. J Bone Joint Surg Am 2001;83:15-24.  Back to cited text no. 31  [PUBMED]  [FULLTEXT]
32.Bhandari M, Montori VM, Devereux PJ, Wilczynski NL, Morgan D, Haynes RB. Doubling the impact: Publication of systematic review articles in orthopaedic journals. J Bone Joint Surg Am 2004;86:1012-6.  Back to cited text no. 32    
33.Audige L, Bhandari M, Griffin D, Middleton P, Reeves BC. Systematic reviews of nonrandomized clinical studies in the orthopaedic literature. Clin Orthop Relat Res 2004;427:249-57.  Back to cited text no. 33    
34.Montori V, Guyatt GH. Summarizing the evidence: Publication bias. In : Guyatt GH, Rennie D, editors. Users' guides to the medical literature: A manual for evidence-based clinical practice/The Evidence-Based Medicine Working Group. Chicago: AMA Press; 2002. p. 529-45.  Back to cited text no. 34    
35.Harris IA, Mourad M, Kadir A, Solomon MJ, Young JM. Publication bias in abstracts presented to the annual meeting of the American Academy of Orthopaedic Surgeons. J Orthop Surg 2007;15:62-6.  Back to cited text no. 35    
36.Hasenboehler EA, Choudhry IK, Newman JT, Smith WR, Ziran BH, Stahel PF. Bias towards publishing positive results in orthopedic and general surgery: A patient safety issue? Patient Safe Surg 2007;1:4-9.  Back to cited text no. 36    
37.Devereux PJ, Bhandari M, Clarke M, Montori VM, Cook DJ, Yusuf S, et al. Need for expertise based randomised controlled trials. BMJ 2005;330:88-92.  Back to cited text no. 37    
38.Boutron I, Guittet L, Estellat C, Moher D, Hrobjartsson A, Ravaud P. Reporting methods of blinding in randomized trials assessing nonpharmacological treatments. PLoS Med 2007;4:370-80.  Back to cited text no. 38    
39.Boutron I, Ravaud P, Nizard R. The design and assessment of prospective randomized, controlled trials in orthopaedic surgery. J Bone Joint Surg Br 2007;89:858-63.  Back to cited text no. 39  [PUBMED]  [FULLTEXT]
40.Chan S, Bhandari M. The quality of reporting of orthopaedic randomized trials with use of a checklist for nonpharmacological therapies. J Bone Joint Surg Am 2007;89:1970-8.  Back to cited text no. 40  [PUBMED]  [FULLTEXT]

Correspondence Address:
Harman Chaudhry
Department of Surgery, Division of Orthopaedic Surgery, McMaster University, Hamilton General Hospital, 293 Wellington Street North, Suite 110, Hamilton, Ontario, L8L 8E7
Login to access the Email id

Source of Support: None, Conflict of Interest: None

DOI: 10.4103/0019-5413.40250

Rights and Permissions


  [Table - 1], [Table - 2]

This article has been cited by
1 Osteochondral Autograft Transfer Versus Microfracture in the Knee: A Meta-analysis of Prospective Comparative Studies at Midterm
Ayoosh Pareek,Patrick J. Reardon,Jeffrey A. Macalena,Bruce A. Levy,Michael J. Stuart,Riley J. Williams,Aaron J. Krych
Arthroscopy: The Journal of Arthroscopic & Related Surgery. 2016; 32(10): 2118
[Pubmed] | [DOI]
2 Editorial Commentary: Cost-Effectiveness of Orthopaedic Procedures: “A Nickel Ainæt Worth a Dime Anymore” – Yogi Berra
Laurence D. Higgins
Arthroscopy: The Journal of Arthroscopic & Related Surgery. 2016; 32(9): 1781
[Pubmed] | [DOI]
3 Sources and quality of literature addressing femoroacetabular impingement: a scoping review 2011–2015
Moin Khan,Kayode O Oduwole,Parul Razdan,Mark Phillips,Seper Ekhtiari,Nolan S Horner,Kristian Samuelsson,Olufemi R Ayeni
Current Reviews in Musculoskeletal Medicine. 2016; 9(4): 396
[Pubmed] | [DOI]
4 An international survey to identify the intrinsic and extrinsic factors of research studies most likely to change orthopaedic practice
P. Thornley,D. de SA,N. Evaniew,F. Farrokhyar,M. Bhandari,M. Ghert
Bone and Joint Research. 2016; 5(4): 130
[Pubmed] | [DOI]
5 Cochrane in CORR ®: Double-bundle Versus Single-bundle Reconstruction for Anterior Cruciate Ligament Rupture in Adults (Review)
Raman Mundi,Mohit Bhandari
Clinical Orthopaedics and Related Research®. 2016; 474(5): 1099
[Pubmed] | [DOI]
6 Articular cartilage solutions for the knee: present challenges and future direction
Aaron J Krych,Alberto Gobbi,Christian Lattermann,Norimasa Nakamura
Journal of ISAKOS: Joint Disorders & Orthopaedic Sports Medicine. 2016; 1(2): 93
[Pubmed] | [DOI]
7 Understanding involvement in surgical orthopaedic randomized controlled trials: A qualitative study of patient and health professional views and experiences
Jeremy Horwood,Emma Johnson,Rachael Gooberman-Hill
International Journal of Orthopaedic and Trauma Nursing. 2016; 20: 3
[Pubmed] | [DOI]
8 Evaluation of Quality of Lower Limb Arthroplasty Observational Studies Using the AQUILA Checklist
James B. Cowan,Ryan A. Mlynarek,Rob G.H.H. Nelissen,Bart G.C.W. Pijls,Joel J. Gagnier
The Journal of Arthroplasty. 2015;
[Pubmed] | [DOI]
9 Methodological quality of systematic reviews addressing femoroacetabular impingement
Marcin Kowalczuk,John Adamich,Nicole Simunovic,Forough Farrokhyar,Olufemi R. Ayeni
Knee Surgery, Sports Traumatology, Arthroscopy. 2015; 23(9): 2583
[Pubmed] | [DOI]
10 Open access of evidence-based publications: the case of the orthopedic and musculoskeletal literature
Kaissar Yammine
Journal of Evidence-Based Medicine. 2015; 8(4): 181
[Pubmed] | [DOI]
11 Network Meta-analysis: Users’ Guide for Surgeons: Part I – Credibility
Clary J. Foote,Harman Chaudhry,Mohit Bhandari,Lehana Thabane,Toshi A. Furukawa,Brad Petrisor,Gordon Guyatt
Clinical Orthopaedics and Related Research®. 2015; 473(7): 2166
[Pubmed] | [DOI]
12 Sources and quality of literature addressing femoroacetabular impingement
Ayeni, O.R. and Chan, K. and Al-Asiri, J. and Chien, T. and Sprague, S. and Liew, S. and Bhandari, M.
Knee Surgery, Sports Traumatology, Arthroscopy. 2013; 21(2): 415-419
13 Critical analysis of scientific publications of the Revista Brasileira de Ortopedia from 2006 to 2010
Eduardo Angeli Malavolta,Riccardo Gomes Gobbi,José Antonio Mancuso Filho,Marco Kawamura Demange
Revista Brasileira de Ortopedia (English Edition). 2013; 48(3): 211
[Pubmed] | [DOI]
14 Análise crítica das publicações científicas da Revista Brasileira de Ortopedia no período de 2006 a 2010
Eduardo Angeli Malavolta,Riccardo Gomes Gobbi,José Antonio Mancuso Filho,Marco Kawamura Demange
Revista Brasileira de Ortopedia. 2013; 48(3): 211
[Pubmed] | [DOI]
15 Sources and quality of literature addressing femoroacetabular impingement
Olufemi R. Ayeni,Kevin Chan,Jamal Al-Asiri,Teresa Chien,Sheila Sprague,Susan Liew,Mohit Bhandari
Knee Surgery, Sports Traumatology, Arthroscopy. 2013; 21(2): 415
[Pubmed] | [DOI]
16 Emerging designs in orthopaedics: Expertise-based randomized controlled trials
Scholtes, V.A. and Nijman, T.H. and Van Beers, L. and Devereaux, P.J. and Poolman, R.W.
Journal of Bone and Joint Surgery - Series A. 2012; 94(SUPPL. 1): 24-28
17 Emerging Designs in Orthopaedics: Expertise-Based Randomized Controlled Trials
Vanessa A Scholtes,Thomas H Nijman,Loes van Beers,P J Devereaux,Rudolf W Poolman
The Journal of Bone and Joint Surgery-American Volume. 2012; 94(Suppl 1): 24
[Pubmed] | [DOI]
Eduardo Angeli Malavolta,Marco Kawamura Demange,Riccardo Gomes Gobbi,Marta Imamura,Felipe Fregni
Revista Brasileira de Ortopedia (English Edition). 2011; 46(4): 452
[Pubmed] | [DOI]
19 Randomized controlled clinical trials in orthopedics: Difficulties and limitations [Ensaios clínicos controlados e randomizados na ortopedia: Dificuldades e limitações]
Malavolta, E.A. and Demange, M.K. and Gobbi, R.G. and Imamura, M. and Fregni, F.
Revista Brasileira de Ortopedia. 2011; 46(4): 452-459
20 The Cochrane Collaboration: A leading role in producing reliable evidence to inform healthcare decisions in musculoskeletal trauma and disorders
Handoll, H., Gillespie, W., Gillespie, L., Madhok, R.
Indian Journal of Orthopaedics. 2008; 42(3): 247-251


    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
    Email Alert *
    Add to My List *
* Registration required (free)  

    Evidence-based M...
    Characteristics ...
    Why are we Conce...
    Quality of the O...
    Systematic Reviews
    Future Direction...
    Article Tables

 Article Access Statistics
    PDF Downloaded280    
    Comments [Add]    
    Cited by others 20    

Recommend this journal