Skip to main content

Epi Lessons - Part 3 - OVERVIEW ARTICLES

As part of our Journal Club summaries our JC Chairs (Drs. Lisa Calder and Ian Stiell @EMO_Daddy) have been tasked with explaining Epidemiological concepts so that everyone in our department can analyze the literature and appraise articles on their own. For this Blog post we have all the "Epi Lesson" as they relate to "Overview Articles". More to follow in the coming weeks.

Clinical vs Statistical Heterogeneity

A meta-analysis may attempt to address a compelling clinical dilemma. But one of the key questions to ask when appraising meta-analyses is whether the pooling of the included studies is appropriate. Clinical heterogeneity reflects clinical differences between study populations, the intervention, co-interventions and/or outcomes when pooling studies in meta-analysis. This is distinct from statistical heterogeneity which can be determined by visually assessing the forest plot, measuring the I2 statistic or the Cochran’s Q. Always ask yourself if the meta-analysis is combining apples with apples. 

Cochrane Collaboration Tool for Assessing Risk of Bias in Intervention Trials
                                                                  By: Dr. Ian Stiell                         September 2014

The use of scales for assessing quality or risk of bias in intervention trials is explicitly discouraged in Cochrane reviews, including the commonly-used scale was developed by Jadad and colleagues for randomized trials in pain research (Jadad 1996). For assessing bias, the Cochrane Collaboration recommends a two-part tool that addresses seven specific domains (sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective outcome reporting and ‘other issues’).  Each domain in the tool includes one or more specific entries in a ‘Risk of bias’ table. 

Goals of Systematic Literature Reviews               By: Dr. Lisa Calder               April 2014

We often consider systematic literature reviews (SLR) and meta-analyses as tools to help us better understand the effectiveness of a given therapy when there are multiple conflicting studies in the literature. Another reasonable goal, however, is to advance the science in a given domain. On occasion, there lack rigorous study definitions to allow for effective research to answer a given clinical question or perhaps studies have been published looking repetitively at the same issue without any advancement on the clinically relevant question. Researchers can use SLRs to highlight gaps in the current literature and the need for a specific study design. This can focus researchers towards answering the question rather than multiple haphazard approaches.

Heterogeneity of Studies in Overviews                    By: Dr Ian Stiell September 2012  

Heterogeneity refers to differences between patients or differences in the results of different studies and must be carefully evaluated in a meta-analysis to determine if results can be statistically pooled. The classical measure of heterogeneity is Cochran’s Q, which is calculated as the weighted sum of squared differences between individual study effects and the pooled effect across studies, with the weights being those used in the pooling method. The I² statistic describes the percentage of variation across studies that is due to heterogeneity rather than chance.  Always ask yourself if the meta-analysis is combining apples with apples. 

Network Meta-analysis
          By: Dr. Lisa Calder                                     December 2014 

Network meta-analyses should not be viewed as the same as other meta-analyses. This specialized methodology is designed to answer complex management questions when there are multiple options possible. This is particularly helpful when a head-to-head RCT is not practical to conduct. The key requirement is that the studies being compared have at least one common or equivalent therapy among 

PICOS Format                                                           By Dr. Lisa Calder            March 2013
When reviewing systematic literature reviews, pay attention to how the research question is phrased. High quality studies will follow the PRISMA guidelines’ suggestion to use the PICOS (participants, interventions, comparisons, outcomes, study design) format. A precise research question will guide study selection and should also dictate how table 1 summarizes the included studies.  Having these details will also assist the reader in evaluating the included studies for generalizability and clinical heterogeneity. 

Pooled Analysis vs. Meta-Analysis                         By Dr. Ian Stiell                    May 2012
In a meta-analysis, researchers assess heterogeneity across studies, examine subgroups of studies to determine if selected subsets of the research data provide similar or different results, and calculate summary relative risk estimates. A pooled analysis is similar to a traditional meta-analysis, except that data are combined (or pooled) from multiple studies and are analyzed as a single dataset. If the data and methods are consider homogeneous across studies and the data are available, then a pooled analysis is a very legitimate approach. 

PRISMA Statement for Reporting Systematic Reviews:   
                                                                            By Dr. Ian Stiell                    January 2012  
The PRISMA statement (revised 2009 to replace QUORUM was developed by an international group to establish preferred reporting guidelines for systematic reviews and meta-analyses. David Moher of the Ottawa Hospital Research Institute is the lead author on the paper that includes the PRISMA 27-item checklist and four-phase flow diagram. 

Publication Bias in Systematic Reviews             By Dr. Ian Stiell                    March 2012 
Even when individual studies included in best evidence summaries have a low risk of bias, publication bias can result in substantial overestimates of effect. Authors should suspect publication bias when available evidence comes from a number of small studies, most of which have been commercially funded. A number of approaches based on examination of the pattern of data are available to help assess publication bias. The most popular of these is the funnel plot; all, however, have substantial limitations. The likelihood of publication bias is less for studies that are not evaluating new drugs or devices.  

QUADAS-2 Tool for Evaluation of Systematic Reviews of Diagnostic Accuracy Studies                                                            By: Dr. Ian Stiell                        September 2014

The QUADAS-2 Tool is recommended for evaluating the quality of systematic reviews of primary diagnostic accuracy studies [as opposed to intervention studies]. QUADAS-2 consists of four key domains: patient selection, index test, reference standard, as well as flow and timing. Each is assessed in terms of risk of bias and the first three in terms of concerns regarding applicability. Signalling questions are included to assist in judgements about risk of bias. QUADAS-2 is applied in four phases. 

QUOROM statement                                        By: Dr. Lisa Calder             December 2012
When critically appraising systematic literature reviews and meta-analyses, the reader will find the QUOROM statement to be a helpful tool. This guide helps you assess adequacy of search strategy, article selection and quality assessment. Furthermore, it provides the key elements which should be reported to allow you to assess the overall validity of the results. This reporting template also assists authors of SLR and MA to both design and write-up their studies in a rigorous way. 

Reporting Standards for Systematic Reviews              By Dr. Ian Stiell     February 2013
The PRISMA statement ( was developed to establish preferred reporting guidelines for systematic reviews and meta-analyses and includes a 27-item checklist and four-phase flow diagram. Observational studies are considered a lower level of evidence for interventions and have had two sets of guidelines developed: MOOSE (Meta-analysis Of Observational Studies in Epidemiology) and STROBE (STrengthening the Reporting of OBservational studies in Epidemiology).

Review of Grey Literature in Systematic Reviews 
                                                                         By: Dr. Ian Stiell                  September 2015

There are many definitions of grey literature, but it is usually understood to mean literature that is not formally published in sources such as books or journal articles. Conference abstracts and other grey literature have been shown to be sources of approximately 10% of the studies referenced in Cochrane reviews. A recently updated Cochrane methodology review showed that published trials showed an overall greater treatment effect than grey literature trials. Conference abstracts are a particularly important source of grey literature although such work has not undergone the extensive peer-reviewed required for most scientific manuscripts. There are numerous on-line grey literature databases.

Should Observational Studies be Included in a Systematic Review?    

                                                                                               By Dr. Ian Stiell         May 2012  Generally only RCTs should be included because the results of non-randomized, observational studies on interventions are subject to a number of biases and often over-estimate the effects. The problem we frequently see in emergency medicine is that there may be very few RCTs in particular content area, like acute pericarditis. Hence, while reviews that include non-randomized studies may be informative, readers must take the results with several grains of salt.  

Statistical Heterogeneity in Systematic Reviews               By: Dr. Christian Vaillancourt

There is no statistical substitution to evaluating “clinical” heterogeneity. Statistical heterogeneity should only be evaluated after a clinical decision is made that study populations, interventions, and outcomes were indeed comparable, and that it is appropriate to combine study results. Cochran’s Q test is a classical measure of heterogeneity. It can be underpowered to detect heterogeneity when the number of included studies is small, and overpowered when it is large. This is not the case when using the I2 instead. A fixed effect can be used to combine studies with minimal statistical heterogeneity, whereas a random effect model should be used otherwise.

Subgroup analyses in Meta-analysis 
                     By Dr. Lisa Calder               May 2013
Subgroup analyses can be an important source of hypothesis generation. The key element to determine is whether the subgroup analyses were planned a priori. These analyses have greater validity than post hoc analyses which result from searching for statistically significant trends in the data which could be present due to chance and which some have labeled “data dredging”.

Systematic review versus meta-analysis      By: Dr. Ian Stiell                  November 2012
To avoid the biases of an unsystematic review (i.e. review article), a systematic review incorporates explicit inclusion and exclusion criteria, a comprehensive search for the evidence, and a summary of the results according to explicit rules. When a systematic review pools data across studies to provide a quantitative estimate of the treatment effect, this is called a meta-analysis. When the data cannot be pooled, the systematic review will provide a narrative synthesis of the evidence. 

To Pool or Not to Pool
                  By: Dr. Lisa Calder                               October 2014
A meta-analysis may attempt to address a compelling clinical question. But one of the key questions to ask when appraising meta-analyses is whether the pooling of the included studies is appropriate. Clinical heterogeneity reflects clinical differences between study populations, the intervention, co-interventions and/or outcomes when pooling studies in meta-analysis. This is distinct from statistical heterogeneity. Always ask yourself if the meta-analysis is combining apples with apples. 

Verification Bias                                By: Dr. Lisa Calder                               October 2014

When critically appraising a study evaluating a diagnostic test, it is important to examine the study population closely. In emergency medicine, we see undifferentiated patients the majority of the time. Studies of specialist referral populations have less generalizability to the ED because inherently the diagnostic test will perform better in a population with a high pre-test probability of disease in the first place. Verification bias, or work-up bias, applies here where the likelihood of performing the gold standard is influenced by the results of the diagnostic test under study. In this case, performing MRI or U/S for rotator cuff disease is influenced by the results of the physical examination tests being studied.


Popular posts from this blog

Tips for Success in your Emergency Medicine Rotation

Our wonderful medical students are preparing to start their first clinical rotations. With this in mind here are some of the top tips for success in your EM rotation
1)Be On Time – show up to your shifts on time, better yet 5 minutes early.That first impression is immensely important.
2)Introduce yourself to the team - “Hi my name is John Doe, I am the medical student on shift today” introduce yourself to the attending, residents, nurses, etc.You will be called on a lot more to help when there is something interesting going on if they know your name.
3)Be goal-oriented – have a goal for each shift, whether it’s a procedure or a type of presentation to see.
4)Don’t just stand there, do something – whenever there is a trauma or code, come to the bedside.Get gowned up for traumas and pay attention.Help with things that are within your scope of practice: chest compressions, moving patient, cardioversion
5)Don’t just stand there, do nothing – there are times in medicine when the best thing to d…

The Canadian Syncope Risk Score

Guest post by Dr. Venkatesh Thiruganasambandamoorthy.
Syncope is an commonly encountered problem in Emergency Medicine, with the potential for significant morbidity and mortality to patients. Unfortunately, to date, there have not been any reliable decision tools or instruments to help us in our assessment of the syncope patient.  This week, Venkatesh Thiruganasambandamoorthy and colleagues at the Ottawa Hospital have published an new Canadian Syncope Risk Score to identify patients at risk of serious adverse events, published in CMAJ.1 Here, Dr. Venk helps provides some further insight and potential utilization of the risk score, he may also be heard discussing this rule on the March edition of Canadian EMRAP.

BackgroundSyncope constitutes 1% of Emergency Department (ED) visits, and approximately 10% of these patients will have serious underlying conditions causing syncope (arrhythmia, MI, serious structural heart disease, pulmonary embolism, subarachnoid or severe hemorrhage). Alarmin…

2014 Canadian Guidelines for AF Management: Part 1: Introduction and CCS “CHADS-65” Algorithm

by Ian Stiell MD @EMO_Daddy

In this and subsequent postings we will discuss the latest recommendations for ED management of atrial fibrillation (AF) as presented in the newly published 2014 Focused Update of the Canadian Cardiovascular Society Guidelines for the Management of Atrial Fibrillation. The Guidelines PDF can be downloaded from the CCS website at 2014 Canadian Cardiovascular Society Guidelines for the Management of Atrial Fibrillation
The 2014 Focused Update uses the GRADE system of evidence evaluation as was the case in the comprehensive 2010 AF Guidelines and the 2012 Update. The CCS AF Guidelines Panel is comprised of Canadian cardiologists plus physicians from internal medicine, family medicine, neurology, and emergency medicine. This 2014 Update provides evidence review and recommendations for 8 aspects of AF care, including ED Management (written by myself and Dr. Laurent Macle of the Montreal Heart Institute). The 2014 Update focuses on advances in oral anticoagulant (O…