Science without sense…double nonsense

Píldoras sobre medicina basada en pruebas

Gathering the gold nuggets


I was thinking about today’s post and I cannot help remembering the gold-seekers of the Alaskan gold rush of the late nineteenth century. They went traveling to Yukon, looking for a good creek like the Bonanza and collecting tons of mud. But that mud was not the last step of the quest. Among the sediments they had to extract the longed gold nuggets, for which they carefully filtered the sediments to keep only the gold, when there was any.

When we look for the best scientific evidence to solve our clinical questions we do something similar. Normally we chose one of the Internet search engines (like Pubmed, our Bonanza Creek) and we usually get a long list of results (our great deal of mud) that, finally, we will have to filter to extract the gold nuggets, if there are any among the search results.

We have already seen in previous posts how to do a simple search (the least specific and which will provide us with more mud) and how to refine the searches by using the MeSH terms or the advanced search form, with which we try to get less mud and more nuggets.

However, the usual situation is that, once we have the list of results, we have to filter it to keep only what interests us most. Well, for that there is a very popular tool within Pubmed that is, oh surprise, the use of filters.

Let’s see an example. Suppose we want to seek information about the relationship between asthma and obesity in childhood. The ideal would be to build a structured clinical question to perform a specific search, but to show more clearly how filters work we will do a simple “bad designed” search with natural language, to obtain a greater number of results.

I open Pubmed’s home page, type asthma and obesity in children in the search box and press the “Search” button. I get 1169 results, although the number may vary if you do the search at another time.

You can see the result in the first figure. If you look closer, in the left margin of the screen there is a list of text with headings such as “Article types”, “text availability”, etc. Each section is one of the filters that I have selected to be shown in my results screen. You see that there are two links below. The first one says “Clear all” and serves to unmark all the filters that we have selected (in this case, still none). The second one says “Show additional filters” and, if we click on it, a screen with all the available filters appears so that we choose which we want them to be displayed on the screen. Take a look at all the possibilities.

When we want to apply a filter, we just have to click on the text under each filter header. In our case we will filter only the clinical trials published in the last five years and of which the full free text is available (without having to pay a subscription). To do this, click on “Clinical Trial”, “Free full text” and “5 years”, as you can see in the second figure. You can see that the list of results has been reduced to 11, a much more manageable figure than the original 1169.

Now we can remove filters one by one (by clicking on the word “clear” next to each filter), remove them all (by clicking “Clear all”) or add new ones (clicking on the filter we want).

Two precautions to take into account with the use of filters. First, filters will remain active until we deactivate them. If we do not realize it and deactivate them, we can apply them to searches that we do later and get fewer results than expected. Second, filters are built using the MeSH terms that have been assigned to each article at the time of indexing, so very recent articles, which has not been indexed yet and, therefore, have not get their MeSH terms allocated, will be lost when applying the filters. That is why it is advisable to apply the filters at the end of the search process, which is better to make more specific using other techniques such as the use of MeSH or advanced search.

Another option we have with indexes is to automate them for all the searches but without reducing the number of results. To do this we have to open an account in Pubmed by clicking on “Sign in to NCBI” in the upper right corner of the screen. Once we use the search engine as a registered user, we can click on a link above to the right that says “Manage filters” and select the filters we want. In the future, the searches that we do will be without filters, but above to the right you will see links to the filters that we have selected with the number of results in parentheses (you can see it in the first two figures that I have shown). By clicking, we will filter the list of results in a similar way as we did with the other filters, which are accessible without registering.

I would not like to leave the topic of Pubmed and its filters without talking about another search resource: Clinical Queries. You can access them by clicking on the “Pubmed Tools” on the home page of the search engine. Clinical Queries are a kind of filter built by Pubmed developers who filter the search so that only articles related to clinical research are shown.

We type the search string in the search box and we obtain the results distributed in three columns, as you see in the third figure attached. In the first column they are sorted according to the type of study (etiology, diagnosis, treatment, prognosis and clinical prediction guidelines) and the scope of the search that may be more specific (“Narrow”) or less (“Broad”). If we select “treatment” and narrow range (“Narrow”), we see that the search is limited to 25 articles.

The second column lists systematic reviews, meta-analyzes, reviews of evidence-based medicine, etc. Finally, the third focuses on papers on genetics.

If we want to see the complete list we can click on “See all” at the bottom of the list. We will then see a screen similar to the results of a simple or advanced search, as you see in the fourth attached figure. If you look at the search box, the search string has been slightly modified. Once we have this list we can modify the search string and press “Search” again, reapply the filters that suit us, etc. As you can see, the possibilities are endless.

And with this I think we’re going to say goodbye to Pubmed. I encourage you to investigate many other options and tools that are explained in the tutorials of the website, some of which will require you to have an account at NCBI (remember it’s free). You can, for example, set alarms so that the searcher warns you when something new related to certain search is published, among many other possibilities. But that’s another story…



We already know what Pubmed MeSH terms are and how an advanced search can be done with them. We saw that the search method by selecting the descriptors can be a bit laborious, but allowed us to select very well, not only the descriptor, but also some of its subheadings, including or not the terms that depended on it in the hierarchy, etc.

Today we are going to see another method of advanced search a little faster when it comes to building the search string, and that allows us to combine several different searches. We will use the Pubmed advanced search form.

To get started, click on the “Advanced” link under the search box on the Pubmed home page. This brings us to the advanced search page, which you can see in the first figure. Let’s take a look.

First there is a box with the text “Use the builder below to create your search” and on which, initially, we cannot write. Here is going to be created the search string that Pubmed will use when we press the “Search” button. This string can be edited by clicking on the link below to the left of the box, “Edit”, which will allow us to remove or put text to the search string that has been elaborated until then, with natural or controlled text, so we can click the “Search” button and repeat the search with the new string. There is also a link below and to the right of the box that says “Clear”, with which we can erase its contents.

Below this text box we have the search string constructor (“Builder”), with several rows of fields. In each row we will introduce a different descriptor, so we can add or remove the rows we need with the “+” and “-” buttons to the right of each row.

Within each row there are several boxes. The first, which is not shown in the first row, is a dropdown with the boolean search operator. By default it marks the AND operator, but we can change it if we want. The following is a drop-down where we can select where we want the descriptor to be searched. By default it marks “All Fields”, all the fields, but we can select only the title, only the author, only last author and many other possibilities. In the center is the text box where we will enter the descriptor. On its right, the “+” and “-” buttons of which we have already spoken. And finally, in the far right there is a link that says “Show index list”. This is a help from Pubmed, because if we click on it, it will give us a list of possible descriptors that fit with what we have written in the text box.

As we are entering terms in the boxes, creating the rows we need and selecting the boolean operators of each row, the search string will be formed, When we are finished we have to options we can take.

The most common will be to press the “Search” button and do the search. But there is another possibility, which is to click on the link “Add to history”, whereupon the search is stored at the bottom of the screen, where it says “History”. This will be very useful since the saved searches can be entered in block in the field of the descriptors when making a new search and combined with other searches or with series of descriptors. Do you think this is a little messy? Let’s be clear with an example.

Suppose I treat my infants with otitis media with amoxicillin, but I want to know if other drugs, specifically cefaclor and cefuroxime, could improve the prognosis. Here are two structured clinical questions. The first one would say “Does cefaclor treatment improve the prognosis of otitis media in infants?” The second one would say the same but changing cefaclor to cefuroxime. So there would be two different searches, one with the terms infants, otitis media, amoxicillin, cefaclor and prognosis, and another with the terms infants, otitis media, amoxicillin, cefuroxime and prognosis.

What we are going to do is to plan three searches. A first one about article about the prognosis of otitis media in infants; a second one about cefaclor; and a third one about cefuroxime. Finally, we will combine the first with the second and the first with the third in two different searches, using the boolean AND.

Let us begin. We write otitis in the text box of the first search row and click on the link “Show index”. A huge drop-down appears with the list of related descriptors (when we see a word followed by the slash and another word it will mean that it is a subheader of the descriptor). If we look down in the list, there is a possibility that says “otitis / media infants” that fits well to what we are interested in, so we select it. We can now close the list of descriptors by clicking the “Hide index list” link. Now in the second box we write prognosis (we must follow the same method: write part in the box and select the term from the index list). We have a third row of boxes (if not, press the “+” button). In this third row we write amoxicillin. Finally, we will exclude from the search those articles dealing with the combination of amoxicillin and clavulanic acid. We write clavulanic and click on “Show index list”, which shows us the descriptor “clavulanic acid”, which we select. Since we want to exclude these articles from the search, we change the boolean operator of that row to NOT.

In the second screen capture you can see what we have done so far. You see that the terms are in quotes. That’s because we’ve chosen the MeSHs from the index list. If we write the text directly in the box it will appear without quotes, which will mean that the search has been done with natural language (so the accuracy of the controlled language of MeSH terms will have been lost). Note also that in the first text box of the form the search string that we have built so far has been written, which says (((“otitis/media infants”) AND prognosis) AND amoxicillin) NOT “clavulanic acid”. If we wanted, we have already said that we could modify it, but we will leave it as it is.

Now we could click “Search” and make the search or directly click on the “Add to history” link. To see how the number of articles found can be reduced, click on “Search”. I get a list with 98 results (the number may depend on when you do the search). Very well, click on the link “Advanced” (at the top of the screen) to return to the advanced search form.

At the bottom of the screen we can see the first search saved, numbered as # 1 (you can see it in the third figure).

What remains to be done is simpler. We write cefaclor in the text box and give the link “Add to history”. We repeat the process with the term cefuroxime. You can see the result of these actions in the fourth screen capture. You see how Pubmed has saved all the three searches in the search history. If we now want to combine them, we just have to click on the number of each one (a window will open for us to choose the boolean we want, in this case all will be AND).

First we click on # 1 and # 2, selecting AND. You see the product in the fith capture. Notice that the search string has been somewhat complicated: (((((otitis/media infants) AND prognosis) AND amoxicillin) NOT clavulanic acid)) AND cefaclor. As a curiosity I will tell you that, if we write this string directly in the simple search box, the result would be the same. It is the method used by those who totally dominate the jargon of this search engine. But we have to do it with the help of the advanced search form. We click on “Search” and we obtain seven results that will (or so we expect and hope) compare amoxicillin with cefaclor for the treatment of otitis media in infants.

We click again on the link “Advanced” and in the form we see that there is a further search, the # 4, which is the combination of # 1 and # 2. You can already have an idea of how complicated the searching could become combining searches with each other, adding or subtracting according to the boolean operator that we choose. Well, we click on # 1 and # 3 and press “Search”, finding five articles that should deal with the problem we are looking for.

We are coming to the end of my comments for today. I think that the fact that the use of MeSH terms and advanced search yields more specific results than simple search has been fully demonstrated. The usual thing with the simple search with natural language is to obtain endless lists of articles, most of them without interest for our clinical question. But we have to keep one thing in mind. We have already mentioned that a number of people are dedicated to assigning the MeSH descriptors to articles that enter the Medline database. Of course, since the article enters the database until it is indexed (the MeSH is assigned), some time passes and during that time we cannot find them using MeSH terms. For this reason, it could not be a bad idea to do a natural language search after the advanced one and see if there are any articles in the top of the list that might interest us and are not indexed yet.

Finally, commenting that searches can be stored by downloading them to your disk (by clicking the link “download history”) or, much better, creating an account in PubMed by clicking on the link on the top right of the screen that says “Sign in to NCBI. ” This is free and allows us to save the search from one time to another, which can be very useful to use other tools such as Clinical Queries or search filters. But that is another story…

The jargon of the search engine


We saw in a previous post how to do a Pubmed search using the simplest system, which is to enter natural language text in the simple search box and press the “Search” button. This method is quite easy and even works quite well when we are looking for something about very rare diseases but, in general, it will give us a very sensitive and unspecific results list, which in this context means that we will get a large number of articles, but many of them will have little to do with what we are looking for.

In these cases we will have to use some tool to make the result more specific: fewer articles and more related to the problem that originates the search. One of the ways is to perform an advanced search instead of a simple search, but for this we will have to use the browser’s own jargon, the so-called thematic descriptors of controlled language.

A descriptor is a term used to construct indexes, also called thesauri. Instead of using the words of the natural language, they are selected or grouped under specific terms, which are to serve as a key in the index of the search engine database.

The thesaurus, formed by the set of descriptors, is specific to each search engine, although many terms may be common. In the case of Pubmed the descriptors are known as MeSH terms, which are the initials of Medical Subject Headings.

This thesaurus or list of terms with controlled vocabulary has also been developed by the National Library of Medicine and constitutes another database with more than 30,000 terms that are updated annually. Within the National Library there are a number of people whose mission is to analyze the new articles that are incorporated into the Medline database and assign them the descriptors that best fit their content. Thus, when we search using a particular descriptor, we will find the articles that are indexed with this descriptor.

But the thing of the descriptors is a little more complicated than it may seem, since they are grouped in hierarchies (MeSH Tree Structures), being able to a same descriptor to belong to several hierarchies, in addition to having subheadings, of such form that we can search using the general MeSH term or further narrow the search using one of its subheaders. The truth is that reading all this makes us want to forget the search using the thesaurus, but we cannot afford that luxury: the search using the MeSH database is the most effective and accurate, since the language has been controlled to eliminate inaccuracies and synonyms of natural language.

Also, the thing is not so complicated when we get to work with it. Let’s see it with the example we use to display the simple search. We want to compare the efficacy of amoxicillin and cefaclor on the duration of otitis media in infants. After elaborating the structured clinical question we obtain our five terms of search, in natural language: otitis, infants, amoxicillin, cefaclor and prognosis.

Now we can go to the Pubmed home page (remember the shortcut: type pubmed in the browser bar and press control-enter). Below the simple search window we saw that there are three columns. We look at the one on the right, “More Resources” and click on the first option, “MeSH Database”, which gives us access to the homepage of the database descriptors (as seen in the first figure).If we write otitis in the search window we see that Pubmed lends us a hand by displaying a list of terms that look like what we are writing. One of them is otitis media, which is what we are interested in, so we select it and Pubmed takes us to the next page where there are several options to choose from. At the moment I do the search there are three options: “Otitis Media”, “Otitis Media, Suppurative” and “Otitis Media with Effusion”. Notice that Pubmed defines each term, so that we understand well what it means with each term. These are the three MeSH terms that fit what we asked for, but we have to choose one.

The simplest thing we can do from this window is to check the selection box to the left of the term that interests us and click the button on the right side of the screen that says “add to search builder”. If we do this, Pubmed begins to construct the search string starting with the chosen term. If we do this with the first term in the list you will see that the text “Otitis Media” [Mesh] appears in the text box “Pubmed Search Builder” , on the top right of the screen (as you can see in the attached figure).But remember that we have said that the MeSH terms have subheaders. To get them, instead of marking the selection box of the term “Otitis Media”, we click on the term, opening the window with the subheadings, as you can see in the second figure.

Each of the terms with their selection box on the left corresponds to a subheading of the descriptor “Otitis Media”. For example, if we were interested in doing a search directed to the cost of the treatment, we could mark the subheading “economics” and then press the button to add to the search. The text that would appear in the text box of the search string would be “Otitis Media / economics” [Mesh] and the search result would be a bit more specific.

Before leaving the MeSH term window let’s look at a few details. In addition to the subheadings, which can be more or less numerous, the bottom of the page shows the hierarchy of the descriptor (MeSH Tree Structure). Our descriptor is in bold, so we can see which terms it depends on and which ones depend on it. In some cases we may be more interested in using a higher term for the search, so we will have to click on it to go to its own window. If we do this, in general, the search will be more sensitive and less specific (more empty vessels).

We can also click on a term that is below the hierarchy, making the search more specific and decreasing the number of results.

And it does not end here. If we select a MeSH term for the search, it includes the terms that are below in the hierarchy. For example, if we select the descriptor “Otitis Media”, Pubmed will include in the search all that hang from it (mastoidits, otitis with effusion, suppurative otitis and petrositis, which may not interest us at all). This can be avoided by checking the box that says “Do not include MeSH terms found under this term in the MeSH hierarchy”.

Well, I think we’re going to end up with this example, if there is still someone who is still reading at this point. Let’s say we chose the simplest way: let’s go to “Otitis Media” and add it to the search. Next we write the second search term in the search window of the database: infants. We get 14 possibilities, select the first (“Infant”) and add it to the search. We do the same with “Amoxicillin”, “Cefaclor” and “Prognosis”. When we have added all of them to the search string (note that the default boolean operator is AND, but we can change it), the search string is as follows: (“(Otitis Media [Mesh]) AND” Infant ” Mesh]) AND “Amoxicillin” [Mesh]) AND “Cefaclor” [Mesh]) AND “Prognosis” [Mesh].

Finally, click the “Search PubMed” button and get the search result, which in this case is a bit more restricted than we obtained with natural language (this is usually the case).

If we wanted to remove the articles about the treatment with clavulanic acid, as we did in the example with the simple search, we could add the term clavulanate as we add the other terms, but changing the boolean operator AND by the NOT operator. But there is another way that is even simpler. If you notice, when Pubmed gives us the list of results, in the search window of Pubmed is written the search string that has been used and we can add or remove terms from this string, using MeSH or natural language terms, which we prefer. So, in our example, to the text string we would add NOT clavulanate in the search box and we would click on the “Search” button again.

And here we are going to leave it for today. Just saying that there are other ways to use MeSH terms, using the advanced search form, and we can further narrow the results using some resources, like the Clinical Queries or using limits. But that is another story…

The oyster with the thousand pearls


We saw in a previous post that our ignorance as doctors is huge, which forces us to ask ourselves questions about what to do with our patients on numerous occasions.

At this point, we will be interested in seeking and finding the best available evidence on the subject that occupies us, for which we will have to do a good bibliographical search. Although the bibliographic search is defined as the set of manual, automatic and intellectual procedures aimed at locating, selecting and retrieving references or works that respond to our question, the vast majority of the time we simplify the process and we just do a digital search.

In these cases we will have to resort to one of the many biomedical databases available to find the pearl that clarifies our doubt and help remedy our ignorance. Of all these databases, there is no doubt that the most widely used is Medline, the database of the National Library of Medicine. The problem is that Medline is a very large database, with about 16 million articles from more than 4800 scientific journals. So, as is easy to assume, finding what you are looking for may not be a simple task on many occasions.

In fact, when we use Medline what we use is a tool that is known as Pubmed. This is a project developed by the National Center for Biotechnology Information (NCBI for friends), which allows access to three National Library of Medicine databases: Medline, PreMedline and AIDS. These databases are not filtered, so we will need critical reading skills to evaluate the results (there are other resources that give the information already filtered), since the searcher provides nothing more (and nothing less) than the article reference and, in many cases, a brief summary. Best of all, it’s free, which is not the case with all the search tools available.

So, if we want to explore this oyster with thousands of pearls, we will have to learn how to use Pubmed to find the pearls we are looking for. You can enter Pubmed by clicking on this link, although a shortcut is to type pubmed in the address bar of the browser and press control-enter. The browser will know where we want to go and will redirect us to the Pubmed home page. Let’s take a look at starting to use it (see the first attached figure) (Pubmed look changes from time to time, so something may have changed since I wrote this post, probably to improve).

The first thing we see is the simple search box, where we can type the search terms to get the results by clicking the “Search” button. You see that under this box there is a link that says “Advanced”, with which we will access the advanced search screen, which we will talk about of another day. Today we will focus on the simple search.

Below are three columns. The first one says “Using PubMed.” Here you can find help on the use of this tool, including tutorials on the different search modalities and tools that includes Pubmed. I advise you to dive in this section to discover many more possibilities of this search engine than the few that I will tell you in this post.

The second column is the “PubMed Tools”. Here are two of special interest, the “Single Citation Matcher”, to find the reference in PubMed of a specific article knowing some aspects of its bibliographic citation, and “Clinical Queries”, that allow us to filter the results of the searches according to the type of studies or their characteristics.

The third column shows search engine’s resources, such as the MeSH database, which is nothing more than the search term thesaurus that includes Pubmed.

Well, let’s get something to practice. Let us think, for example, that I want to know if it is better to use amoxicillin or cefaclor for the treatment of otitis in infants so that the evolution of the disease is less prolonged. Logically, I can not write this as it is. First I have to build my structured clinical question and then use the components of the question as search terms.

My question would be: in (P) infants with otitis, (I) treatment with cefaclor (C) compared to treatment with amoxicillin, (0) reduces the duration of disease ?. So, with this example, we could use five search terms: otitis, infants, amoxicillin, cefaclor and duration.

In the simple search we will simply enter the words in the search box (natural language) and click on the “Search” box.

The search box supports boolean operators, which are “y”, “o” and “not” (they are often capitalized: AND, OR and NOT). When we put several words in a row without any boolean operators, Pubmed understands that the words are separated by AND. Thus, if we have a term consisting of two words and we want it to be considered as one, we will have to write it in quotation marks. For example, if we write acute appendicitis and we want it to count as a single term, we will have to introduce “acute appendicitis”.

Another useful operator is the truncation, which is to place an asterisk (a wild mark) at the end of the root of the word to search for all words that begin with that root. For example, infan * will search for infant, infancy…

Let’s go with our example. We write otitis AND infants AND amoxicillin AND cefaclor AND course and click on “Search” (see the second attached figure). We were lucky enough; we get only 11 results (you can get a different number if you do the search at another time).

We take a look and see that the works are more or less adjusted to what we are looking for. The only drawback is that it includes articles that study the effect of amoxicillin-clavulanate, which we are not interested in. Well, we’re going to take them off. To the text of search we add NOT clavulanate, and we get an even more limited search.

We only have to select or click on the works that interest us to get the summary (if available) and, in some cases, even get access to the full text, although this will depend on whether the text is freely accessible or on the permissions or subscriptions of the institution from which we access to Pubmed.

So far we have seen the simplest way to search with Pubmed: simple search with free text. The problem is that using this form of search is not always going to get such a specific result, it will be much more frequent that we get thousands of results, most of them without any interest for us. In these cases we will have to resort to other resources such as advanced search, the use of MeSH terms or the use of Pubmed Clinical Queries. But that is another story…

Those who have no questions…


…will never have answers. My biochemistry’s teacher taught me this almost two lives ago, when I was a medicine freshman. I don’t remember what else she taught me but I have this etched into my memory because, I don’t want to remember how many years later, it still remains valid.

And it turns out that the wheel of evidence-based medicine starts spinning with a question. Of course, the problem is that, in medicine, we do not always get an answer for a lot of questions, and according to some, in four out of five times we will not get a satisfactory answer, no matter how well we look for it.

We physicians, let’s face it, are pretty ignorant, and anyone who thinks otherwise is because he doesn’t know how ignorant he’s, which is much worse and more dangerous. We are often challenged by gaps in our knowledge that we want to fill with the available information. It has been estimated that, at Primary Care level, we ask two questions for every 10 patients we receive, increasing this number to five for each patient admitted to Hospital Care. It is easy to understand that we cannot do a bibliography search every time we have a question, so we will have to set priorities.

At our beginnings, when we are very, very ignorant, the questions are quite general. These are called background questions, seeking information on general aspects of diseases and treatments. They are usually composed of a root with a word like how, how much, when or something similar, and a verb followed by the disease or whatever we are dealing with. Questions of this kind are, for example, “what germ causes risperidiosis?” or “how do we treat a dander attack?”

In general, the answer to background questions can be found in textbooks or review articles. There are digital sources of reviews on general topics, such as the one that is undoubtedly one of the most worshiped: UpToDate. We will all meet some uptodater, who are people easily recognizable because, in the first hour of the morning, they already have the latest information obtained from UpToDate, so they give you the answer even before you have asked yourself the question.

But, as we become wiser, the questions that we ask start to involve specific aspects of treatment, prognosis or whatever of a disease in a given patient or population. These advanced or foreground questions often have characteristics that differ qualitatively from that of the background questions: they are usually asked as part of the clinical decision making when we are seeking for information about any problem we are interested in.

Therefore, it’s essential to set them properly and formulate them clearly because if not, they won’t serve to plan the search strategy and to make the right decision that we’re looking for. They are formed by what is known as a structured clinical question, also known in the jargon of evidence-based medicine as PICO questions, after the initial of its components, as we can see below.

P stands for patient, but also for the problem of interest or the clinical description of the situation that we are studying. We must define very well the most relevant characteristics of the group of patients or the population that originated the question, trying not to restrict too much the characteristics of the group, because it may happen that later we find nothing that answers the question. It is often preferable to select the population more generally and, if the search is unspecific (we have many results), we can always restrict it later.

I represents the main intervention, which can be a treatment, a diagnostic test, a risk factor or exposure, etc. C refers to the comparison with which we contrast the intervention, and may be another treatment, placebo or, sometimes, do nothing. This component is not mandatory in the structure of the question, so we can avoid it in cases that we do not need it.

Finally, O represents the outcome of clinical interest in our question, whether in terms of symptoms, complications, quality of life, morbidity, mortality, or any other outcome variable we choose. Thus, it is important to emphasize that the result that we choose should have importance from the clinical point of view, especially importance from the point of view of the patient. For example, in a study to prevent coronary disease, we can measure the effect by decreasing troponin, but the patient will certainly appreciate it more if we estimate the decrease in mortality from myocardial infarction.

Sometimes, as I have already said, it’s not relevant to do any comparison with anything, so PICO becomes PIO. Some people add a fifth parameter, the time, and PICO becomes PICOt. You can also see it as PECO or PECOt if you prefer to say exposure rather than intervention. But, no matter what letters you use, the important thing is to divide the question into its components, because these elements will be the ones that will determine the keywords for the search of information and the type of study design that we’ll need to find the answer (some people add the type of study design as a fifth or sixth letter to PICO).

It’s very important to find a good balance between the scope and accuracy of the question. For instance, the question “in infants with cranial traumatism, do treatment with corticoids improve the prognosis?” may be too general to be of any use. In addition, “in 3-6 month-old infants who fall from the crib from 20 centimeters high and suffer a left side of his forehead traumatism against a carpeted floor, can we improve the prognosis using methylprednisolone at a dose of 2 mg/kg/day during five days?” seems to me as too specific to be used in the search strategy or to be useful for the clinical decision making. A better way of structure the question would be something like “in infants with minor cranial traumatism (minor trauma’s criteria must be previously stablished) does steroid treatment improve the prognosis?” P would be the infants who suffer the trauma, I the treatment with corticosteroids, C would be, in this case, not to give steroids and, finally, O would be the prognosis (which could be replaced by something more specific such as the probability of hospital admission, time until discharge, death, etc).

Let’s see another example: in (P) infants with bronchiolitis, (I) the use of intravenous corticosteroids, instead of inhaled (C), decreases the risk of hospital admission (O)?. Or this one: in (P) infants with otitis, does the use of antibiotics (I) shorten the duration of illness (O)?.

Depending on the type of answer that they are looking for, clinical questions can be classified into four basic types: diagnosis, treatment, prognosis and etiology/harm. Diagnostic questions are about how to select and interpret diagnostic tests. Treatment questions have to do with the treatment we can choose to provide more benefits than risks and with lower economic cost and resources. Prognosis questions give us the probability of a certain clinical course and anticipate complications. Finally, etiology/harm questions are those that serve to identify the causes of diseases, including iatrogenic.

The type of question is important because it will define the type of study design that most likely will answer to our question. Thus, diagnostic questions are best answered with studies with a design that is specific for the evaluation of diagnostic tests. Treatment or harm questions can be answered with clinical trials (ideally) or with observational studies. However, prognostic questions usually require observational studies to find the answer. Finally, just to mention that there’re other types of clinical questions besides the four basic ones such as the frequency (which will be answered using systematic reviews and observational studies) or cost-benefit questions (who need economic evaluation studies).

A well-structured clinical question can help us to solve a clinical problem but it also often serves to make more questions, which with we can fill the gaps of our knowledge and become a little less ignorant. In addition, if we don’t structure our question in its different components, it will be practically impossible to find useful information. Those of you that don’t believe what I’m saying, just write “asthma” in the search field of PubMed or any other search engine and see the number of results. Some browsers, such us Trip Database, even allow search using the PICO structure of the clinically structure question. But unfortunately, in most cases we’ll must find synonyms of each component and find the right descriptor for the database where we are doing the search, usually using advances search techniques. But that’s another story…

The three pillars of wisdom


Surely all of us, with a greater frequency tan we would like, have found a small hole in our knowledge that made us doubt about the diagnostic or treatment steps to take with any of our patients. Following the usual practice, and trying to save time and effort, we have certainly asked to our closest colleagues, hoping that they solve the problem, avoiding us the need to deal with the dread PubMed (who said Google!?). As a last resort, we have consulted a medical book in a desperate attempt to get answers, but nor even the fattest books can free us from having to search on a database occasionally.

And in order to do it well, we should follow the five steps of Evidence-Based Medicine: formulating our question in a structured way (first step), doing our bibliographic search (second step) and critically appraise the articles we find and that we consider relevant to the theme (third step), ending with the last two steps that will be to combine what we have found with our experience and the preferences of the patient (fourth step) and to evaluate how it influences our performance (fifth step).

So we roll up our sleeves, make our structured clinical question, and enter PubMed, Embase or TRIP looking for answers. Covered in a cold sweat, we come up with the initial number of 15234 results and get the desired article that we hope to enlighten our ignorance with. But, even though our search has been impeccable, are we really sure we have found what we need?. Here it starts the arduous task of critically appraise the article to assess its actual utility to solve our problem.

This step, the third of the five we have seen and perhaps the most feared of all, is indispensable within the methodological flow of Evidence-Based Medicine. And this is so because all that glitters is not gold: even articles published in prestigious journals by well-known authors may have poor quality, contain methodological errors, have nothing to do with our problem or have errors in the way of analyzing or presenting the results, often in a suspiciously interested way. And this is not true because I say so, because there are even people who think that the best place to send 90% out of what is published is the trash can, regardless of whether the journal has impact fact or if the authors are more famous than Julio Iglesias (or his son Enrique, for that matter). Our poor excuse to justify our lack of knowledge about how to produce and publish scientific papers is that we are clinicians rather than researchers, and of course the same is often the case with journals reviewers, who overlook all the mistakes that clinicians make.

Thus, it is easy to understand that critical appraising is a fundamental step in order to take full advantage of the scientific literature, especially in an era in which information abounds but we have little time available to evaluate it.

The first thing we must do is always to assess whether the article answers to our question. This is usually the case if we have developed the clinical question correctly and we have done a good search of the available evidence but, anyway, we should always check that the study population, the intervention, etc., match with what we are seeking.

Before entering into the systematic of critically appraising, we will take a look over the document and its summary to try to see if the article in question can meet our expectations. The first step we must always take is to evaluate whether the paper answers our question. This is often the case if we have correctly elaborated the structured clinical question and we have made a good search for the available evidence, but it is always appropriate to check that the type of population, study, intervention, etc. are in line with what we are looking for.

Once we are convinced that the article is what we need, we will perform a critical appraising. Although the details depend on the type of study design, we are always based on three basic pillars: validity, relevance and applicability.

Appraising validity consist on checking the scientific rigor of the paper to find out how much close to the true it is. There are a number of common criteria to all studies, such as the correct design, an adequate population, the existence of homogeneous intervention and control groups at the beginning of the study, a proper monitoring, etc. Someone thought this term should be best called internal validity, so we can find it with this name.

The second pillar is clinical importance, which measures the magnitude of the effect found. Imagine that a new hypotensive is better than the usual one with a p-value with many zeroes, but that it decrease blood pressure an average of 5 mmHg. No matter how many zeroes the p-value have (which is statistically significant, we cannot denied it), we have to admit that the clinical effect is rather ridiculous.

The last pillar is the clinical applicability, which consist in assessing whether the context, patients and intervention of the study are sufficiently similar to our environment as to generalize the results. The applicability is also known as external validity.

Not all scientific papers can be described favorably in these three aspects. It may happen that a valid study (internal validity) finds a significant effect that cannot be applied to our patients. And we must not forget that we are using just a working tool. Even the most suitable study must be appraise in terms of benefits, harms and costs, and patient preferences, the latter aspect one that we forget more often than it would be desirable.

For those with a fish memory, there are some templates by group CASP that are recommended to use as a guide to make critical reading without forgetting any important aspect. Logically, the specifics measures of association and impact and the requirements to meet internal validity criteria depend specifically on the type of the study design that we are dealing with. But that’s another story…

A bias by absence


We can find strength through unity. It is a fact. Great goals are achieved more easily with the joining of the effort of many. And this is also true in statistics.
In fact, there are times when clinical trials do not have the power to demonstrate what they are pursuing, either because of lack of sample due to time, money or difficulty recruiting participants, or because of other methodological limitations. In these cases, it is possible to resort to a technique that allows us sometimes to combine the effort of multiple trials in order to reach the conclusion that we would not reach with any of the trials separately. This technique is meta-analysis.
Meta-analysis gives us an exact quantitative mathematical synthesis of the studies included in the analysis, generally the studies retrieved during a systematic review. Logically, if we include all the studies that have been done on a topic (or, at least, all that are relevant to our research), that synthesis will reflect the current knowledge on the subject. However, if the collection is biased and we lack studies, the result will reflect only the articles collected, not the total available knowledge.
When planning the review we must establish a global search structure to try to find all the articles. If we do not do this we can make a recovery bias, which will have the same effect on the quantitative analysis as the publication bias has. But even with modern electronic searches, it is very difficult to find all the relevant information on a particular topic.
In cases of missing studies, the importance of the effect will depend on how the studies are lost. If they are lost at random, everything will be in a problem of less information, so the accuracy of our results will be less and the confidence intervals will be broader, but our conclusions may be correct. However, if the articles that we do not find are systematically different from those we find, the result of our analysis may be biased, since our conclusions can only be applied to that sample of papers, which will be a biased sample.
There are a number of factors that may contribute to the publication bias. First, the studies with meaningful results are more likely to be published and, within these, they are more likely to be published when the effect is greater. This means that studies with negative results or with effects of small magnitude may not be published, so we will draw a biased conclusion from the analysis of only large studies with a positive result.
Secondly, of course, published studies are more likely to come into our hands than those that are not published in scientific journals. This is the case of doctoral theses, communications to congresses, reports from government agencies or even studies pending to be published by researchers of the subject we are dealing with. For this reason it is so important to do a search that includes this type of work, which fall within the term of gray literature.
Finally, a number of biases can be listed that influence the likelihood that a paper will be published or retrieved by the investigator performing the systematic review such as language bias (we limit the search by language), availability bias (to include only those studies that are easy to retrieve by the researcher), cost bias (to include studies that are free or cheap), familiarity bias (only those of the discipline of the investigator), duplication bias (those who have significant outcomes are more likely to be published more than once) and citation bias (studies with significant outcome are more likely to be cited by other authors).
One may think that losing studies during the review cannot be so serious, since it could be argued that unpublished studies in peer-reviewed journals are often of poorer quality, so they do not deserve to be included in the meta-analysis. However, it is not clear that the scientific journals ensure the methodological quality of the study or that this is the only method to do so. There are researchers, such as government agencies, who are not interested in publishing in scientific journals, but in producing reports for those who commission them. In addition, peer review is not a quality assurance because, too often, neither the researcher who performs the study nor those in charge of reviewing it have a methodology training that ensures the quality of the final product.
There are tools to assess the risk of publication bias. Perhaps the simplest may be to represent a forest plot ordered with the most accurate studies at the top and the less at the bottom. As we move down the precision of the results decreases, so that the effect must oscillate to both sides of the summary measure result. If it only oscillates towards one of the sides, we can indirectly assume that we have not detected the works that must exist that oscillate towards the opposite side, reason why surely we will have a bias of publication.
funnel_sesgoAnother similar procedure is the use of the funnel plot, as seen in the attached image. In this graph the effect size is plotted on the X axis and on the Y axis a measure of the variance or the sample size, inverted. Thus, at the top will be the largest and most accurate studies. Once again, as we go down the graph, the accuracy of the studies is smaller and they are shifted sideways by random error. When there is publication bias this displacement is asymmetrical. The problem of the funnel plot is that its interpretation can be subjective, so there are numerical methods to try to detect the existence of publication bias.
And, at this point, what should we do in the face of a publication bias? Perhaps the most appropriate thing is not to ask if there is bias, but how much it affects my results (and assume that we have left studies without being included in the analysis).
The only way to know if publication bias affects our estimates would be to compare the effect on recovered and unrecovered studies, but of course, then we would not have to worry about publication bias.
In order to know if the observed result is robust or, conversely, it is susceptible to be biased by a publication bias, two methods have been devised called as the fail-safe N methods.
The first method is the Rosenthal’s fail-safe N method. Suppose we have a meta-analysis with an effect that is statistically significant, for instance, a relative risk greater than one with a p <0.05 (or a 95% confidence interval that does not include the null value, one). Then we ask ourselves a question: how many studies with RR = 1 (null value) will have to be included until p is not significant? If we need few studies (less than 10) to invalidate the value of the effect, we may be concerned that the effect may actually be null and our significance is the result of a publication bias. Conversely, if many studies are needed, the effect is likely to be truly significant. This number of studies is what the letter N of the method name means. The problem with this method is that it focuses on statistical significance rather than on the relevance of results. The correct thing would be to look for how many studies are necessary so that the result loses clinical relevance, not statistical significance. In addition, it assumes that the effects of missing studies are zero (one in the case of relative risks and odds ratios, zero in cases of mean differences), when the effect of missing studies may go the other way than the effect we detected or In the same direction but of smaller magnitude. To avoid these drawbacks there is a variation of the previous formula which values statistical significance and clinical significance. With this method, which is called the Orwin´s fail-safe N, we calculate how many studies are needed to bring the value of the effect to a specific value, which will generally be the smallest effect that is clinically important. This method also allows specifying the average effect of missing studies.
And here we leave the meta-analysis and publication bias for today. We have not talked about any other mathematical methods to detect publication bias like Begg’s and Egger’s. There is even some graphic method apart from the ones we have mentioned, such as the trim and fill method. But that is another story…

Three feet of a cat


To look for three legs of a cat, or splitting hairs, is a popular Spanish saying. It seems that when one looks for three feet of a cat he tries to demonstrate something impossible, generally with tricks and deceptions. As the English speakers say, if it ain’t broke, don’t fix it. In fact, the initial saying referred to looking for five feet instead of three. This seems more logical, since as cats have four legs, finding three of them is easy, but finding five is impossible, unless we consider the tail of the cat as another foot, which does not make much sense.

But today we will not talk about cats with three, four or five feet. Let’s talk about something a little more ethereal, such as multivariate multiple linear regression models. This is a cat with a lot of feet, but we are going to focus only on three of them that are called collinearity, tolerance and inflation factor (or increase) of the variance. Do not be discouraged, it’s easier than it may seem.

We saw in a previous post how simple linear regression models related two variables to each other, so that the variations of one of them (the independent variable or predictor) could be used to calculate how the other variable would change (the dependent variable). These models were represented by the equation y = a + bx, where x is the independent variable and y the dependent variable.

However, multiple linear regression adds more independent variables, so that it allows to make predictions of the dependent variable according to the values of the predictor or independent variables. The generic formula would be as follows:

y = a + bx1 + cx2 + dx3 + … + nxn, where n is the number of independent variables.

One of the conditions for the multiple linear regression models to work properly is that the independent variables are actually independent and uncorrelated.

Imagine an absurd example in which we put in the model the weight in kilograms and the weight in pounds. Both variables will vary in the same way. In fact the correlation coefficient, R, will be 1, since practically the two represent the same variable. Such foolish examples are difficult to see in scientific work, but there are others less obvious (including, for example, height and body mass index, which is calculated from weight and height) and others that are not at all evident for the researcher. This is what is called collinearity, which is nothing more than the existence of a linear association between the set of independent variables.

Collinearity is a serious problem for the multivariate model, since the estimates obtained by it are very unstable, as it becomes more difficult to separate the effect of each predictor variable.

Well, to determine if our model suffers from collinearity we can construct a matrix where the coefficients of correlation, R, of some variables with others are shown. In those cases in which we observe high R, we can suspect that there is collinearity. However, if we want to quantify this we will resort to the other two feet of the cat that we mentioned at the beginning: tolerance and inflation factor of variance.

If we square the coefficient R we obtain the coefficient of determination (R2), which represents the percentage of the variation (or variance) of a variable that is explained by the variation in the other variable. Thus, we find the concept of tolerance, which is calculated as the complement of R2 (1-R2) and represents the proportion of the variability of that variable that is not explained by the rest of the independent variables included in the regression model.

In this way, the lower the tolerance, the more likely there is collinearity. Collinearity is generally considered to exist when R2 is greater than 0.9 and therefore the tolerance is below 0.1.

We only have to explain the third foot, which is the inflation factor of the variance. This is calculated as the inverse of the tolerance (1 / T) and represents the proportion of the variability (or variance) of the variable that is explained by the rest of the predictor variables of the model. Of course, the greater the inflation factor of the variance, the greater the likelihood of collinearity. Generally, collinearity is considered to exist when the inflation factor between two variables is greater than 10 or when the mean of all inflation factors of all independent variables is much greater than one.

And here we are going to leave the multivariate models for today. Needless to say, everything we have told is done in practice using computer programs that calculate these parameters in a simple way.

We have seen here some aspects of multiple linear regression, perhaps the most widely used multivariate model. But there are others, such as multivariate analysis of variance (MANOVA), factors analysis, or clusters analysis. But that is another story…

In search of causality


In medicine we often try to look for cause-effect relationships. If we want to show that the drug X produces an effect, we have only to select two groups of people, one group we give the drug, the other group we do not give it and see if there are differences.

But it is not so simple, because we can never be sure that differences in effect between the two groups actually are due to other factors than the treatment we have used. These factors are the so-called confounding factors, which may be known or unknown and which may bias the results of the comparison.

To resolve this problem a key element of a clinical trial, randomization, was invented. If we divide the participants in the trial between the two branches randomly we will get these confounding variables to be distributed homogeneously between the two arms of the trial, so any difference between the two will have to be due to the intervention. Only in this way can we establish cause-effect relationships between our exposure or treatment and the outcome variable we measure.

The problem of quasi-experimental and observational studies is that they lack randomization. For this reason, we can never be sure that the differences are due to exposure and not to any confounding variable, so we cannot safely establish causal relationships.

This is an annoying inconvenience, since it will often be impossible to carry out randomized trials either for ethical, economic reasons, the nature of the intervention or whatever. That is why some tricks have been invented in order to establish causal relations in the absence of randomization. One of these techniques is the propensity score we saw in an earlier post. Another is the one we are going to develop today, which has the nice name of regression discontinuity.

Regression discontinuity is a quasi-experimental design that allows causal inference in the absence of randomization. It can be applied when discontinuity regression_thresholdthe exposure of interest is assigned, at least partially, according to the value of a continuous random variable if this variable falls above or below a certain threshold value.

Consider, for example, a hypocholesterolemic drug that we will use when LDL cholesterol rises above a given value, or an antiretroviral therapy in an AIDS patient that we will indicate when his CD4 count falls below a certain value. There is a discontinuity in the threshold value of the variable that produces a sudden change in the probability of assignment to the intervention group, as I show in the attached figure.

In these cases where the allocation of treatment depends, at least in part, on the value of a continuous variable, the allocation in the vicinity of the threshold is almost as if it were random. Why? Because determinations are subject to random variability by sampling error (in addition to the variability of biological variables themselves), which makes individuals very close to the threshold, above or below, very similar in terms of the variables that may act as confounders (being above or below the threshold may depend on the random variability of the result of the measurement of the variable), similar to what happens in a clinical trial. At the end of the day, we may think that a clinical trial is nothing more than a discontinuity design in which the threshold is a random number.

The math of regression discontinuity is not for beginners so I do not intend to explain it here (I would have to understand it first), so we will settle for knowing some terms that will help us to understand the works that use this methodology.

Regression discontinuity may be sharp or fuzzy. In the sharp one, the probability of assignment changes from zero to one at the threshold (the allocation of treatment follows a deterministic rule). For example, treatment is initiated when the threshold is crossed, regardless of other factors. On the other hand, in the fuzzy regression there are other factors at stake that make the probability of allocation change in the threshold, but not from zero to one, but may depend on those other factors added.

Thus, the result of the regression model varies somewhat depending on whether it is a sharp or fuzzy regression discontinuity. In the case of sharp regression, the so-called average causal effect is calculated, according to which participants are assigned to the intervention with certainty if they cross the threshold. In the case of fuzzy regression, the allocation is no longer performed according to a deterministic model, but according to a probabilistic one (according to the threshold value and other factors that the researcher may consider important). In these cases, an intention-to-treat analysis should be done according to the difference in the probability of allocation near the cut-off point (some may not exceed the threshold but be assigned to the intervention because the investigator considers the other factors).

Thus, the probabilistic model will have to measure the effect on the compliers (those assigned to the intervention), so the regression model will give us the complier average causal effect, which is the typical measure of fuzzy regression discontinuity.

And I think we’re going to leave it for today. We have not said anything about the regression equation, but suffice it to say that it takes into account the slopes of the probability function of allocation before and after the threshold and an interaction variable for the possibility that the effects of the treatment are heterogeneous on both sides of the threshold. As you see, everything is quite complicated, but for that are the statistical packages like R or Stata that implement these models with little effort.

Finally, to say only that it is usual to see models that use linear regression for quantitative outcome variables, but there are extensions of the model that use dichotomous variables and logistic regression techniques, and even models with survival studies and time-to-event variables. But that is another story…



In the best-known sense, censorship is the action of examining a work intended for the public, suppressing or modifying the part that does not fit certain political, moral or religious aspect, to determine whether or not it can be published or exhibited. So what do we mean in statistics when we talk about censored data? Nothing to do with politics, morality or religion. In order to explain what a censored data is, we must first discuss the time-to-event variables and survival analyzes.

In general, we can say that there are three types of variables: quantitative, qualitative and time-to-event. The first two are fairly well understood in general, but the time-to-event are a little more complicated to understand.-

Imagine that we want to study the mortality of that terrible disease that fildulastrosis is. We could count the number of deaths at the end of the study period and divide them by the total population at the beginning. For example, if at the beginning there are 50 patients and four die during follow-up, we could calculate the mortality as 4/50 = 0.08, or 8%. Thus, if we have followed the population for five years, we can say that the survival of the disease at five years is 92% (100-8 = 92).

Simple, isn’t it? The problem is that this is only valid when all subjects have the same follow-up period and no losses or dropouts occur throughout the study, a situation that is often far from the reality in most cases.

In these cases, the correct thing to do is to measure not only if death occurs (which would be a dichotomous variable), but also when it occurs, also taking into account the different follow-up period and the losses. Thus, we would use a time-to-event variable, which is composed of a dichotomous variable (the event being measured) and a continuous variable (the follow-up time when it occurs).

Following the example above, participants in the study could be classified into three types: those who die during follow-up, those who remain alive at the end of the study, and those who are lost during follow-up.

Of those who die we can calculate their survival but, what is the survival of those who are alive at the end of the study? And what is the survival of those who are lost during follow-up? It is clear that some of the lost may have died at the end of the study without us detecting it, so our measure of mortality will not be accurate.

And this is where we find the censored data. All those who do not present the event during the survival study are called censored (losses and those who finish the study without presenting the event). The importance of these censored data is that they must be taken into account when doing the survival study, as we will see below.

The methodology to be followed is to create a survival table that takes into account the events (in this case the deaths) and the censored data, as we can see in the attached table.

The columns of the table represent the following: x, the year number of the follow-up; Nx, the number of participants alive at the beginning of that year; Cx, the number of losses of that year (censored); Mx, the number of deaths during that period; PD, probability of dying in that period; PPS, the probability of surviving in that period (the probability of not presenting the event); And PGS, the global probability of survival up to that point.censoringAs we see, the first year we started with 50 participants, one of whom died. The probability of dying in that period is 1/50 = 0.02, so the probability of survival in the period (which is equal to the global since it is the first period) is 1-0.02 = 0, 98.

In the second period we start with 49 and no one dies or is lost. The PD in the period is zero and survival one. Thus, the overall probability will be 1×0.98 = 0.98.

In the third period we continue with 49. Two are lost and one dies. The PD is 1/49 = 0.0204 and the PPS is 1-0.0204 = 0.9796. If we multiply the PSP by the global of the previous period, we obtain the overall survival of this period: 0.9796×0.98 = 0.96.

In the fourth period we started with 46 participants, resulting in five losses and two deaths. The PD will be 2/46 = 0.0434, the PPS of 1-0.0434 = 0.9566 and the PGS of 0.9566×0.96 = 0.9183.

And last, in the fifth period we started with 39 participants. We have two censored and no event (death). PD is zero, PPS is equal to one (no one dies in this period) and PGS 1×0.9183 = 0.9183.

Finally, taking into account the censored data, we can say that the overall survival at five years of fildulastrosis is 91.83%.

And with this we are going to leave it for today. We have seen how a survival table with censored data is constructed to take into account unequal follow-up of participants and losses during follow-up.

Only two thoughts before finishing. First, even if we talk about survival analysis, the event does not have to be the death of the participants. It can be any event that occurs throughout the study follow-up.

Second, the time-to-event and censored data are the basis for performing other statistical techniques that estimate the probability of occurrence of the event under study at a given time, such as the Cox regression models. But that is another story…