Menu
Menu
In a seminal 1989 article published in the New England Journal of Medicine, the American nephrologist Jerome P. Kassirer wrote, “Absolute certainty in diagnosis is unattainable, no matter how much information we gather, how many observations we make or how many tests we perform.”
A British National Health System survey in 2009 reported that 15% of its patients were misdiagnosed, and according to a study published in 2014, each year in the US, approximately 12 million adults (5% of the total adult patient population) who sought outpatient medical care were misdiagnosed in hospital settings, in outpatient clinics and doctors’ offices.
During the ongoing COVID-19 pandemic, there is a lot of buzz regarding the possible errors in diagnoses with both RT-PCR tests and the faster antibody-based tests, all over the world. To understand how serious these errors might be during a pandemic, we need to understand the nature of different types of errors.
When a new test is rapidly created and deployed, as in the case of the current coronavirus, its accuracy cannot be exactly predicted beforehand. A test developed under controlled lab conditions might behave in a different way when applied in the real world, and this might enhance the likelihood of errors due to many unforeseen events.
In reality, we encounter four types of scenarios:
While (1) and (4) are desirable situations, (2) and (3) types of testing errors.
Now, the ratio of (1) + (4) to (1) + (2) + (3) + (4) obviously represents the proportion of correct results from the test, and is a measure called accuracy. The ratio of (1) to (1) + (2) is the proportion of correct diagnoses out of all those who tested positive, and is a measure called precision.
Note that both (1) and (3) both represent people who actually have COVID-19. The likelihood of a test result coming back positive when the person actually has COVID-19 is known as sensitivity. It is equal to the ratio of (1) to (1) + (3). Finally, the specificity of the test is the ratio of (4) to (2) + (4). Here both (2) and (4) stand for people who do not have COVID-19, so specificity is the likelihood of a test result coming back negative when COVID-19 is absent in the person.
A quick recap:
Accuracy = true positives + true negatives / all results
Precision = true positives / true positives + false positives
Sensitivity = true positives / true positives + false negatives
Specificity = true negatives / true negatives + false positives
Different values of the sensitivity of RT-PCR tests to COVID-19 have been reported in different parts of the globe. According to one analysis with 51 people in China, up to 29% of people with the coronavirus tested negative. Studies in the US returned multiple values – sometimes it was 95%, sometimes 85%, and even 75%.
If the sensitivity of one RT-PCR test kit is 90%, for example, and two successive negative test results are used to declare someone free of disease, there is still a 1% chance that a person with the disease would be declared negative. And if that is so, about 200,000 COVID-19 patients could have been wrongly diagnosed among the more than 20 million tests worldwide! This may be sufficient ground for quarantining patients with ‘negative’ test results for the recommended period (say, 14 days) in order to restrict the virus’s spread.
Antibody tests could help to find people who can be presumed to be immune (our experience with other viruses is that after the first infection has been defeated, a body becomes immune to the causative pathogen for a certain period; we still need studies to confirm this is true of the new coronavirus as well). But at the moment, we don’t have enough information on the accuracy of antibody tests. The very limited data from other countries suggests that such tests might have fewer false negative results than RT-PCR tests but more false positives.
Usually, researchers design their tests to be as sensitive and as specific as possible – but despite their best efforts, Kassirer’s statement holds: no test can be 100% accurate. So when a positive (or negative) result is obtained, what is the probability of having (or not having) COVID-19?
An 18th century concept in statistics, known as Bayes’s theorem, can help us. This theorem tells us how to calculate the probability of an event given that another event has happened. For example, say people in a particular colony are being tested, and 20% of them actually have the disease. Next, say the sensitivity (probability of a positive result given the disease is present) of the test being used is 80% and its specificity (probability of a negative test result given the disease is not present) is 90%. A little bit of math yields the following probabilities:
According to Bayes’s theorem, the probability that the disease is present given a negative test result can be obtained by multiplying the probability of disease in the locality (0.20) and the probability of a negative result given the disease is present (0.20), then dividing this by the probability of a negative test result (0.76). This value comes out to be 5.26%. That is, a little more than 1 in 20 people who test negative may actually have the disease. Similarly, the probability of ‘no disease’ given a positive test result is 33.3%.
If the disease’s prevalence in the colony rises to 50%, these two figures become 18.2% and 11.1%, respectively. If the prevalence increases to 80%, these figures become 47.1% and 3%, respectively.
In his 2015 book The Laws of Medicine: Field Notes From an Uncertain Science, Siddhartha Mukherjee, the Pulitzer Prize-winning author and one of the world’s foremost cancer researchers, wrote that “a strong intuition is much more powerful than a weak test”. Thus, a test result for a person should not depend on the accuracy of the test alone but also on the estimated risk of disease before testing.
June 10, 2020 — In a new study, Johns Hopkins researchers found that testing people for SARS-CoV-2 (COVID-19) too early in the course of infection is likely to result in a false negative test, even though they may eventually test positive for the virus.[1] This is important to understand since many hospitals are using these COVID tests to screen patients before imaging exams, diagnostic testing or procedures.
The report found even a week after infection, one in five people who had the virus had a negative test result. The findings was published in the May 13 issue of Annals of Internal Medicine.
“A negative test, whether or not a person has symptoms, doesn’t guarantee that they aren’t infected by the virus,” said Lauren Kucirka, M.D., Ph.D., M.Sc., obstetrics and gynecology resident at Johns Hopkins Medicine. “How we respond to, and interpret, a negative test is very important because we place others at risk when we assume the test is perfect. However, those infected with the virus are still able to potentially spread the virus.”
Kucirka said patients who have a high-risk exposure should be treated as if they are infected, particularly if they have symptoms consistent with COVID-19. This means communicating with patients about the tests’ shortcomings. One of several ways to assess for the presence of SARS-CoV-2 infection is a method called reverse transcriptase polymerase chain reaction (RT-PCR). These tests rapidly make copies of and detect the virus’s genetic material. However, as shown in tests for other viruses such as influenza, if a swab misses collecting cells infected with the virus, or if virus levels are very low early during the infection, some RT-PCR tests can produce negative results. Since the tests return relatively rapid results, they have been widely used among high-risk populations such as nursing home residents, hospitalized patients and healthcare workers. Previous studies have shown or suggested false negatives in these populations.
For the new analysis, Johns Hopkins Medicine researchers reviewed RT-PCR test data from seven prior studies, including two preprints and five peer-reviewed articles. The studies covered a combined total of 1,330 respiratory swab samples from a variety of subjects including hospitalized patients and those identified via contact tracing in an outpatient setting.
Using RT-PCR test results, along with reported time of exposure to the virus or time of onset of measurable symptoms such as fever, cough and breathing problems, the researchers calculated the probability that someone infected with SARS-CoV-2 would have a negative test result when they had the virus infection. In the published studies, healthcare providers collected nasal and throat samples from patients and noted the time of virus exposure or symptom onset and sample collection.
From this data, the Johns Hopkins researchers calculated daily false-negative rates, and have made their statistical code and data publicly available so results can be updated as more data are published.
The researchers estimated that those tested with SARS-CoV-2 in the four days after infection were 67 percent more likely to test negative, even if they had the virus. When the average patient began displaying symptoms of the virus, the false-negative rate was 38 percent. The test performed best eight days after infection (on average, three days after symptom onset), but even then had a false negative rate of 20 percent, meaning one in five people who had the virus had a negative test result.
“We are using these tests to rule out COVID-19, and basing decisions about what steps we take to prevent onward transmission, such as selection of personal protective equipment for healthcare workers,” Kucirka explained. “As we develop strategies to reopen services, businesses and other venues that rely on testing and contact tracing, it is important to understand the limitations of these tests.”
Ongoing efforts to improve tests and better understand their performance in a variety of contexts will be critical as more people are infected with the virus and more testing is required. The sooner people can be accurately tested and isolated from others, the better we can control the spread of the virus, the researchers said.
Another John Hopkins study in March found the average incubation period for COVID-19 was approximately five days.[2] This was originally used as a guide by some in developing quarantine guidelines. However, this new study shows that test COVID PCR test results are not reliable for a firm diagnosis until well after a week of infection.
Additional authors include Denali Boon, Stephen Lauer, Oliver Layendecker and Justin Lessler and of Johns Hopkins.
Funding for the study was provided by the National Institute of Allergy and Infectious Diseases (R01AI135115 and T32DA007292), the Johns Hopkins Health System and the U.S. Centers for Disease Control and Prevention (NU2GGH002000).
