Recent Study: Get the Facts

Apparently, doctors who order a lot of imaging tests are more likely to have electronic systems that let them view those images in their offices.

That is the finding of an article appearing in the March 2012 issue of Health Affairs by McCormick and colleagues. This is not a particularly surprising observation. What is surprising is that the authors go far beyond the scope of their research to conclude that “the federal government’s ongoing, multibillion-dollar effort to promote the adoption of health information technology may not yield anticipated cost savings from reductions in duplicative diagnostic testing. Indeed, it is possible that computerization will drive costs in this area up, not down.”

While such interpretations may make for attention-getting media headlines, it’s important to get the facts. There are several reasons why McCormick’s study ultimately tells us little about the ability of electronic health records (EHRs) to reduce costs, and why it tells us nothing about the impact of EHRs on improving care.

The Study Was Not About EHRs At All, Much Less Their “Meaningful Use”

McCormick’s article considered how physicians react to electronic viewing of imaging results, not EHRs. In fact, when the authors  looked at EHR use, they found that “use of an electronic health record system showed no association with test ordering.” Also, the study data were from 2008, before the passage of the HITECH Act and the linking of payment incentives to the meaningful use of EHRs.

Moreover, the authors did not consider clinical decision support, which helps give providers the data tools they need to make appropriate care recommendations and the ability to exchange information electronically. These are two of the most critical features of certified EHRs, which  have been shown in multiple well-designed studies to reduce unnecessary and duplicative tests.

The Study Falls Prey to the Classic Fallacy of Using Association to Suggest Causality

Ordering more tests may lead to buying an image viewing system, not the other way around. As my predecessor David Brailer pointed out in the New York Times Exit Disclaimer , this was not a randomized trial, but an observational study (the National Ambulatory Medical Survey) that was not designed to answer the question of cost, or associations between EHRs and quality.

As a result, many other variables that could affect physician behavior could not be examined in this study, including:

  • Having a sicker patient population,
  • Level of physician training,
  • Approach to defensive medicine, and, importantly,
  • Financial arrangements.

The authors themselves correctly note that physicians who have a financial stake in imaging are more likely to purchase systems with image results capabilities, making self-referral a likely explanation for their results.

The Study Did Not Consider the Appropriateness of Imaging Tests

The study also looked at the quantity of tests ordered without any regard to whether the tests were medically necessary. As both patients and providers well know, an appropriate follow-up for a suspicious nodule is a test you want to have. Clearly, we need to assess the success of EHR systems based on how they improve the care that patients receive, not just the number of tests ordered. Unfortunately, this study focused on the numbers, while ignoring the patients.

Reducing Test Orders Is Not the Way that Health IT Is Meant to Reduce Costs

The ultimate impact of EHRs on reducing cost will be through improvements in the coordination and quality of care, and the prevention of unnecessary and costly complications and hospitalizations. Providers who are embracing new delivery and payment models such as Accountable Care Organizations and Patient-Centered Medical Homes know that meaningful use of EHRs is a critical foundation for being able to improve quality while reducing cost.

Don’t Believe the Hype

Systematic reviews Exit Disclaimer of the evidence show that EHRs have the ability to give providers the information and tools to provide better care and reduce waste. A 2011 study in Health Affairs itself found that 92 percent of studies on health IT were “positive overall” about the prospect that technology would improve efficiency, reduce unnecessary tests, and improve the quality of care that patients receive.

Seemingly surprising headlines can be tempting, but it’s important to get the facts. The evidence shows we are on the right track to establishing the health IT foundations for a true 21st Century U.S. health system where patients get better care, while we reduce health care costs.

17 Comments

  1. Tom M. Gomez says:

    Thanks for stepping up Farzad!!

  2. Lew Sandy MD says:

    great post! After reading all the media reports on this, I went back and read the actual Health Affairs paper. The media are widely (and wildly) over/misinterpreting this study.

    “Ordering more tests may lead to buying an image viewing system, not the other way around” is in my view the most likely explanation of the finding of association (or they are both correlated to a third, unmeasured variable).

    This cross-sectional study of the “risk” of ordering tests in conjuction with office vists really cannot offer much insight into the impact of HIT on utilization, appropriateness, safety or quality.

  3. Doctor Dave says:

    Authors are always looking for a way to make a story even at the cost of trying to blame the goverment poor spending and make up a lot of there content to claim headlines while most of there research is either old outdated or not enough or proper data was collected on the topic. That’s why articles like this are valuable because they provide a great information resource for people who look for the truth rather then just believe everything that is posted on the news.

  4. Doctor Ray says:

    Well, believe what you will, but I find it difficult to disagree with the conclusion from the source paper:

    “Whatever the explanation for our findings, they emphasize the importance of establishing the benefits of computerization rather than estimating them in the absence of data, or generalizing from small studies at a few atypical institutions. History urges caution in assuming that advances in medical technology will result in cost savings. In fact, the opposite is more often the case.”

    Burden of proof is on you.

  5. Todd Stein says:

    Thank you for this post. The limitations of this study are significant. It’s clear that the authors should have considered whether CPOE and/or decision support would decrease redundant and unnecessary tests. They acknowledge that other studies of systems with CPOE and decision support have shown a decrease in duplicate testing.

    For more on this: http://blog.allscripts.com/2012/03/08/a-flawed-approach-to-studying-ehr-cost-effectiveness/

  6. dr fotsch says:

    A sanguine review of the study and stated conclusions leads to concurrence with Dr. Mostashari. In addition to not accounting for decision support and related functionality mentioned by Dr. Mostashari, the study authors apparently ignored the sample bias of physicians who were early adopters of EHRs vs non-adopter cohorts. They claim to have accounted for this by “adjusting for patient and physician characteristics,” but the one characteristic (‘and what may be one of the most important ones) is that they were comparing the workflow and technology characteristics of the minority of docs who were early adopters of EHRs vs the majority of non-adopters physicians. Keep in mind that the study time period pre-dates the most recent uptick in EHR adoption so the sampling bias is even more substantial than it would be had the data come from 2011 or 2012. With that in mind the study data seems to indicate that docs who willingly choose and use technology in their offices are more likely to order technology-based tests for their patients. This is akin to proving that those who were first to buy an iPad were more likely to give on as a gift. Also of note is the fact that docs who were early adopters of EHRs were more likely to be employed by organizations where tests are performed, highly accessible and a source of institutional revenue- but that’s another story.

    If the authors really wanted to demonstrate the impact of EHR adoption on physician imaging and test ordering they would have had to compare the ordering habits for the SAME docs before and after EHR adoption- or to otherwise create a real control group and do a real test. Dr Brailer quoted in the related WSJ review of the study referenced the same flaw when he stated that he was unconvinced by the study’s conclusions because they were based on a correlation in the data and were not the result of a controlled test.

    EHRs may have an impact on imaging and other clinical test ordering. The referenced study doesn’t seem to shed any meaningful light on the topic

  7. Dantes says:

    “The ultimate impact of EHRs on reducing cost will be through improvements in the coordination and quality of care, and the prevention of unnecessary and costly complications and hospitalizations. ”

    Zero, Zip, Nada real wold evidence that this assertion has any basis in fact. Costly complications and hospitalizations cannot be reduced to zero, nor is there evidence that coordination of care is improved under the EMR model…in fact, it seems to be getting worse. As for “quality of care” there is an implicit assumption that there is not quality care, therefore there are high health care costs, and by improving quality of care, illnesses will be prevented and save money.

    Nope. It isn’t that simple.

    The promises of the EMR remind me of all the promises made by the central planners about “green energy” and moving from an oil based economy overnight. It isn’t happening, and mounting real world evidence (Chevy volt, wind power, ethanol, solar energy) is proving that to be a pipe dream at best.

  8. Arjen says:

    It is nice that Dr. Mostashari has taken the time to argue what seems like a flimsy conclusion from and even more flimsy study. While I think that Dr. Mostashari would agree that there is still plenty of room for improvement when it comes to EHRs and MU, it is important to remember that we are still in the very early stages of adoption and development. I think that considering how new the EHR movement is the improvements EHRs are allowing in healthcare are impressive. As with anything worth doing perfecting the EHR will not be easy and will take time, but I think it fair to say that we are well on the way. http://www.vitalblog.com

  9. Doctor Doubtndeb says:

    Quality of what I get from the users of EHR’s is dismal. Back with our first implementation to “take us paperless” I had hoped my life would be made better by these programs, 10 years have passed and it still hasn’t happened. Right now my practice is using twice the paper; I get 19 pages from a simple ER visit for a sore throat. Consultant’s letters have bloated up to 4 and 5 pages of the same information over and over. I do find I order more with the order sets putting the options out there in front of me. I’m not sure the payoff with EHR is what the government expects. Time will tell.

  10. Raymond Fink says:

    The National Ambulatory Medical Care Survey is based on physician reports of services provided when the service is provided. If a test is requested anytime (even minutes) after the service is provided it is not recorded on the NAMCS form. It is evident that physicians with electronic access may do this conveniently while others may be delayed in requesting tests. The study cannot measure requests made following the visit, which probably accounts for the vastly greater rate of testing requests made by physicians with IT available.

  11. Jake says:

    To say that, “the federal government’s ongoing, multibillion-dollar effort to promote the adoption of health information technology may not yield anticipated cost savings from reductions in duplicative diagnostic testing. Indeed, it is possible that computerization will drive costs in this area up, not down.” .. is nothing more than a biased speculation. Will it really drive costs up? I think this is highly unlikely, but that’s not even the point. I think most people would agree that the government is doing a good thing for the public by promoting health information technology. It’s called progress people…

  12. EHRs have the potential to save hundreds of billions of dollars by identifying and eliminating unnecessary medical goods and services and assisting physicians with the decision making process. Unfortunately, the EHRs developed by our Medical-Industrial Complex have just taken us from the “Paper Storm” to “Chaos at Light Speed.” As long as program developers turn out “source oriented” EHRs focusing on maximizing provider profit (how else can they sell software?), the longer we shall stay in the dark age. We will not see much progress either in cost reduction or improved quality until our “Chaos at Light Speed” source oriented records evolve into a “Problem Oriented” electronic health record interface with fully integrated (in real time) medical AI decision support tools (check out the IBM Watson ads). Until then, EHRs simply provide chaos at light speed and facilitate the physician’s dysfunctional global memory-based decision-making process. It is possible that EHRs have temporarilary made our inefficient dysfunctional Medical-Industrial Complex more expensive because we have not idenfified and focused on the real problem. The source of the American health care crisis is the dysfunctional global subjective memory-based decision-making process used by our providers. The health care crisis is, “Just what the doctor ordered.” Want to learn more? Read my book, Discovering the Cause and the Cure for America’s Health Care Crisis – A physician’s memoir.”

  13. Emo says:

    Thank goodness we have bureaucrats who are kind enough to get us “the facts” ;)

    Also I think it’s “Dr.” McCormick, and why don’t you link to the article?

    There are no facts. Only arguments. At least in this game.

  14. wine barrel says:

    Thank you for this post. The limitations of this study are significant. It’s clear that the authors should have considered whether CPOE and/or decision support would decrease redundant and unnecessary tests. They acknowledge that other studies of systems with CPOE and decision support have shown a decrease in duplicate testing.

  15. Great post, thanks for sharing!

Leave a ReplyComment Policy


*