Some 25 years ago, as a novice in the world of healthcare quality, I sat in a ballroom with 800 others listening to the sage advice of Dr. W. Edwards Deming at his four-day seminar on quality and productivity. At one point Dr. Deming proclaimed, “Benchmarking will be the downfall of Western civilization.” My brain froze. “Did he say what I thought he said?” I wondered.
It was a throw-away line for Dr. Deming. He was off to another subject but I was stuck on the topic of benchmarking. I muttered, “I thought benchmarking was good. Why does he think benchmarking is bad?” I had been performing benchmarking. I had been advising my clients on benchmarking. What was I to make of this rash statement by such a brilliant thinker?
Now let me just say that I am not, and never have been, a guru purist. I don’t believe in doing things just because Dr. Deming or any other thought leader said to. However, Dr. Deming was a wise man with incredible insight regarding quality and I have always looked to his work and words for inspiration and rejuvenation. He was also known to make extreme statements when he was frustrated with small minds and wrong-headed leadership. “If you can’t do it right just don’t do it at all!” he tended to say in irritation.
So, long ago I began my own soul-searching with regard to benchmarking. Here is what I believe. Benchmarking will not cause the collapse of society. Benchmarking can add value if done well. Benchmarking is often not done well.
Before we go further, let me define 2 basic kinds of benchmarking. Metric benchmarking is looking at others’ performance data and comparing those data to your own. Process benchmarking is looking at others’ processes and systems to see how they do things compared to you.
With those definitions in mind, below are five things I’ve learned over the years about benchmarking.
1. When performing metric benchmarking, it is always important to understand the context of the data. If I’m comparing my performance data with yours, how is your organization different? How is your patient population different? How is your staffing different? It could be argued that no two healthcare organizations are enough alike to be compared, but we might as well let that argument go. You’ll get nowhere with the feds or payors on that one. So, we must compare to the most similar organizations and yet understand the differences between us and them.
2. When performing metric benchmarking, remember that being at the top may just mean you’re the cream of the crap. (More on that subject in this past blog post by Ken Lowe) Being the top of the garbage heap may get you reimbursement advantage and that’s good, but it doesn’t mean that your patients are getting the care they need. Recently I was studying the national HEDIS benchmarks on care of the frail and elderly. The Medicare 90th percentile performance on osteoporosis management in women who have had a fracture is at 29%. That means that to get into the top decile of performance, you only have to give appropriate care to less than 1/3 of the aging women who have had a fracture. Not a lofty goal to which to aspire. More reimbursement? Maybe. Better care? Not so much.
3. Despite item 2 above, metric benchmarking can be a place to begin. It helps to know where you fall in the pack. It helps you determine if you’re delusional about the level of your performance. Just remember that external metric benchmarking is the starting point. The real test is when you benchmark against yourself by watching your performance over time. This is where understanding data over time, understanding variation, and skill with run/control charts is critical.
4. Metric benchmarking with other organizations is most powerful when coupled with process benchmarking. Once I can compare my numbers with yours, I need to uncover how you do things compared to how I do things in order to bring context to the data and to effectively evaluate what might be useful in my own organization.Which leads me to item #5:
5. Looking at how someone else does something and trying to exactly copy it into your own organization is a recipe for disaster. Without understanding how their organization is different and without customizing changes based on those differences, most changes are destined to fail. One size does not fit all. Innovations are not plug and play.
So will benchmarking be the end of the U.S. economy as we know it? I think Dr. Deming was a little off on that, but it does take care and thought to use both metric and process benchmarking in an effective way that truly fuels improvement.
Today’s blog was written by Blair Nickle, senior consulting director at HPP.For more than 25 years, Blair has dedicated her career to the improvement of processes, quality, safety, patient satisfaction, employee engagement, physician satisfaction, and financial vitality in healthcare organizations. Her content areas of expertise include instructional design, performance measurement and improvement methodologies, information systems implementation, strategic planning and deployment, project management, and human resource development.
Blair holds Master of Business Administration and Master of Science in Library and Information Science degrees from the University of Tennessee. Her undergraduate work was performed at Emory & Henry College.