Medical Device Root Cause Analysis: Assembling Your Team and Coming Up with a Plan
Now that you have convinced management that they should dedicate resources to solving a specific problem, it’s time to get started on solving it.
Woohoo, let’s go!
Not so fast…you’ll probably need some help.
First, it is important to recognize that problem solving is a project – that is, a temporary endeavor to accomplish a given goal. Second, finding the root cause of a problem typically requires a team. For example, a quality engineer assigned to lead the team may need assistance from Manufacturing and Engineering staff members, who usually are permanent team members. That said, permanent team members may need the assistance of others as the project evolves. An example would be support from statisticians related to applying statistical tools. This is where your sales skills come in. You need to convince management that there is a need for a team and supporting personnel. This will ensure the availability of resources to move the project along with a sense of urgency.
Very few problems are cast in stone from the beginning. Most projects evolve. During each phase of the problem-solving process you’ll learn more about what is really going on. With this in mind, you need to be open to revisiting the scope, definition, and purpose of the project. It is very tempting to want to jump directly into action. After all, you are paid to get results, not hold endless meetings to discuss taking action!
You should recognize that our innate bias toward action has pitfalls. Foremost among them is the fact that every specialist on your team will see a problem through his/her own lens. Regardless of whether your team will be around for 3 months or 3+ years, make sure it is cross-functional. You will need and want different points of view. Your team will likely have some strong personalities who will pontificate about probable causes with confidence, sometimes before you even have any data. As the leader, it is important that you listen and frame these theories as possible causes, not probable causes. Your ultimate goal is to refine and focus the problem, collect facts, develop probable causes, verify causes against the facts, and then divide and conquer.
What data will you need?
With your team assembled, one of the first things you must define is what data you need to gather, and how you are going to capture and evaluate it. There may be some instances in which you will use historical data, and other instances in which you will need to collect data. Do not collect data under the false assumption that “more is better.” In problem solving, we sometimes grab all the data we can find in the hope that it will somehow lead to enlightenment. But if data is not collected specifically to resolve a problem, it simply wastes time and money. Ask your team which questions need answers and what data will provide those answers. The primary purpose of the data is to localize the problem. At some point you will need to decide whether the cost of obtaining more data and refining the focus is worth the investment.
It is important that before you look for data needed to address the problem, you write a specific, concrete, and measurable operational definition. You want to:
- Determine the purpose of the measure
- Identify the work object (item, event, behavior) to be measured
- Identify the characteristic to be measured and define it in operational language
- Locate the process point at which the data is available
- Plan the sampling and analysis
- Implement the measuring system and refine it
Probable causes are only as valid as the data that supports them. The data you use provides an agreed-on basis for decisions. The data should be understood by all who use it and cannot be “gamed” to support a specific agenda. It is important to objectively verify your sources of data. Here’s an example illustrating why.
Management wanted to improve the testing cycle times at its five test labs. The goal was to turn around 80% of all test results within 48 hours. After a few months, the results came in.
Wow, at first glance the data would seem to indicate that Lab B is not meeting expectations, right? Not so fast. It turns out that Lab B calculates their turnaround time based on the actual time samples arrive at their facility, even though samples always arrive late in the afternoon. The other four labs start the clock the next morning, when they actually start processing the samples – on average, nearly 16 hours later than Lab B. After Lab B applied the same rules being used by other labs, their performance jumped to 77%.
When evaluating data, embrace the Russian proverb:
“Trust but verify.”
Coming up with your sampling and analysis plan
Now that you have defined what data you require, you need to determine how much to collect, as well as how often and by what means it will be collected. If you are using historical data, what period should you consider? The last 3 months? What data collection sheets will be needed? What other data will be required to enable analysis or diagnosis? You have to determine both your method of analysis and what statistical tests will be performed. For example, how will you plot the data for analysis? Quite often, looking at a checking sheet or spreadsheet will make the analysis difficult. Graphs such as histograms or time plots illuminate the data, thus making the analysis more effective and efficient.
Also, recognize that there will be inconsistencies in how people describe the same defect. For instance, “clear bag” and “unprinted bag” might be terms used by different inspectors to mean the same defect. This can create confusion in tabulations. In general, you want to collect data at the earliest point in the process at which it becomes available. Break down the data into useful categories using data stratification techniques (e.g., what, who, where, when). Never use data simply because it is available.
Beware of unconscious bias, which includes “judgment samples” (you selectively pick the units to be included) or “convenience samples” (typically the most accessible and convenient units for you to gather). An example of this would be the most recent batch of components received from a supplier, which may or may not be representative of the larger pool of components. With regard to quantity, oversampling can waste a lot of time with very little additional accuracy. Undersampling can lead to incorrect conclusions. In deciding how many to select, one quick option is to use an online statistical sampling calculator. This will tell you how many samples you should select to achieve an acceptable degree of accuracy.
Want to learn more?
Continue reading the last blog post in this series. There is still plenty to learn about medical device root cause analysis. If you enjoyed this article and want to take the next step in advancing your knowledge, consider Oriel STAT A MATRIX’s training class on root cause analysis.