Dealing with Uncertainty in Process Mining

We encounter lots of different situations that can be filled with uncertainty as we go down the process mining process. There are many new stones to be turned over. In this post, we outline some of the uncertainties that can be faced along the way to process mining and how we've found ourselves addressing them. 

Our process mining data shows us doing things we don't do!

This is a common problem we encounter. Most often, it is caused by a disconnect in the great telephone game being played as we try to decipher the data. We speak with the business, distill what we hear, then ask for that information from the IT department. The best way to resolve this is to focus in the short term on where the data does show the process accurately. Then use the process mining tool to catalog all the wrong paths identified. This becomes a great problem set to work on and determine if it is a miscommunication or actually something that could be an opportunity for improvement.

Our process mining data is missing a lot of information!

When we install an out of the box connector, regardless of the tool, it is usually only 70-80% of the way complete. And often, some of that could be stored differently in an organization's data and needs to be translated. Our approach is to always have an iterative validation process where we attempt to determine where the missing data is located. Sometimes it can be stored in a different column, like a custom column created by the organization. Other times, it has been hidden in previous reports that an organization has created, and the business is unaware that that data is even missing! Its important to be agile during the validation process and keep a detailed log of your discoveries.

Is process mining our data accurate?

This is the toughest question to answer, and always has been for a consultant. Most often in the world of process mining, there is very little data created. However, there are sometimes changes made to the data in the ETL process that could have a poor interaction with the source system. For instance, if the ETL tool was made assuming all regions used unique identifiers when in reality they could have duplicated an identifier, and the organization uses the region and the id as the foreign key. This could break the entire ETL pipeline and has in our experience. It is important for the process mining consultant to analyze the whole pipeline of the data and determine where the accuracy could become fuzzy. 

Please comment any uncertainties you may have experienced in a process mining project with your data!

Sign Up for Tip of the Week