Expert Conversation About How AI, ML and Business Rules Complement Each Other Perfectly

Artificial intelligence is finding its way into many areas of business. This applies in particular wherever already automated business decisions utilize a set of rules (“business rules”). In its report, “How to Use Machine Learning, Business Rules and Optimization in Decision Management“, world-leading IT consultancy Gartner makes it clear that it can be difficult for companies to identify the best strategic path when choosing the most appropriate technology.

In ACTICO, companies have a partner that has been using AI techniques such as machine learning in automating business decisions for many years, and one that has a deep understanding of the practical considerations that Gartner highlights for a larger audience in its report. We had the opportunity to interview Thomas Cotic, Co-founder and Chairman of ACTICO’s Advisory Board, about the use of AI and machine learning in Decision Management Systems, and the theses outlined in the Gartner report.

Gartner talks about the automation of business decisions. Where are these automations used primarily?

At ACTICO we like to summarize this under the umbrella term, “intelligent automation”. This is used, for example, when granting a loan or recommending a product within the financial sector. But there are also trading companies that use such systems to control the flow of goods.

Take inventory management in a supermarket for example. The shelves should always be well stocked, but there should not be too many goods in the warehouse (which could spoil). Nowadays you no longer order manually, but make the amount you order dependent on seasonal conditions, current stock and the next delivery dates. Predictive maintenance approaches can also be integrated to help control the supply chain.

But you also need historical data to build on for that, right?

Exactly. There is timing and material category information available for this which is very suitable for exerting control. Other customers also use these kind of systems to ensure optimum operation of their production machinery. Instead of waiting for the machines to fail, they use certain characteristics and metrics to predict when maintenance is needed.

Aren’t such systems very complex to use?

That’s why the focus in these cases is on automating complex decision-making logic. These pieces of logic are traditionally mapped in a programming language. Often you want to adapt this logic to take account of newer knowledge, and that request comes from a non-programmer.

That’s why we follow a graphic modeling approach, which also includes ​​machine learning. We try to simplify it so that it isn’t just usable by the data scientist, but can also be implemented by a business analyst. These are often technically savvy employees who have no programming background. If these people are good at using Excel, they can work with our system too.

How efficient is a DMS when a lot of rules have to be applied?

Regardless of whether we are talking about purely rule-based or machine learning systems, the graphical modeling generates Java code that is highly efficient. The use of Decision Management Systems makes sense wherever there is a large volume of rules that are also complex and need to be adapted frequently.

Do these systems also make decisions independently, or are they intended to play more of a support role?

The goal is to achieve a high degree of automation. Sometimes this is not possible because an approval process is still necessary, or because information is still missing. Then the business user can make the final decision within the application. But that should be the exception. The goal is to make the automation as complete as possible.

Isn’t a real person confirming the decision again at the end more of an advantage than a hindrance?

In the modern Internet era, this is often no longer possible: If you go to a bank website today and want to get approval for a loan immediately (for example to finance a television or a car), you will receive an answer immediately as to whether you qualify or not. You just fill in the required information and expect an immediate decision. The crux of the matter here (in contrast to business process management systems) is being able to make valid decisions in real time, even with high-volume queries with large amounts of data.

Of course, the automation here provides enormous cost saving potential. However, it requires a high degree of digitization within the company to be able to access all the relevant information automatically – which is in itself a driving force for digitization within companies.

What knowledge do you need to introduce a DMS? Is the data sufficient or do you also need specific expertise?

Both: On the one hand, you have expert knowledge that you really need and that sits in the minds of the experts. To access it, we use our graphic modeling approach to transfer the knowledge into the system. On the other hand, we also have knowledge and decision-making logic in the form of data. This can be extracted using machine learning.

However, both approaches can also be used in combination. Let’s take an example from the financial sector: If you have a customer who has an affinity for certain financial products based on the available data, it is far from clear whether the financial institution is allowed to offer the customer these products based on this knowledge (if, for example, the bank knows that the customer only wants to invest his money with low risk). In the US, this consideration is known as Fiduciary Duty and is a legal requirement. This is based on rule-based knowledge that, for example, a bank advisor has. Since it’s based on clear rules, data-centric learning is not appropriate here.

As a result, the compliance aspects are more likely to be mapped using rules. To find out which products are suitable for a customer, one tends to fall back on data that indicates what other similar customers have bought. There are quite a few areas in which you would want to combine data-driven and rule-based knowledge. And it’s easy to do that using our platform. But you definitely need the expert knowledge, which we can map in rules, in order to make effective decisions.

Aren’t the experts a bit like turkeys voting for Thanksgiving when they transfer their knowledge to the DMS?

Of course, this also means that the experts’s roles change. Instead of always making the decisions themselves, they are more concerned with maintaining and optimizing the rules in the system. But the company also benefits from the system’s ability to process larger volumes more quickly. After all, these experts are a scarce resource that are difficult to recruit, which in turn often slows growth.

How do you ensure that the models learn new patterns when new data becomes available, especially in the area of ​​machine learning?

The corona virus pandemic provides a good example of this. Because it is very dynamic, we have no historical data and new data is constantly flowing into the data pool. Many machine learning applications can fall back on years of data that does not exist in this case.

That’s why we are pursuing an approach that automatically adapts to new data. The key words here are Machine Learning Lifecycle Automation and Continuous Intelligence. In contrast to data scientists, who generate their own models from the data and who rarely find their way from analysis to production, we can use ML to automatically create models that we integrate into decision-making.

And by Continuous Intelligence we mean the ability of the models to automatically adapt to the new data, the so-called “drift”. Usually, the ML model is based on the data with which it was originally trained. If new data are now available, the ML model must be re-trained. But not every single new pattern has statistical relevance. So it’s about recognizing the drift of a model. When the model’s predictions slowly get worse, the art is in finding the right time to automatically retrain such systems. This automation is part of the ML lifecycle and works without the intervention of a data scientist.

Are systems also used that are based on dynamic models with current data, or are the ML systems usually trained beforehand?

Currently it is often the case that an ML model is created, but retraining is usually not carried out. In our experience, aging effects can already be seen after a few months if the models are not retrained. If you then hand this over to a data scientist who creates a new model and gives it to IT, it takes too long. By the time a new model is used, more recent data will already be available. If you automate this whole process, you can react much more quickly to changes in market conditions.

Do you only transfer the data from other systems, or can you also connect external ML systems?

We can of course connect external systems, either at the data level or via programs in Python or R. We are very flexible in this regard.

Isn’t there a risk that such a system will become independent if the learning itself is also automated?

Yes, but that can be controlled. For example, you can monitor the results from machine learning using rules created for this purpose. For example, it makes sense to specify value ranges within which the results of the AI ​​must act. If they are outside of the range, the system can intervene in a rule-based manner and, for example, override it with expert knowledge or indicate that a new training session has to be initiated. This behavior is part of AI governance, in which rule-based knowledge is used to monitor machine learning.

Can the benefits be expressed in numbers? If so, what would they typically look like?

In the fraud environment, our customers have 40% to 80% fewer cases that require manual investigation. You can also use our systems in marketing to determine which campaigns are activated and rolled out on websites, and when. The aim here is to increase effectiveness and achieve the maximum benefit within a fixed budget. A downstream analysis showed that the ML system was 65% more effective than the traditional marketing approach.

How is people’s work affected?

When you start with machine learning and use large amounts of data with it, the question arises relatively quickly: “Why was this result produced now?” And that’s a question that isn’t that easy to answer. With statistical analysis you can filter out patterns based on a maximum of 4 to 5 factors. But our machine learning models often take 50 to 100 influencing factors into account.

This is where explainability comes into play. If you vary the influencing factors automatically, you can determine how changes in the initial data affect the result. If you then add one or two rules that are based on expert knowledge, completely new constellations arise. So in a sense, humans learn from the machine. This is also because the ML system finds patterns that no one has seen before.

In one case within context of fraud detection, we compared a large number of decisions made by experts with the results of the ML system. In many cases, both agreed, but in a number of cases, they didn’t. Who is right in that scenario? We were able to look at the decisions in more detail through supervision and play back the findings. This generated new insights for the experts, and we were able to adapt the machine learning model and improve the decision-making rate.

Where do you see the challenges in implementing automated decision-making systems?

Many customers are now working intensively with machine learning, but there is also a lot of resentment and ultimately it has rarely been used productively. The road to a productive system is often still a rocky one.

Why is that?

There are often a lack of mechanisms and processes for deployment and adjustments. You also have to take into account the aging of the models used, i.e. retrain and ensure that they are transferred back to the operational system. Gartner talks about MLOps based on DevOps. The separation between the business units that develop models and the IT department that looks after deployment does not work. The job description of MLOps, with responsibility for everything from development to production, rarely exists in reality. But it is necessary.

MLOps is the key to ensuring that ML-based decisions can also be sustainably adopted in production systems, and that an automated lifecycle can be implemented.

Don’t companies have an urgent need to automate decisions?

Many large companies today have a large number of databases and even more applications that access the data within them. The ‘intelligence’ of the applications for lending or fraud detection is in the Java code. These systems often make very similar decisions. Also, whenever laws change, you have to adapt all the systems accordingly. It would be better to only have to do this in one place.

At the moment, the ‘intelligence’ of these systems is scattered between them and is therefore not reusable. This generates very high costs, especially when it comes to maintenance. Companies should therefore think about whether it would be better to implement a central source of authority for intelligence. In this context, we are talking about a Central Decision Engine that all other systems can access.

This is also important for compliance: You have to give a customer the right information, regardless of which system he uses to contact you (phone, email, website, app, on site). And that process must be flexible and rapidly adjustable, in order to be able to react to current developments such as the Corona virus pandemic. As it stands, many companies have to wait for the next release cycle.