How to make sure the output of the AI agent remains high quality

This is a short message based on some comment discussions from my recent LinkedIn post.

The post was about the importance in including maintenance in the AI agent business case and maybe there is something around the monitoring as well.

So if the AI agent fails, the result will be obvious as a process will stop and someone will fix the issue as per how the business has decided to act.

The risk here is if the AI agent is slightly off in its answers. Maybe it mixes two supplier values or it suggests a forecast value that is in range of acceptance, but is in fact not what it is supposed to suggest.

In this case, the agent is still operating but the risk around it is increased or the business value it is generating is reduced.

This type of deviation will happen in some cases as context and memory might get polluted over a longer time period.

I propose a human that verifies the performance of the agent. My immediate thought will be that the performance should be verified a few times per year by a human but obviously this is business and risk related and will depend on the task of the agent.

Another option is to build agents that verifies agents but that means that someone (like a human) still have to verify the ultimate human.

The important thing is that the documentation around the expectation of an agent and how to verify it, is clearly identified. Like that the verification process will go smooth and there can be records of how the agent was tested.