Honest communication vs. the principle agent problem

Recently I read an interesting article by Jesse Smith [1] discussing how the HVAC industry in New Jersey had a poor track record of correctly installing systems. Key for this discussion, they also had a poor record of verifying their installation once finished. Under performing HVAC systems is the natural result with low airflow which has been especially detrimental during the COVID years. Any specialty can suffer from these sorts of issues including my own, advanced analytics.

In economics this is described as the Principal-Agent problem [2]. The root of the problem in the HVAC case is that the people buying (the principal) don’t fully understand what the installer (the agent) is doing and so cannot hold them to account even if they were to inspect the work. There is an incentive for the installer to not look too closely at their own work otherwise they might find errors which will require rework. People working in the data and analytics space need to keep this principle in mind, both in-house analysts and external consultants. Instead of hiding our results, it is imperative that we bring the project sponsors on our journey and let them into as much of the process as practical so that they can have confidence in our work.

As an example, imagine that we’re developing a machine learning (ML) process to detect if bottled wine is oxidised. In the same way we treat the super fine details of the ML program (the neural weights) as a black box that builds to the neural network, we can treat the whole ML process as a black box that builds to results we can communicate. What does this look like? One, make sure the project sponsor understands what input measures you’re considering, what output results you’re producing and how those outputs relate to their goals (in this case, product assurance). Two, demonstrate how the ML program produces results by showing specific examples of bottle assessments. And three, communicate the efficacy of the process: don’t hide the error rate but champion it, show the project sponsor how few errors there are and how much better this new ML process is than what was used previously.

Of course this requires that your project is effective and delivers results with value. In the case that your demonstration shows that your project does not deliver value, you shouldn’t shirk the demonstrations but still communicate what it does deliver. The discussion may be unsavoury but it will lead to a way forward that can actually result in success. On the other hand, if you can’t demonstrate value to the project sponsor and instead hide your results, then you’re as ineffective as those HVAC installers and will never improve.

If you’d like some analytics help from someone that wants to convince, not confuse, get in touch at Hello@NorthCardinal.com.au.

[1] https://asteriskmag.com/issues/05/lies-damned-lies-and-manometer-readings
[2] https://en.wikipedia.org/wiki/Principal%E2%80%93agent_problem