Data-driven decision-making
Data-driven decision-making is an area of research that lies at the intersection of predictive and prescriptive analytics. The recent explosion of data availability and the development of sophisticated and accurate predictive modelling tools have led to an important question: what does one do with good predictions?
The idea of data-driven decision-making is to integrate data models within decision models so as to optimise decision making in the face of uncertainty. Traditionally, this optimisation would involve only the main information necessary for decision making. Optimising a humanitarian supply chain would access demand information to make relief items allocation decisions. Optimising power systems expansion with green generators would take into account the intermittence of wind and solar power.
Over time, it has become evident that side information (also known as covariates) is as crucial in decision-making. The age, vulnerability and affluence of affected people are essential in making equitable humanitarian relief allocation. Other meteorological information matter when deciding the types of power generators to install in order to avoid blackouts.
The field of data-driven decision-making, also known as contextual optimisation, is rapidly expanding and recent research have unlocked the huge potential of the area in not only analytic and algorithmic terms, but also in actual practical impact.
The aim of this project is to bring together academics with research interest in the area and build a collaboration network that draws on each other’s strengths to contribute exciting and state-of-the-art research in data-driven decision-making.
Decision-based divergence for ambiguity characterisation
I visited the Centre for Operational Research, Management Sciences and Information Systems (CORMSIS) at the University of Southampton and the School of Economics in KU Leuven, each for two weeks. I delivered seminars during my stays. These seminars were about the use of divergences to enhance decision-making under ambiguity.
Ambiguity draws on the concept of unknown unknowns, in the sense that the probability distributions that characterise uncertain data are themselves subject to uncertainty. I presented my recent publication in the area and kickstarted research discussions on how to make ambiguity characterisation decision-dependent.
Characterising ambiguity can be a tough task that involves intractable divergence functions. A way to resolve intractability is by fitting simpler functions that closely map the behaviours of complicated divergence functions.
So far, fitting has been purely function-driven, in the sense that the simpler function would only care what the divergence function it is attempting to map looks like. However, perfect fitting is impossible and errors are always expected. In fact, every error is different. A minor error that causes the decision model to assign high probabilities to high-impact rare events can lead to serious over-conservatism in decision-making. We are currently exploring ways to minimize decision-making errors in fitting divergence functions.
Anomaly detection in optimal planning
My last stop was a two-week visit to the Management School in Lancaster University. The school is famous for its research in prescriptive analytics under uncertainty. I delivered a seminar on the use of anomaly detection models within decision models, so as to weed out anomalies during disaster preparedness planning.
Incidents that are one-of-a-kind can offer a distorted view of uncertainty. For instance, if we look at the past 20 years of disaster data in Brazil, the 2012 megadisaster in Rio de Janeiro would be an anomaly. In the same way, Covid-19 would be an anomaly in terms of health disasters. If we were to build planning mechanisms that seek optimality to these anomalies, we would favour overly conservative decisions. For instance, we would pre-stock a lot more relief items that necessary in case another megadisaster happens. We would keep social distancing and huge stockpiles of vaccines in case another Covid-19 happens.
My research with colleagues at Lancaster University involves how to detect and recognise anomalies in decision-making, without swaying optimal planning emphatically towards them.
Conclusions
This network has the potential to make significant impacts in the area of data-driven decision-making. The projects that we have kickstarted are expected to seed new ones and eventually, the ambitious goal would be to create a large network of collaborators working together on the same theme. This can lead to further grant applications and impactful research.
Aakil Caunhye is a Senior Lecturer in Business Analytics.