Data Analytics and Internal Auditing DATA ANALYTICS AND INTERNAL AUDIT COVID-19, L6, Aristotle, and the crucial importance of data analytics in planning The White House announced in February 2015 that a gentleman by the name of DJ Patil would be appointed by President Barack Obama as the first US Chief Data Scientist. In DJ Patil’s own words, the mission of the Chief Data Scientist [1] was “…to responsibly unleash the power of data to the benefit of Americans”. Fast forward 5 years to 2020 as the world lurched into the epoch-defining event of COVID-19. Writer Michael Lewis in his most recent book entitled “The Premonition”[2] describes how the emergence of COVID-19 was tracked by a group of US scientists in the early part of 2020 who anticipated the enormity of what was to come. A key event occurs when Governor Gavin Newsom appointed a team to develop a strategy to shape the Californian response to COVID-19. Included in the team was none other than the former US Chief Data Scientist DJ Patil. As the team scrambled to get their bearings, DJ Patil, along with the Chief Economic Advisor to Governor Newsom, came upon a Santa Barbara doctor and epidemiologist, Charity Dean, who appeared to hold all the right data. DJ Patil and the economic advisor met with her, heard what she had to say, and declared that she was “the L6”. She held the right data that could launch a strategic response to COVID-19 for the State of California. Lewis explains that the term “L6” refers to Layer 6. You need to go six layers down in an organisation to find the right person with the right data to understand what is really going on. While this concept will resonate with root cause analysis enthusiasts in the internal audit profession (“ask why 5 times” etc.), it is worthwhile to reflect on the event that Michael Lewis so well describes, and, the broader learnings that the internal audit profession can derive from COVID-19 in the context of data analytics. COVID-19 has, after all, made us a planet of epidemiologists. Through necessity, in late 2019 and early 2020, humanity needed to understand what was going on and to do so at speed. R.0 rates, case numbers, mortalities, surge capacities and variants became core data inputs that shaped our day-to-day lives. Overnight, we all became data analysts on COVID-19 matters. Overnight, we all became our very own L6. Data Analytics Data analytics, as a concept, should not be something new to an internal auditor in 2021. Data (etymology: Latin, datum, i.e. what is given) and analysis (etymology: Greek: analuein, i.e. loosen up) ought to be at the heart of what we do. We loosen up what is given. The mission of internal audit is “to enhance and protect organisational value by providing risk-based and objective assurance, advice and insight”[3] . And in line with Performance Standard 2310, internal auditors “must identify sufficient, reliable, relevant and useful information to achieve the engagement’s objectives”[4] . In this regard, we analyse data. We form fact-based conclusions. We do this because our international standards demand that we do this. We do this as a matter of course. But stepping back, this is not really something unique to internal auditors. Aristotle said “each man judges well the things he knows”[5] . Human beings are constantly observing data and making judgements in all facets of life[6] . But as COVID-19 has shown, an area of data that was not in the mainstream of human attention (virology), suddenly became the single most important type of data to know. Up to 2019, virology data simply was not mainstream. And this is at the core of what we can learn from COVID-19 as internal auditors. Aristotle said that we judge well the things we know. But if you invert the Aristotle quote, the corollary is that we are not able to judge the things we do not know. And in this author’s opinion, this is the key learning on data analytics from COVID-19. The things we do not know... Use of Data Analytics in Internal Auditing The use of data analytics in internal auditing is well versed. The Institute of Internal Auditors and the Chartered Institute of Internal Auditors have both written extensively on the use of data analytics in internal auditing7. The use of data analytics to provide greater levels of assurances through whole-of-population testing and continuous auditing is not in dispute. In a world of greater levels of data, and more sophisticated tools to analyse that data, internal audit undoubtedly can spot more. But to introduce Aristotle (yet again), this enables us as a profession to better judge the things we know. If we are auditing payroll, we can review more and more (if not all) transactions. But how do we use data to know if we should be looking at payroll in the first place? This is the more unnerving aspect of being an internal auditor. We cannot judge what we do not know. What is the extent (and risk profile) of what we do not know? Another central character in the Michael Lewis Book was Carter Mecher, a US doctor who was key to the early data analysis as the facts around Wuhan emerged. As Lewis explains, Carter Mecher had “no formal training in epidemiology or virology or any other relevant field. He simply had a nose for data”. But Carter’s nose drew on data from the 1918 Influenza, SARS and H1N1 (Swine Flu), and he developed a model of what he felt the relative spread of COVID-19 cases could be in the USA, in the case of a full-blown pandemic. Weeks later, when the Diamond Princess cruise-liner was quarantined in Tokyo, real numbers emerged of the speed of the spread of the disease in a confined space. The worst fears of Carter Mecher were confirmed. His mental model was proven. Lewis writes that Carter Mecher said “I can’t understand how nobody is paying attention to this”. Soon, Carter Mecher would become a major actor in the US response to COVID-19. But the point remains as Mecher presents, we cannot judge what we do not know. And this draws in perspective, the internal audit planning process, and the concept of radical uncertainty. How do we assess that which we do not know? Planning and radical uncertainty Performance Standard 2010[8] requires that internal auditors establish a risk-based plan to determine the priorities of the internal audit activity, consistent with the organisation's goals. In so doing, internal audit should consult with senior management and the board and obtain an understanding of the organisation’s strategies, key business objectives, associated risks and risk management processes. In this manner, at its core, internal audit is risk-based. It dances to the tune of risk, whatever those risks may be. And in this context, COVID-19 has shone a bright light on the brittleness of our understanding of the contours of risk. Few internal auditors can say they saw this coming. Risk played a tune we did not expect, and internal audit had to dance to this new tune. The former Governor of the Bank of England, Lord Mervyn King, has written extensively on risk and, more generally, what he characterises as “radical uncertainty”. Lord King notes that9 “…radical uncertainty cannot be described in the probabilistic terms applicable to a game of chance. It is not just that we do not know what will happen. We often do not even know the kinds of things that might happen”. In this regard, the question then becomes this: how do we as internal auditors develop risk based plans, when we cannot fully appreciate the contours of the very risks that shape the internal audit plan? What data do we analyse? Datum analuein. The simple learning we can take away from Lord King is that no amount of data analytics can trump radical uncertainty. There are things we simply cannot be expected to know. And internal auditors can surely be forgiven for not knowing what we cannot reasonably know. But, internal auditors are less likely to be forgiven for not adapting to the emergence of a new risk, once it has emerged. The key question is what do we do when a hitherto unknown, becomes known? And this, in this author’s opinion, is where data analytics becomes crucially important. COVID-19, and more generally the prospect of a global pandemic, was not centre stage in internal audit plans prior to 2020. Then suddenly, a red light began to flicker, then pulse, then beam brightly in every risk register across all industries, across all service lines and across all borders. While initially the focus of internal audit was on service continuity (i.e. can we/ how can we audit remotely?), the next step was for internal audit to reassess the plan itself. One could even position this as morphing into a more existential relevance selfassessment: how can internal audit fulfil its mission to enhance and protect organisational value by providing risk-based and objective assurance, advice and insight, in a pandemic? And this required speed and dexterity in analysing data. Operating environments had transformed for all organisations. And in this, our age of big data, big data began to flow, and flow at high speed. Governments released COVID-19 response plans. A vast array of social and economic interventions were announced. Central Banks and economic think tanks issued economic analyses. State agencies and supranational bodies released policy positions and guidance. Each man judges well the things he knows. Now, as a profession, we were facing vast amounts of new data, all of which did, could, or would alter our audit universe. The audit universe had changed. All changed, changed utterly. To play on etymology, we had new data and this needed to be unloosened. The true measure of an internal audit function then became how proficiently this was done. To illustrate by example, this could have been done: • By analysing national economic projections, what does COVID-19 do to demand/ activity levels in my organisation? How are forecast and business plans being reshaped? Possible audit themes: business planning, forecasting, strategic modelling. • By analysing national COVID-19 response plans, is my organisation eligible for economic supports/ grants/ subvention? How are controls being redesigned to deal with this? Possible audit themes: COVID-19 grant claims, grant accounting, grant reconciliations. • By studying public health policies, what responsibilities rest on my organisation? How are processes being designed to ensure compliance, and how can we get comfortable that they are operating effectively? Possible audit themes: public health policy process design, public health policy process compliance. • By analysing reports from regulatory bodies, is COVID-19 leading to changes in other preexisting risk profiles: e.g. increased exposure to cyber-crime, upticks in financial crime? Does internal audit need to heighten testing in these key areas? None of these assessments were made in a data vacuum. They were informed by data. And they were informed by data flows that were new and data flows that were travelling (and continue to travel) at high velocity. But intercepting this data and distilling it into relevant audit themes was quintessentially important. It enhanced relevance. It positioned internal audit to better deliver on its mission. And this is how data analytics can truly transform the relevance and efficacy of internal audit. Look beyond data analytics as being a mechanism to enhance testing and increase population size. Look at data analytics as being the key to defining the very areas that should be tested. Look at data analytics as informing our plan. Look at data analytics as informing our internal audit strategy. Look at data analytics as defining internal audit itself. While internal auditors must get comfortable with the inherent discomfort of radical uncertainty, once risks become visible, using the right data can reshape internal audit plans to add value and offer enhanced assurance and insights. When DJ Patil met Charity Dean, he knew he had found the right data to inform decision making. He found his L6. As internal auditors in a world of radical uncertainty, we need to build our capabilities to find the right data to inform internal audit plan plans. We need our own L6. This is our challenge. Each man judges well the things he knows. It all falls back to Aristotle, the philosopher who inspired the Renaissance. Brian Hayes | Partner | Cork Brian.Hayes@mooreireland.ie +353 21 427 5176 With assistance from Caoimhe Doyle & Lauren Moore of Dublin City University on an intern programme with Moore. References A Memo to the American People from U.S. Chief Data Scientist, Dr DJ Patil, (February 2015) The Premonition – A Pandemic Story, Michael Lewis, Penguin Books (2021) International Professional Practice Framework (“IPPF”) , Institute of Internal Auditors Ibid Aristotle, The Nicomachean Ethics (c.340 BC) For more on the importance of judgement, see “The Value of Big Data Isn’t The Data”, Harvard Business Review (2013). Kristian Hammond notes: “If we’re going to really capitalize on big data, we need to get human insight at machine scale”. See Data Analytics - Is it time to take the first step?, Chartered Institute of Internal Auditors, (2017), and, Data Analytics Mandate, Institute of Internal Auditors (2019) International Professional Practice Framework (“IPPF”) , Institute of Internal Auditors 9 Lord Mervyn King and John Kay, Radical Uncertainty: Decision-making for an unknowable future (2020)