The output storyboard:
From the function & AI CoE teams/ project team, to the executive leadership
"Over the past 5 years, in more than 90% cases, the revenue forecasting and subsequent guidance from our company to the market have been grossly inaccurate across all projection parameters. All our shareholders, investors, employees and clients have expressed their frustrations with these projections. These errors have created cascading problems for these stakeholders, given that their portfolio returns, investment plans and project plans became more uncertain and unpredictable hence unmanageable. Consequently, for the stakeholders that are directly affected by our revenue projections, our NPS has been steadily going down in past 5 years- currently standing only at 30%.
To solve this key strategic issue, our target AI MVS has been to improve accuracy of our revenue projections across all parameters, to at least 95%. This in turn is expected to improve the NPS of all the affected stakeholders groups to at least 60% and above. In our AI-ML CoE we had at least 10 expert-level and 3 master-level ML engineers with relevant experience ranging from 3 to 10+ years in building and deploying time-series based financial modeling and forecasting solutions.
As this initiative was planned based on the requirement identified by our CFO function, we had access to the data and experience of the as-is manual process from that team. We also sought 3 senior-level domain and process talents from our AI tech and service partners, who have successfully developed and deployed similar AI usecases in other organizations. We got past 5 years' financial data and also the projections and the actual parameters and the calculated variances. The data was voluminous and there were challenges in terms of consistency of values. Our ML teams spent 2 months in data collection, initial quality analysis, integration and wrangling- preparing the data for modelling.
Given the volume of data and the complexity of the algorithms used, we had to use our internal cloud and dedicated partner IaaS and PaaS cloud instances for compute, storage and API libraries & containers. Our ML team had to build and try out multiple models, to compare their accuracy and performance, to arrive at the most optimal solution.
But ultimately, this initiative succeeded only because there was a clear mandate and communication directly from our CFO office, both to their own teams/ functions responsible for this process, and the AI CoE and specifically the ML team assembled for the project. Despite all the support, sponsorship and mandate, the erstwhile process owners were initially quite reluctant to give the ML teams access to data at the detailed, internal level.
Even when the final models showed projections accuracy of 90%+ consistently, for 8 quarters of test-data, the team wasn't too confident to utilize it in prod. A key change lever was the CFO's direct involvement in the project reviews in later part. The CFO gave the accuracy improvement mandate directly to the process owners. Post this intervention, the finance teams came in complete sync with the ML team and even contributed further in terms of ideas, in fine-tuning the model and the hyper-parameters.
Since the model has been deployed in prod, it has been generating revenue projections with 92-94% accuracy. The market and key stakeholders have already expressed high satisfaction and credibility of these projections vis-à-vis what used to be reported earlier. We are planning the NPS survey after 2 more quarters of usage of the models, and we expect the NPS scores to cross 80%+, much beyond the targeted 60%, w.r.t. this strategically important activity