August 2021

New Technology: The Projected Total Economic Impact™ Of Microsoft Azure Machine Learning

Cost Savings And Business Benefits Enabled By Azure Machine Learning

Microsoft logo

A Forrester Total Economic Impact™ Study Commissioned By Microsoft, August 2021

Table Of Contents

Enterprises are implementing a new generation of machine learning operations (MLOps) platforms that help them democratize and operationalize AI, allowing them to accelerate and govern the end-to-end machine learning (ML) model lifecycle. This drives faster development and deployment of more ML models, more accurate models, and lower business risk. The result is more AI-driven applications, more AI-driven decisions, and the ability to unlock the value of AI in new parts of the enterprise.

Microsoft Azure Machine Learning (ML) is a cloudbased MLOps platform that helps enterprises accelerate the development, operationalization, and governance of AI models. It enables data scientists, data engineers, analysts, and other users to complete each stage of the ML lifecycle from accessing the data, tools, and infrastructure they need to developing models through deployment, monitoring, and retraining. It also provides the data and model lineage capabilities, as well as the enterprise-grade security features, necessary to reduce risk and ensure compliance.

Microsoft commissioned Forrester Consulting to conduct a Total Economic ImpactTM (TEI) study and examine the potential return on investment (ROI) enterprises may realize by deploying Azure Machine Learning.1 The purpose of this study is to provide readers with a framework to evaluate the potential financial impact of Azure ML on their organizations.1

To better understand the benefits, costs, and risks associated with this investment, Forrester interviewed five customers with experience using Azure ML and surveyed 199 data science, ML, or AI decision-makers. For the purposes of this study, Forrester aggregated the experiences of the interviewed customers and combined the results into a single composite organization.

Prior to using Azure ML, the customers either had little-to-no existing machine learning capabilities, leveraged third party and open-source applications, or had disconnected and incomplete legacy, on-premises ML tools. Data scientists and other users struggled to get infrastructure and environment provisioned to undertake AI projects and encountered, at best, delays getting their models into production or, at worst, the inability to deploy their models at all. When models were deployed, they struggled to monitor and retrain the models effectively resulting in wasted manual effort, poorly performing models, and hidden risk. The net result was delayed innovation inefficiency and lower business impact.

After the investment in Azure ML, the customers experienced numerous improvements in their ability to develop new ML projects and orchestrate the pipelines necessary to deploy them, leading to greater revenue and lower operating costs. They noted greater productivity across a wide range of users, faster time to value from faster onboarding, and lower costs thanks to their ability to move off legacy solutions. Less tangibly, the investment helped them innovate, accelerating the democratization of AI and access to the latest ML innovations with better security and governance.

Consulting Team:
  • Luca Son, Julia Fadzeyeva

Key Statistics

  • Project return on investment (ROI)
    189% to 335%
  • Projected net present value (NPV)
    $2.3M to $4.0M

Key Findings

Quantified projected benefits. Risk-adjusted present value (PV) quantified benefits include:

  • Improved data scientist productivity by up to 25% and data engineering productivity by up to 40%.

    Microsoft Azure Machine Learning and its integration with the Azure data stack facilitated faster provisioning of infrastructure. It also provided broader access to ML capabilities and frameworks, such as centralized model registries, along with access to hyperparameter tuning and modular model training and deployment pipelines. By consolidating on the same platform, data scientists and data engineers collaborated more effectively leading to greater productivity. Azure ML sped up each step in the AI lifecycle from data access and engineering to model training, validation, and deployment, as well as ongoing processes of model monitoring and retraining. This saved the organizations between $561K to $866K in data scientist and data engineering productivity over three years.

  • Up to $2.5M in increased revenue and cost savings from better machine learning insights.

    Azure ML enabled the deployment of a higher quality and volume of ML models, helping organizations improve business outcomes. The solution also kept these models accurate over time, thanks to better model monitoring and more frequent retraining. This led to increased revenue generating and cost saving ML applications. The revenue uplift and operating costs savings is worth $1.6M to $2.5M over three years.

  • Up to $1,703,982 reduced cost of retiring legacy solution in three years.

    By adopting Azure ML, interviewees’ organizations decommissioned legacy on-premises solutions, reducing expensive legacy licensing, development, administration, and support costs.

  • Reduction in time to onboard new data scientists by 55%.

    Onboarding data scientists on legacy environments was complex and slow. Data scientists were often unable to use their preferred ML tools and frameworks and required to learn numerous disjointed solutions and processes. With Azure ML, environments for new data scientists and projects could be provisioned rapidly and, thanks to familiar, centralized tools, data scientists got up to speed faster. The reduced training time saved $93K to $186K over three years.

Unquantified benefits. Benefits that are not quantified for this study include:

  • Enabled employees outside of data scientist role to take full advantage of machine-learning capabilities.
  • Increased business-user and SME productivity.
  • Improved interoperability and extensibility of machine-learning platforms and open-source ML frameworks.
  • Improved responsible ML and model compliance, governance, and security.

Costs. Risk-adjusted PV costs include:

  • Azure Machine Learning and supporting solutions costs totaling $320,397.

    Microsoft Azure ML pricing is based on machine-learning computation and data-storage consumption. Costs vary depending on each organization’s characteristics and needs. There are no additional charges to use Azure ML. Forrester captures costs based on the composite organization characteristics, as well as costs of supporting applications used in conjunction with Azure ML. For further pricing information, please speak with a sales specialist.

  • Implementation and ongoing costs of $888,341.

    Implementation includes an initial professional services fee for change management, integration, and training. Ongoing maintenance costs include ML engineer FTE labor.

Forrester modeled a range of projected low-, medium-, and high-impact outcomes based on evaluated risk. This financial analysis projects that the composite organization accrues the following three-year net present value (NPV) for each scenario by enabling Microsoft Azure ML:

  • Projected high impact of a $4,043,762 NPV and projected ROI of 335%.
  • Projected medium impact of a $3,162,349 NPV and projected ROI of 262%.
  • Projected low impact of a $2,280,936 NPV and projected ROI of 189%.
Figure 1
"Which of the following benefits has your organization experienced/do you expect to experience due to the investment in your ML solution?"
figure

Base: 199 data science, ML, or AI decision-makers
Source: A commissioned study conducted by Forrester Consulting on behalf of Microsoft, April 2021

New Tech TEI Framework And Methodology

From the information provided in the interviews, Forrester constructed a New Technology: Projected Total Economic ImpactTM (New Tech TEI) framework for those organizations considering an investment in Azure ML.

The objective of the framework is to identify the potential cost, benefit, flexibility, and risk factors that affect the investment decision. Forrester took a multistep approach to evaluate the projected impact that Azure ML can have on an organization.

  • DUE DILIGENCE

    Interviewed Microsoft stakeholders and Forrester analysts to gather data relative to the Azure Machine Learning.

  • EARLY-IMPLEMENTATION CUSTOMER INTERVIEWS AND SURVEY

    Interviewed eight decision-makers at five organizations using the Azure Machine Learning in a pilot or beta stage and surveyed 199 data science, ML, and AI decision-makers to obtain data with respect to projected costs, benefits, and risks.

  • COMPOSITE ORGANIZATION

    Designed a composite organization based on characteristics of the interviewed and surveyed organizations.

  • PROJECTED FINANCIAL MODEL FRAMEWORK

    Constructed a projected financial model representative of the interviews and survey using the New Tech TEI methodology and risk-adjusted the financial model based on issues and concerns of the interviewed organizations.

  • CASE STUDY

    Employed four fundamental elements of New Tech TEI in modeling the investment’s potential impact: benefits, costs, flexibility, and risks. Given the increasing sophistication of ROI analyses related to IT investments, Forrester’s TEI methodology provides a complete picture of the total economic impact of purchase decisions. Please see Appendix A for additional information on the TEI methodology.

DISCLOSURES

Readers should be aware of the following:

This study is commissioned by Microsoft and delivered by Forrester Consulting. It is not meant to be used as a competitive analysis.

Forrester makes no assumptions as to the potential ROI that other organizations will receive. Forrester strongly advises that readers use their own estimates within the framework provided in the study to determine the appropriateness of an investment in the Azure Machine Learning.

Microsoft reviewed and provided feedback to Forrester, but Forrester maintains editorial control over the study and its findings and does not accept changes to the study that contradict Forrester’s findings or obscure the meaning of the study.

Microsoft provided the customer names for the interviews but did not participate in the interviews.

MLOPS OVERVIEW: EVERY ENTERPRISE NEEDS AN MLOPS PLATFORM TO OPERATIONALIZE AI AT SCALE

Transforming your organization with AI isn’t about developing the optimal machine-learning model — it is about developing and operationalizing dozens, hundreds, or even thousands of ML models and continuously retraining them to keep them optimal. This digital transformation is about seamlessly orchestrating the infrastructure needed to train and retrain these models and serving instantaneous predictions using real-time data to applications across your business. AI also allows your data scientists and citizen data scientists to leverage the latest, cutting edge ML innovations and the model development tools of their choice — all with enterprise grade security. Finally, this technology manages governed processes that tirelessly monitor your models for drift, track lineage, and provide the transparency necessary to manage risk and satisfy regulators.

Building these capabilities, collectively known as MLOps (the AI equivalent of DevOps), is a top priority for most organizations that have started on their AI journey. ML engineers are highly sought after, but don’t scale. Even the most advanced AI organizations are investing in MLOps platforms that provide integrated capabilities to accelerate the AI lifecycle and use automation to scale the organization’s ability to apply AI, unlock its business impact, and govern its growth. For all organizations that use or plan to use AI benefit from an MLOps platform, the type of benefits they can expect depend on their level of AI maturity:

  • MLOps platforms give novice organizations an AI best-practice boost.

    When just starting, organizations reinvent the wheel with each AI project, reducing the chances of success and leading to a proliferation of separate, inefficient processes and disconnected tools that will hamper collaboration and efficiency efforts down the line. Since MLOps platforms encode AI process best practices, organizations can leapfrog directly to standardized processes honed for shepherding AI projects across the lifecycle and an architecture of integrated tools to support it.

  • Developing AI organizations scale successfully with MLOps platforms.

    Developing AI organizations have had several AI successes but each has required superhuman effort. New projects can’t get off the ground because they cannot get the infrastructure and environments they need, and operationalizing projects takes months as each model must be refactored from scratch to run on production systems. MLOps platforms, especially scalable cloud platforms, solve the bottlenecks. These platforms automate the orchestration of infrastructure, the provisioning of environments, and the deployment of ML pipelines, getting models into production in days or even minutes instead of months.

  • Advanced AI organizations unlock the transformative AI with MLOps platforms.

    These organizations have built their own MLOps solutions to support their multitude of AI projects, but these homegrown solutions hold them back. They cannot incorporate the latest developments in AI tools and methods fast enough, preventing their data scientists from using the tools to tackle new use cases and increasing productivity. Furthermore, the development effort and security risks proliferate with every new addition to their AI ecosystem, while hidden governance challenges increasingly worry leadership. MLOps platforms overcome the challenge of updating and incorporating new AI innovations by spreading the development costs across organizations. Combined with their lineage, model monitoring, and security capabilities, the organization can now transform a greater share of the business, driving competitive advantage against their stymied rivals.

NEXT SECTION: The Microsoft Dynamics Azure Machine Learning Customer Journey

Key Challenges

Interviewed and surveyed organizations (see Appendix B) came from a variety of previous environments from lacking any machine-learning capabilities and tooling to utilizing static analysis software and point solutions to using homegrown, on-premises solutions. Several organizations utilized consulting services to provide prebuilt models.

The interviewees’ organizations and surveyed respondents struggled with common challenges, including:

  • Scarce and siloed tools.

    Organizations either lacked the tooling to unlock the potential of their data and knowledge or were roadblocked by the complexities of multiple, disjointed tooling. The IT director of data analytics in a manufacturing organization stated: “We have a very capable workforce who understands the value of advanced analytics in our business. What was holding us back from doing more analytics was the access to tools to do it.”

  • Poor collaboration among data scientists and engineers lead to diminished model quality.

    Collaboration amongst data scientists and between data science and data engineering teams suffered from disjointed tooling and workflows. Data scientists did not have visibility, control, and could not collaborate effectively, especially in cases where organizations were leveraging prebuilt models from consulting services. This led to diminished model quality and left insights on the table.

  • Labor-intensive data collection, cleaning, and categorization.

    Without a solution that integrates siloed data sources, interviewees struggled with manual efforts to create data pipelines. The head of industrial digitalization in a manufacturing organization said: “It has always been major manual effort to collect the data, to find it, to extract it, to clean it, and then actually to combine it with other data sources. We want to automize this work so that the people will get the data available when they need it and avoid these manual processes.”

  • Low machine-learning speed, iteration, and accuracy.

    Organizations struggled to execute on machine-learning insights quickly and accurately. Large, complex models were difficult to manage and limited the quantity of models that organizations could put into production.

  • Cost-prohibitive legacy solutions.

    On-premises and homegrown legacy solutions commanded expensive licensing, operational, and support costs, all while limiting scalability and innovation.

  • Inability to scale.

    Because of cost restrictions and complexity of tools and processes, interviewees reported an inability to scale ML and data science labor and workloads in their organizations’ prior environments. The IT director of data analytics in a manufacturing organization said: “One challenge that stopped us from leveraging [legacy] platforms was that we struggled spreading out the skills required to leverage [legacy] platforms to the masses. The other part of it was cost. When you compare Azure Machine Learning to some of those other tools, the reason that we can scale Azure Machine Learning is because it is not cost prohibitive.”

“In terms of pain points, our [legacy] platform was just incredibly expensive. We looked at the trajectory of machine learning technology and said, 'Okay, we can either keep going with this sort of custom-grown solution and watch the rest of people blow by us or are we can move towards a vendor solution.'”

Group SVP of application development, healthcare

“[Azure ML] gives us a tool that allows us to develop, train, and deploy the model theme. Then, using Azure or Microsoft products like Power Apps, we can very rapidly produce a prototype or a [minimum viable product] (MVP) to justify further investment and time in ML. It’s something that none of the other platforms could give us in our data environment. That made the choice to go with Azure ML pretty much a done deal.”

Digital transformation manager, manufacturing

Why Microsoft

Interviewees evaluated several ML solutions, including alternative enterprise vendors or extending existing homegrown and third-party tools. They cited the following reasons for ultimately selecting Microsoft Azure ML:

  • Scalable, pay-as-you go cloud solution.

    Interviewed decision-makers chose Azure ML for its dynamic scalability and cost efficiencies. As a cloud solution, Azure ML gives organizations instant access to provision and scale high-capacity resources. This drastically reduces effort provisioning infrastructure, gives data scientists the flexibility to iterate faster, and allows teams to scale more models into production. In addition, organizations only pay for what they consume, leading to additional cost savings and flexibility. Using this model, interviewees’ organizations quickly built out proof of concepts (POCs) and proved ML business value to increase ML and Azure ML adoption.

  • Operationalize AI with MLOps.

    Interviewees chose Azure ML for its MLOps capabilities (i.e., its ability to create, deploy, monitor, retrain, and orchestrate production ML pipelines at scale). MLOps allows organizations to automate the manual processes of deploying models and monitoring and retraining them, thus enabling them to dramatically accelerate the time-to-value of ML projects, as well as operationalize models at a different order of magnitude (from a handful to hundreds/thousands). Simultaneously, it tracks data and model lineage ensuring reproducibility, regulatory compliance, and the ability to retrain models in the future.

    In addition, nonprofessional and professional data scientists alike are given entry points that were not available in legacy solutions, improving machine learning adoption, usability, and extensibility.

  • End-to-end platform with interoperability efficiencies.

    Interviewees searched for a solution that integrated with their existing Azure stack and open-source technology, such as Azure Synapse Analytics, Power BI, Arc, Data Lake, Data Factory, and Python, to drive efficiencies across the end-to-end machine learning lifecycle. One manufacturer used Microsoft Power Apps to build out a front end for their temperature-predicting machine-learning model, enabling increased model insight consumption.

Figure 2
“What are the most important outcomes that your organization is looking to drive with its AA/ML/AI initiatives?”

Base: 199 data science, ML, or AI decision-makers
Source: A commissioned study conducted by Forrester Consulting on behalf of Microsoft, April 2021

“We went with Azure over competitors because of the investments our IT has made in Azure. It gives us a complete end-to-end solution. We have a very simple process for data ingestion from any of our complex data sources into Azure. Azure lets us access all of those together.”

IT director of data analytics, manufacturing

Investment Objectives

The interviewees’ organizations searched for a solution that could improve their efforts to:

  • Up-level machine-learning maturity and bring data scientists in-house.

    A major goal for interviewees’ organizations leveraging prebuilt models was to bring data scientist capabilities in-house to improve control, visibility, and quality of machine-learning insights. A lead solutions delivery architect in a utility organization explained: “We don’t want to have to rely on an external party to create models for us. We want to use the capabilities within Azure ML to build new models ourselves and then execute them once built. Azure ML is flexible enough for us to do both. We also hope to take models from external sources and integrate them in when we choose not to build our own.”

  • Operationalize and scale machine learning projects.

    Interviewees’ organizations wished to utilize MLOps to develop, deploy, and monitor models with greater speed, efficiency, and governance. Organizations sought to take advantage of seamless infrastructure orchestration; reproducible ML pipelines; reusable model training and development components; built-in monitoring and alerting of ML applications; and enhanced governance and security to improve business outcomes.

  • Broaden data science capabilities to nonprofessional data scientists.

    Azure ML’s ease of use and ability to support nonprofessional data scientist users was a major selling factor for interviewees’ organizations. Azure ML empowered other personas, such as business users, subject matter experts, and engineers, to take full advantage of the platform and fulfill data science and data engineering functions. This allowed organizations lower in machine-learning maturity receive machine-learning benefits faster and with fewer models in production.

“The cost of Azure Machine Learning is not a problem at least at this stage, because for me the benefits are greater than the investment. If I’m conservative, the benefits are about 10 times greater than the costs.”

Head of industrial digitalization, manufacturing

Composite Organization

Based on the interviews and survey, Forrester constructed a TEI framework, a composite company, and a ROI analysis that illustrates the areas financially affected. The composite organization is representative of the five companies that Forrester interviewed and 199 survey respondents’ companies and is used to present the aggregate financial analysis in the next section. The composite organization has the following characteristics:

  • Description of composite.

    The composite organization is a global enterprise with $7 billion in annual revenue. Before investing in Microsoft Azure ML, the composite organization leverages a custom, legacy on-premises machine-learning solution, as well as point solutions for statistical and predictive analysis. The composite organization is an existing Azure customer and leverages supporting services to streamline the machine-learning lifecycle.

  • Deployment characteristics.

    The composite implements Microsoft Azure ML by employing a land-and-expand strategy. Beginning with a team of five data scientists and a mix of three data engineers, DevOps, and IT, the composite organization realizes positive business benefits after initially deploying Azure ML, justifying further adoption across the organization. In Year 2, the composite expands Azure ML usage to existing and new data science teams. The composite nearly doubles the adoption of Azure ML in Year 2 and quadruples adoption in Year 3 as it reaches steady state. The benefits the composite organization realizes scale proportionately with internal adoption of Azure ML.

Key assumptions
  • $7 billion annual revenue
  • Azure ML users
    • Year 1: 8
    • Year 2: 15
    • Year 3: 60
NEXT SECTION: Analysis Of Benefits

Total Projected Benefits

Projected Benefits Year 1 Year 2 Year 3 Total Present Value
Total projected benefits (low) $147,186 $720,308 $3,674,321 $4,541,815 $3,489,674
Total projected benefits (mid) $316,761 $960,687 $4,377,880 $5,655,328 $4,371,087
Total projected benefits (high) $486,337 $1,201,066 $5,081,439 $6,768,842 $5,252,500

Please see Appendix C to view full benefits breakdown

Increased Operational Efficiency

“Azure ML improves our project productivity and collaboration by at least 20% to 25%.”

Director of data analytics, telecommunications

  • Evidence and data.

    Microsoft Azure Machine Learning sped up each step in the ML and AI lifecycle from data access and engineering to model training, validation, and deployment, as well as ongoing processes of model monitoring and retraining. Customers reported that Azure ML:

    • Improved productivity and collaboration across data scientists.

      Data scientists and data engineers collaborated more effectively by consolidating onto the same platform. For example, with shared notebooks and increased model lineage interviewees reduced siloed work and improved efficiency. Azure ML provided broader access to ML methods and frameworks, facilitating higher ease of use and accelerated ML lifecycles.

      • 81% of survey respondents said their machine-learning solution makes it easier to share, merge, and/or analyze data.
    • Reduces time to provision infrastructure and gather and prepare date.

      Data engineers spent significantly less time provisioning infrastructure and preparing data pipelines to feed into Azure ML due to greater access and interoperability with other data solutions. The IT director of data analytics told Forrester: “We used to have a team of six people gathering data for five customers and that was a full-time job. Now, through automating data pipelines, that data is available daily for all of our customers, which we can then use for predictive analysis and developing machine-learning models related to cash forecasting.”

      • 59% of survey respondents who improved model development experienced quicker infrastructure provisioning and data preparation for new tests.
      • A utility organization reduced the time to provision resources and select and build pipelines from three months to three weeks with Azure ML.
“Training the whole data set was certainly taking anywhere between two to three weeks. It has reduced to 8 to 10 hours now.”

Sr director, healthcare

  • Streamlined model development, training, validation, and deployment.

    Interviewees’ organizations nascent in their AI/ML initiatives quickly proved out the value of machine learning through Azure ML’s ability to standardize best practices, broaden capabilities across data scientists, and leverage prebuilt models and auto-ML. Organizations automated provisioning and deployment of ML pipelines, getting more models into production, faster. Shared notebooks, reproducible workflows, autoscaling and auto-ML all played a role in improving organizations ML operations. The director data analytics at a telco organization said: “MLOps absolutely allows my team to collaborate more effectively. We have improved productivity and are now collaboratively developing models.”

    • 70% of survey respondents who improved deployment spent less time to get models into production while 61% of respondents made less effort.
    • The director of data analytics at a telecommunications organization told Forrester that data scientists were 25% more productive in the development cycle and 100% more effective deploying models.
  • Operationalized model explainability and monitoring.

    Customers reported and anticipated that model monitoring with Azure ML will help avoid tireless manual effort monitoring models for drift, track lineage, and to provide risk transparency. The director of data at a manufacturing organization stated: “Considering our limited resources, streamlining the end-to-end machine learning pipelines, just like any of our data pipelines, is beneficial to us. This means our machine-learning teams don’t have to spend their lives on support or monitoring. They can move onto other initiatives and other projects.”

    • The director of data analytics at a telecommunication organization developed an automated email notification system for model monitoring and alerting that saved 5 hours a week for each of their data scientists.
    • 63% of survey respondents who improved monitoring efficiency said they improved their ability to mitigate prediction drift.
“Everything in Azure ML is organized and clean, such as pipelines after training scripts, preprocessing, relationship scripts, and deployment scripts. It’s also easy to do hyperparameter tuning and to train the model. [Azure ML] is very organized and it’s helping us a lot in that scenario from beginning to end.”

Sr director, healthcare

Figure 3
“You indicated that using ML helped you “increase business efficiency” for your organization. Has your organization experienced any of the following benefits as a result of your investment in your current ML solution?”
chart

Base: 199 data science, ML, or AI decision-makers
Source: A commissioned study conducted by Forrester Consulting on behalf of Microsoft, April 2021

  • Modeling and assumptions.

    For the composite organization, Forrester assumes that:

    • The composite scales its Microsoft Azure ML from 8 users in Year 1, 15 users in Year 2, and 60 users in Year 3.
    • Data scientists and ML engineers spend approximately 66% of their time doing data machine-learning tasks.
    • Data scientists and ML engineers realize time savings of up to 15% in the low state, 5% to 20% in the mid-state, and 10 to 25% in the high state.
    • The average annual fully burdened salary for a data scientist or ML engineer is $148,500.
    • Data engineers and other users spend approximately 70% of their time on data engineering tasks.
    • Data engineers, DevOps, and IT realize time savings of 25% to 30% in the low state, 30% to 35% in the mid-state, and 35% to 40% in the high state.
    • The average fully burdened salary for a data engineer is $124,200.
  • Results.

    This yields a three-year projected PV (discounted at 10%) ranging from $560,980 (low) to $865,495 (high).

“Increased Operational Efficiency Module: Range of Three-Year Cumulative Impact, PV”
chart
icon
Improved data scientist and ML engineer productivity:

15% to 25%

Increased Operational Efficiency

Ref. Metric Source Year 1 Year 2 Year 3
A1 Number of data scientists and ML engineering FTEs Composite 5 10 40
A2 Annual hours spent on ML activities per FTE Composite 1,373 1,373 1,373
A3Low
A3Mid
A3High
Data scientist and ML engineer time savings using Azure ML
Interviews		 0%
5%
10%		 15%
20%
25%		 15%
20%
25%
A4 Productivity recapture rate TEI standard 50% 50% 50%
A5 Average fully burdened hourly salary: data scientist and ML engineer (rounded) $110K*1.35 (benefit load)/2,080 hours per year $71 $71 $71
A6Low
A6Mid
A6High
Subtotal: Improved data scientist and ML engineer productivity
A1*A2*A3*A4*A5		 $0
$12,185
$24,371		 $73,112
$97,483
$121,854		 $292,449
$389,932
$487,415
A7 Number of data engineering, DevOps, and IT FTEs Composite 3 5 20
A8 Annual hours spent on ML activities per FTE Composite 1,456 1,456 1,456
A9Low
A9Mid
A9High
Data engineering, DevOps, and IT time savings using Azure ML
Interviews		 25%
30%
35%		 30%
35%
40%		 30%
35%
40%
A10 Average fully burdened hourly salary: Data engineer, DevOps, and IT FTE (rounded) $92K*1.35 (benefit load)/2,080 hours per year $60 $60 $60
A11Low
A11Mid
A11High
Subtotal: Improved data engineer, DevOps, and IT productivity
A7*A8*A9*A4*A10		 $32,760
$39,312
$45,864		 $65,520
$76,440
$87,360		 $262,080
$305,760
$349,440
AtLow
AtMid
AtHigh
Increased data science productivity
A6+A11		 $32,760
$51,497
$70,235		 $138,632
$173,923
$209,214		 $554,529
$695,692
$836,855
Three-year total:$725,921 to $1,116,304 Three-year present value: $560,980 to $865,495

Increased Income and Cost Savings From ML Insights

  • Evidence and data.

    Microsoft Azure ML enabled interviewees’ organizations to increase model throughput, accuracy, and quality. Customers kept models accurate over time thanks to better model monitoring and more frequent retraining. Ultimately, organizations achieved more revenue generating and cost saving ML applications with Azure ML than in their legacy environments. Customers reported that Azure ML:

    • Increased time-to-value of ML initiatives.

      Organizations saw a better time-to-value from accelerating their ML lifecycle and using automation to scale models in production. The lead solutions delivery architect in a utility organization said: “Using resources and platforms other than Azure ML, it took us three months to complete an AI proof of concept, from the point of provisioning compute resources, selecting data, annotating data for training, and then building and evaluating the model. We’ve been able to do the majority of that in three weeks using Azure ML.”

      • Surveyed respondents reduced the time reach insights by 37% on average.
      • Surveyed respondents had 54 models in production and increased that to 373 models with their current ML solution, on average.
  • Improved model accuracy.

    Interviewees’ organizations improved model accuracy and outcomes using Azure ML. Data scientists created robust models and maintained model integrity through frequent retraining. The group SVP of application development told Forrester that they increased the accuracy of their customer spend standardization use case by 80% to 85% and decreased the matching process from three weeks down to one day. Survey respondents on average increased their organizations’ machine-learning model accuracy by 9 percentage points.

  • Increased revenue from ML insights.

    Customers increased revenue across various use cases from improving upsell and renewal rates of consulting services to creating net-new revenue-generating utility infrastructure maintenance service.

    • The lead solutions delivery architect said: “Using Azure ML, we project being able to win more work of the type that we bid on now because we will have improved efficiency and quality of service delivery. Additionally, we anticipate new revenue streams, resulting from services that we really couldn’t enable without AI.”
    • The group SVP of application development in a healthcare organization said: “We have been able to retrospectively classify all of our 130 million rows of data representing $150 billion dollars’ worth of spend. That’s just something we would have never ever, ever been able to do [without Azure ML]. Because of that, we have more insight into regional consulting opportunities. Our ability to win that new business is going to go up. Our ability for net-new opportunities is going to go up.”
  • Increased cost savings ML insights.

    Customers reported a variety of cost saving use cases from predictive models that improved manufacturing energy consumption to models that optimized consumer goods inventory and supply chain efficiency.

    • The IT director of data analytics and a telecommunications company said: “We improved and reduced inventory by 8% to 10%, which is big money for us. That 10% is about $100 million basically of inventory reduction.”
    • The digital transformation officer at a manufacturing firm explained how one project helped decrease energy consumption in the manufacturing process, resulting in significant cost savings. The interviewee said: “We are looking at about $250,000 savings a year from our top projects. That is a very conservative number. Once fully implemented I’m sure we will exceed that.”
  • Modeling and assumptions.

    For the composite organization, Forrester assumes that:

    • Azure ML affects 1% of revenue in Year 1, 2% in Year 2, and 8% in Year 3 due to increased adoption and consumption.
    • The composite organization operating margin is 8%.
    • Azure ML affects 0.25% of costs in Year 1, 0.50% in costs in Year 2, and 2% of costs in Year 3 due to increased adoption and consumption.
  • Results.

    This yields a three-year projected PV (discounted at 10%) ranging from $1,566,914 (low) to $2,497,030 (high).

“Increased Income And Cost Savings From Ml Insights Module: Range of Three-Year Cumulative Impact, PV”
figure
icon
Incremental revenue uplift and cost savings from improved ML insights

$2.5M
“The last time we developed a model that we’re replacing with Azure ML was a nine-month exercise to build and prototype and get to operation. We got the new ML model in six weeks.”

IT director of data analytics, manufacturing

Increased Income And Cost Savings From ML Insights

Ref. Metric Source Year 1 Year 2 Year 3
B1 Annual revenue Composite $7,000,000,000 $7,000,000,000 $7,000,000,000
B2 Percent of revenue affected by Azure ML Composite 1% 2% 8%
B3Low
B3Mid
B3High
Revenue uplift from new and improved ML insights
Interviews		 1.00%
1.25%
1.50%		 1.25%
1.50%
1.75%		 1.50%
1.75%
2.00%
B4 Operating margin Composite 8% 8% 8%
B5Low
B5Mid
B5High
Subtotal: Incremental income from new ML insights
B1*B2*B3*B4		 $56,000
$70,000
$84,000		 $140,000
$168,000
$196,000		 $672,000
$784,000
$896,000
B6 Operating costs Composite $6,440,000,000 $6,440,000,000 $6,440,000,000
B7 Percent of operating costs affected by Azure ML Composite 0.25% 0.50% 2.00%
B8Low
B8Mid
B8High
Reduction in costs due to new and improved ML insights
Interviews		 0.25%
0.50%
0.75%		 0.50%
0.75%
1.00%		 0.75%
1.00%
1.25%
B9Low
B9Mid
B9High
Subtotal: Cost savings from new ML insights
B6*B7*B8		 $40,250
$80,500
$120,750		 $161,000
$241,500
$322,000		 $966,000
$1,288,000
$1,610,000
BtLow
BtMid
BtHigh
Increased income and cost savings from ML insights
B5+B9		 $96,250
$150,500
$204,750		 $301,000
$409,500
$518,000		 $1,638,000
$2,072,000
$2,506,000
Three-year total: $2,035,250 to $3,228,750 Three-year present value: $1,566,914 to $2,497,030

Cost Savings From Retiring Legacy Solutions

  • Evidence and data.

    By investing in Microsoft Azure Machine Learning, interviewees decommissioned legacy on-premises and third-party point solutions in favor of a more cost-efficient, pay-for-what-you-consume cloud licensing model. In doing so, organizations realized savings from legacy licensing, maintenance, administration, and support costs.

    As a result of switching to a cloud-based solution, customers also praised the instant access to on-demand, scalable, and powerful resources. The lead solutions delivery architect at a utility organization said: “We invested in a cloud-based machine-learning environment because of the ability to provision very high-capacity resources and only pay for them when they are in use. The ability to get access these resources when you need them and only pay for what you’re using is significantly more cost-effective than physical, on-premises solutions”.

  • Modeling and assumptions.

    For the composite organization, Forrester assumes that:

    • The composite organization incurs a $1,400,000 annual license cost for its legacy solution.
    • The cost to develop and support on the legacy environment was $350,000 annually.
    • The composite organization decommissions the legacy solution incrementally as they transition their data science teams to Azure ML.
  • Results.

    This yields a three-year projected PV (discounted at 10%) ranging from $1,268,783 (low) to $1,703,982 (high).

Figure 4
"You indicated that 'Reducing/eliminating on-premises hardware costs' had value for your organization. Using your best estimation, please estimate the combined hardware and software costs that you are saving by retiring your former architecture/licenses pertaining to ML over the next three years?"
figure

Base: 32 data science, ML, or AI decision-makers
Source: A commissioned study conducted by Forrester Consulting on behalf of Microsoft, April 2021

"Cost Savings From Retiring Legacy Technology Module: Range of Three Year Cumulative Impact, PV"
figure

Cost Savings From Retiring Legacy Technology

Ref. Metric Source Year 1 Year 2 Year 3
C1 Cost of legacy on-premises licensing and infrastructure Interviews $1,400,000 $1,400,000 $1,400,000
C2 Cost of legacy on-premises development and support Interviews $350,000 $350,000 $350,000
C3Low
C3Mid
C3High
Percent of legacy tooling decommissioned
Composite		 0%
5%
10%		 15%
20%
25%		 80%
85%
90%
CtLow
CtMid
CtHigh
Cost savings from retiring legacy technology
(C1+C2)*C3		 $0
$87,500
$175,000		 $262,500
$350,000
$437,500		 $1,400,000
$1,487,500
$1,575,000
Three-year total: $1,662,500 to $2,187,500 Three-year present value: $1,268,783 to $1,703,982

Cost Savings From Faster Onboarding

  • Evidence and data.

    Onboarding data scientists on Azure ML is simple and fast. Legacy environments were complex, had manual tasks, and required data scientists to learn more tools. Interviewees explained that Azure ML provides standardized working environments, automated workflows, and a simplified environment, allowing new users to get up to speed faster. Although onboarding existing data scientists to a new, notebook-and-code-first approach solution may challenge adoption, organizations projected that net-new data scientists will benefit from Azure ML’s architecture’s speed and scale.

    • On average, survey respondents decreased the time to onboard and train new data scientists by 35% after implementing their current machine-learning solution.
    • Survey respondents reduced their total onboarding spend by 14% on average.
    • Several interviewees stated that their organizations efficiently onboarded non-professional data scientists, such as engineers or business SMEs to become productive in Azure ML.
  • Modeling and assumptions.

    For the composite organization, Forrester assumes that:

    • The composite onboards two new data scientists in Years 1 and 2 and nine in Year 3.
    • 30% of data scientists that adopt Azure ML are net-new hires and require additional Azure ML training than existing data scientists.
  • Results.

    This yields a three-year projected PV (discounted at 10%) ranging from $92,997 (low) to $185,993 (high).

"Cost Savings From Faster Onboarding Module: Range of Three Year Cumulative Impact, PV"
chart
icon
Improved data scientist onboarding & training up to

40%
Figure 5
"You indicated that 'onboarding resources faster' had value for your organization. Using your best estimation, please estimate the decrease, if any, of the onboarding time necessary to be provided to your data scientists for you currrent ML solution as compared to your previous solution."
chart

Base: 17 data science, ML, or AI decision-makers
Source: A commissioned study conducted by Forrester Consulting on behalf of Microsoft, April 2021

Cost Savings From Faster Onboarding

Ref. Metric Calculation Year 1 Year 2 Year 3
D1 Number of new data scientists and ML engineers per year 30% of net-new FTEs 2 2 9
D2 Hours to train new data scientist and ML engineers in legacy systems Composite 640 640 640
D3Low
D3Mid
D3High
Reduction in time to train new data scientists and ML engineers
Composite		 20%
30%
40%		 20%
30%
40%		 20%
30%
40%
D4 Average fully burdened hourly salary: data scientist and ML engineer (rounded) $110K*1.35 (benefit load)/2,080 hours per year $71 $71 $71
DtLow
DtMid
DtHigh
Cost savings from faster onboarding
D1*D2*D3*D4		 $18,176
$27,264
$36,352		 $18,176
$27,264
$36,352		 $81,792
$122,688
$163,584
Three-year total: $118,144 to $236,288 Three-year present value: $92,997 to $185,993

Unquantified Benefits

“Microsoft Azure ML has allowed for the democratizing of our machine learning capabilities. Non-data scientists can run it all on their own and it’s allowed our true data scientist folks to further focus on wilder use cases, if you will, around what our customers really need”

Group SVP of application development, healthcare

Additional benefits that customers experienced but were not able to quantify include:

  • Made data scientist capabilities available to other groups within the organization.

    Interviewees reported that Azure MLs enabled nonprofessional data scientists to take full advantage of machine-learning capabilities. This allowed organizations to scale ML applications, increase model throughput, and enable data scientists to spend more time on higher-value tasks. The lead solutions delivery architect in a utility organization said, “Using Azure ML, some of the processes to support our machine-learning capabilities are not even done by our technical teams but rather our functional teams.”

  • Increased business-user and subject matter expert (SME) productivity.

    The lead solutions delivery architect at a utility organization told Forrester that incorporating machine-learning workloads into business processes enabled business users to be more effective and allowed them to scale their workloads. The interviewee said: “Using computer vision, we are creating an AI-supported-human-in-the-loop inspection process. We want these AI functions to make the job of the human SME in the inspection loop easier and more scalable, allowing them to shift their time to more valuable areas. Through machine learning and AI processes, we can take work off their plate.”

  • Improved interoperability and extensibility of machine-learning platforms and open-source ML frameworks.

    Interviewees told Forrester that using Azure ML allows for easier integration across machine-learning platforms and open-source frameworks, future proofing their ML platform investment. The head of industrial digitalization in a manufacturing organization said: “You are able to do so many different things with the Microsoft and Azure tools from collecting live data, analyzing and building models, and then releasing models into applications we have built. It’s the wholistic package that makes Azure ML so valuable.”

  • Improved responsible ML and model compliance, governance, and security.

    Azure ML provided customers enough transparency and security to manage risk and satisfy regulators. Model lineage and explainability also support responsible ML goals. The director of data analytics in a telecommunications organization explained how his organization improved management visibility and oversight, stating: “Azure ML has really interesting explainability tools that can help us deep dive into the contributing variables driving our results. It is a great tool, especially when your leadership teams ask us to explain how we reached our results. We can even prepare this data ahead of time before they even ask.”

The director of analytics in a telecommunications organization said: “It’s easier to adhere to company compliance standards within the Azure environment. If we hadn’t had this platform available to us, then I would have to hire software engineers to develop those applications, put it in production, and check all types of company compliance on cloud environment or the security, etc. It would take a few months to get through that for even one application. Right now, that’s all within the parameters drawn for us within the Azure ML platform.”

“Because we are already hosting our existing technical platform in Azure, we believe the investment in Azure ML yields the least total cost approach for us in terms of ownership and support of our AI infrastructure.”

Lead solutions delivery architect, utility

“The fact that we’re running in Azure has made it a lot easier for us to respond to customer questions regarding data storage, hosting, business continuity, intrusion detection, and data encryption with respect to our AI/machine-learning services.”

Lead solutions delivery architect, utility

Flexibility

“We have been able to semi- automate reporting by shifting excel reporting into Power BI. Because of that, we have so many more people in different departments who can consume, adopt, change, or develop reporting and visualize data.”

Head of industrial digitalization, manufacturing

The value of flexibility is unique to each customer. There are multiple scenarios in which a customer might implement Azure Machine Learning and later realize additional uses and business opportunities, including:

  • Further operationalizing AI with MLOps.

    Azure ML provides the features and extensibility necessary for low to mid-maturity ML organizations to continue building out ML pipelines at scale. As organizations continue to scale models to production, Azure ML helps track data and model lineage ensuring reproducibility, regulatory compliance, and the ability to retrain models in the future.

  • Enabling digital transformation.

    Customers told Forrester that Microsoft Azure ML is an enabling technology that sets the tone in their digital transformation journey. The digital transformation manager in a manufacturing organization stated: “We are at the apex of a strong push towards digital transformation. Azure ML is an enabling technology whose use cases provides justification for its value. People can see that the benefits are real.”

  • Increasing innovation and productivity from the cross-pollination of ideas.

    By standardizing machine-learning environments, organizations may improve innovation and knowledge sharing from other data science teams. The lead solutions delivery architect in utility said: “There has definitely been some cross-pollination and sharing of information that’s made my group more efficient in the work that they’re doing.”

  • Realizing storage savings.

    The lead solutions architect in a utility organization told Forrester that using Azure ML along with other Azure solutions will enable further cost efficiencies. The interviewee said: “As Azure is our primary storage solution for our overall platform, we will see additional savings over time because we can use the same data to support our traditional production processes and our machine learning development processes. I’m estimating that over time our costs using Azure will be half of what we would be paying for an alternative provider model.”

Flexibility would also be quantified when evaluated as part of a specific project (described in more detail in Appendix A).

Figure 6
"On a scale of 1 to 5, where 1 means do not agree at all and 5 means strongly agree, how much do you agree with the following statements? Your current ML solution..."
figure

Base: 199 data science, ML, or AI decision-makers
Source: A commissioned study conducted by Forrester Consulting on behalf of Microsoft, April 2021

NEXT SECTION: Analysis Of Costs

Total Costs

Ref. Cost Initial Year 1 Year 2 Year 3 Total Present Value
Etr Cost of Azure ML and supporting applications $0 $37,375 $74,750 $299,000 $411,125 $320,397
Ftr Implementation and ongoing costs $495,000 $81,675 $163,350 $245,025 $985,050 $888,341
Total costs (risk-adjusted) $495,000 $119,050 $238,100 $544,025 $1,396,175 $1,208,738

Cost Of Azure ML And Supporting Applications

  • Evidence and data.

    Microsoft Azure ML pricing is based on machine-learning computation and data storage consumption and varies depending on each organization’s characteristics and needs. There are no additional charges to use Azure ML. Customers can choose pay-as-you-go pricing or select a one or three-year reserved pricing for the instance, vCPU(s), and RAM, depending on:

    • General purpose use for websites, small-to-medium databases, and other everyday applications.
    • Compute optimized use for high CPU-to-memory ratio good for medium traffic web servers, network appliances, batch processes, and application servers.
    • Memory optimized use for high memory-to-core ratio good for relational database servers, medium-to-large caches, and in-memory analytics.
    • Graphics Processing Unit (GPU) use for specialized virtual machines targeted for heavy graphic rendering and video editing available with single or multiple GPUs.

Customers incurred separate charges from Azure ML for other Azure services consumed, including but not limited to Azure Blob Storage, Azure Key Vault, Azure Container Registry and Azure Application Insights. Interviewed customers also leveraged other supporting services, such as Azure Synapse, Azure Data Lake Storage Gen2, Power Apps, and Power BI to integrate data pipelines and operationalize machine-learning insights and reporting. For further pricing information, please speak with a sales specialist.

  • Modeling and assumptions.

    For the composite organization, Forrester assumes that:

    • The composite organization’s compute and storage doubles in Year 2 and quadruples in Year 3 in relation to increased adoption and usage with the organization.
    • The composite pays for supporting applications and data engineering compute and storage scaled with Azure ML usage.
  • Risks.

    The expected financial impact is subject to risk and variation based on several factors, including:

    • Variance in compute and storage consumption.
    • Cost of supporting applications.
  • Results.

    To account for these risks, Forrester adjusted this cost upward by 15%, yielding a three-year, risk-adjusted total PV (discounted at 10%) of $320,397.

Cost Of Azure ML And Supporting Applications

Ref. Metric Calculation Initial Year 1 Year 2 Year 3
E1 Cost of Azure ML compute and storage Composite $25,000 $50,000 $200,000
E2 Cost of supporting applications Composite $7,500 $15,000 $60,000
Et Cost of Azure ML and supporting applications E1+E2 $0 $32,500 $65,000 $260,000
Risk adjustment ↑15%
Etr Cost of Azure ML and supporting applications (risk-adjusted) $0 $37,375 $74,750 $299,000
Three-year total: $411,125 Three-year present value: $320,397

Implementation And Ongoing Costs

  • Evidence and data.

    Customers relied on internal labor and third-party services to plan, implement, and train data scientists, and integrate data sources and tooling. The extent of professional services and training required vary by organization. Several interviewees also noted internal labor needed for ongoing maintenance such as upgrades, testing, integration, and oversight.

    • The lead solutions delivery architect noted cost savings efficiency in supporting Azure ML because of internal labor skills supporting other Azure solutions: “There’s so much interplay between the environment management and DevOps parts of Azure ML and the regular Azure. The cost for support is probably half what it would be if we were using different solutions for each aspect of our platform.”
  • Modeling and assumptions.

    For the composite organization, Forrester assumes that:

    • The composite utilizes professional services to support planning, change management, implementation, integration, and data scientist training during the initial implementation period.
    • The composite also leverages internal ML engineer labor for ongoing maintenance of the Azure ML platform.
  • Risks.

    The expected financial impact is subject to risk and variation based on several factors, including:

    • Organizations experience with Azure.
    • Complexity of change management and training.
    • Size of implementation and integrations.
    • Frequency of updates.
  • Results.

    To account for these risks, Forrester adjusted this cost upward by 10%, yielding a three-year, risk-adjusted total PV of $888,341.

Implementation And Ongoing Costs

Ref. Metric Calculation Initial Year 1 Year 2 Year 3
F1 Total professional services fees and training Composite $450,000
F2 FTE required for ongoing maintenance and ML operationalization Composite 0.5 1.0 1.5
F3 Average fully burdened hourly salary: ML engineer (rounded) $110K*1.35 (benefit load) $148,500 $148,500 $148,500
Ft Implementation and ongoing costs F1+(F2*F3) $450,000 $74,250 $148,500 $222,750
Risk adjustment ↑10%
Ftr Implementation and ongoing costs (risk-adjusted) $495,000 $81,675 $163,350 $245,025
Three-year total: $985,050 Three-year present value: $888,341
NEXT SECTION: Financial Summary

CONSOLIDATED THREE-YEAR RISK-ADJUSTED METRICS

  • These risk-adjusted PROI and projected NPV values are determined by applying risk-adjustment factors to the unadjusted results in each Benefit and Cost section.

Three-Year Projected Financial Analysis For The Composite Organization

Cash Flow Table (Risk-Adjusted Estimates)

Initial Year 1 Year 2 Year 3 Total Present Value
Total costs ($495,000) ($119,050) ($238,100) ($544,025) ($1,396,175) ($1,208,738)
Total benefits (low) $0 $147,186 $720,308 $3,674,321 $4,541,815 $3,489,674
Total benefits (mid) $0 $316,761 $960,687 $4,377,880 $5,655,328 $4,371,087
Total benefits (high) $0 $486,337 $1,201,066 $5,081,439 $6,768,842 $5,252,500
Net benefits (low) ($495,000) $28,136 $482,208 $3,130,296 $3,145,640 $2,280,936
Net benefits (mid) ($495,000) $197,711 $722,587 $3,833,855 $4,259,153 $3,162,349
Net benefits (high) ($495,000) $367,287 $962,966 $4,537,414 $5,372,667 $4,043,762
PROI (low) 189%
PROI (mid) 262%
PROI (high) 335%

The financial results calculated in the Benefits and Costs sections can be used to determine the PROI and projected NPV for the composite organization’s investment. Forrester assumes a yearly discount rate of 10% for this analysis.

NEXT SECTION: Appendix

Appendix A: New Technology: Projected Total Economic Impact

New Technology: Projected Total Economic Impact (New Tech TEI) is a methodology developed by Forrester Research that enhances a company’s technology decision-making processes and assists vendors in communicating the value of their products and services to clients. The New Tech TEI methodology helps companies demonstrate and justify the projected tangible value of IT initiatives to senior management and key business stakeholders.

Total Economic Impact Approach

  • Projected Benefits represent the projected value to be delivered to the business by the product. The New Tech TEI methodology places equal weight on the measure of projected benefits and the measure of projected costs, allowing for a full examination of the effect of the technology on the entire organization.

  • Projected Costs consider all expenses necessary to deliver the proposed value of the product. The projected cost category within New Tech TEI captures incremental ongoing costs over the existing environment that are associated with the solution.

  • Flexibility represents the strategic value that can be obtained for some future additional investment building on top of the initial investment already made. Having the ability to capture that benefit has a PV that can be estimated.

  • Risks measure the uncertainty of benefit and cost estimates given: 1) the likelihood that estimates will meet original projections and 2) the likelihood that estimates will be tracked over time. TEI risk factors are based on “triangular distribution.”

  • PRESENT VALUE (PV)

    The present or current value of (discounted) cost and benefit estimates given at an interest rate (the discount rate). The PV of costs and benefits feed into the total NPV of cash flows.

  • NET PRESENT VALUE (NPV)

    The present or current value of (discounted) future net cash flows given an interest rate (the discount rate). A positive project NPV normally indicates that the investment should be made, unless other projects have higher NPVs.

  • RETURN ON INVESTMENT (ROI)

    A project’s expected return in percentage terms. ROI is calculated by dividing net benefits (benefits less costs) by costs.

  • DISCOUNT RATE

    The interest rate used in cash flow analysis to take into account the time value of money. Organizations typically use discount rates between 8% and 16%

  • PAYBACK PERIOD

    The breakeven point for an investment. This is the point in time at which net benefits (benefits minus costs) equal initial investment or cost.

The initial investment column contains costs incurred at “time 0” or at the beginning of Year 1 that are not discounted. All other cash flows are discounted using the discount rate at the end of the year. PV calculations are calculated for each total cost and benefit estimate. NPV calculations in the summary tables are the sum of the initial investment and the discounted cash flows in each year. Sums and present value calculations of the Total Benefits, Total Costs, and Cash Flow tables may not exactly add up, as some rounding may occur.

Appendix B: Interview And Survey Demographics

Interviewed Organizations

Industry Region Interviewee Revenue
Telecommunications Global, EMEA HQ Director of data analytics $40 billion+
Use cases:Deliver intelligent supply chain and inventory efficiencies through revenue forecasting across thousands of locations and product families
Manufacturing Global, APAC HQ
  • IT director of data analytics
  • Data transformation manager
 $5 billion+
Use cases:Build more agile factories and optimize processes through predictive temperature models; improve operational outcomes through spend analytics and cost forecasting; know customers through predictive customer behavior
Manufacturing Global, EMEA HQ Head of industrial digitalization $5 billion+
Use cass: Build more agile factories and decrease energy footprint through predictive energy consumption models, automated quality inspection, and predictive maintenance and repair
Healthcare United States
  • Group SVP of application development
  • Senior director
  • Director of software engineering
 $1 billion+
Use cass: Improve operational outcomes through automatic categorization of products and standardizing customer spend
Utility United States Lead solutions delivery architect Private
Use cass: Transform the workforce through automation using computer vision to supplement SMEs

Survey Demographics

Region

Annual Revenue (USD)

Size

Industry

Base: 199 data science, ML, or AI descision-makers
Note: Percentages may not total 100 because of rounding.

Appendix C: Benefit Tables

Total Project Benefits - Low

Ref. Benefit Year 1 Year 2 Year 3 Total Present Value
AtLow Increased operational efficiency: Calculation Table $32,760 $138,632 $554,529 $725,921 $560,980
BtLow Increased income and cost savings from ML insights: Calculation Table $96,250 $301,000 $1,638,000 $2,035,250 $1,566,914
CtLow Cost savings from retiring legacy technology: Calculation Table $0 $262,500 $1,400,000 $1,662,500 $1,268,783
DtLow Cost savings from faster onboarding: Calculation Table $18,176 $18,176 $81,792 $118,144 $92,997
Total projected benefits - Low $147,186 $720,308 $3,674,321 $4,541,815 $3,489,674

Total Project Benefits - Mid

Ref. Benefit Year 1 Year 2 Year 3 Total Present Value
AtMid Increased operational efficiency: Calculation Table $51,497 $173,923 $695,692 $921,112 $713,238
BtMid Increased income and cost savings from ML insights: Calculation Table $150,500 $409,500 $2,072,000 $2,632,000 $2,031,972
CtMid Cost savings from retiring legacy technology: Calculation Table $87,500 $350,000 $1,487,500 $1,925,000 $1,486,382
DtMid Cost savings from faster onboarding: Calculation Table $27,264 $27,264 $122,688 $177,216 $139,495
Total projected benefits - Mid $316,761 $960,687 $4,377,880 $5,655,328 $4,371,087

Total Project Benefits - High

Ref. Benefit Year 1 Year 2 Year 3 Total Present Value
AtHigh Increased operational efficiency: Calculation Table $70,235 $209,214 $836,855 $1,116,304 $865,495
BtHigh Increased income and cost savings from ML insights: Calculation Table $204,750 $518,000 $2,506,000 $3,228,750 $2,497,030
CtHigh Cost savings from retiring legacy technology: Calculation Table $175,000 $437,500 $1,575,000 $2,187,500 $1,703,982
DtHigh Cost savings from faster onboarding: Calculation Table $36,352 $36,352 $163,584 $236,288 $185,993
Total projected benefits - High $486,337 $1,201,066 $5,081,439 $6,768,842 $5,252,500

Appendix D: Endnotes

1 Total Economic Impact is a methodology developed by Forrester Research that enhances a company’s technology decision-making processes and assists vendors in communicating the value proposition of their products and services to clients. The TEI methodology helps companies demonstrate, justify, and realize the tangible value of IT initiatives to both senior management and other key business stakeholders

About Forrester Consulting

Forrester Consulting provides independent and objective research-based consulting to help leaders succeed in their organizations. Ranging in scope from a short strategy session to custom projects, Forrester’s Consulting services connect you directly with research analysts who apply expert insight to your specific business challenges. For more information, visit forrester.com/consulting.

Forrester Research, Inc.© All rights reserved. Unauthorized reproduction is strictly prohibited. Information is based on best available resources. Opinions reflect judgment at the time and are subject to change. Forrester®, Technographics®, Forrester Wave, RoleView, TechRadar, and Total Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective companies. For additional information, go to forrester.com.

Cookie Settings