April 2020

A Forrester New Technology: Project Total Economic Impact™ Study Commissioned By Microsoft

New Technology: The Projected Total Economic Impact™ Of Microsoft Search In Bing

Executive Summary

Enterprise content management poses a significant challenge for companies today.1 As organizations continue to grow the amount of data they store, it is critical for IT to make this data discoverable. Organizations should leverage technology to provide employees with both easy access to information and the ability to quickly collaborate with their colleagues to improve their daily work experiences and achieve better business outcomes.2

Microsoft provides a free enterprise search solution that relies on Microsoft’s AI and knowledge graph to help employees quickly find content and people across the Microsoft 365 ecosystem. Microsoft commissioned Forrester Consulting to conduct a Total Economic Impact™ (TEI) study and examine the potential return on investment (ROI) enterprises may realize by deploying Microsoft Search in Bing. The purpose of this study is to provide readers with a framework to evaluate the potential financial impact of the Microsoft Search in Bing on their organizations.

To better understand the benefits, costs, and risks associated with this investment, Forrester interviewed seven customers with two to three years of experience using Microsoft Search in Bing.

Prior to using Microsoft Search in Bing, the IT organizations at the interviewed companies struggled to make the growing volume of enterprise data accessible for employees. Employees expressed frustration with current search solutions that were fragmented and inconsistent, often returning irrelevant results. They lost time, duplicated work, and relied on other teams within the organization (think help desk or HR) for help.

Microsoft Search in Bing transformed the enterprise search experience: it made enterprise data discoverable, increased employees’ productive time, enhanced collaboration, and reduced workplace frustration. From their pilot programs, organizations saw the potential to reduce the number of support tickets by answering employee questions via Microsoft Search in Bing. Once they saw they recognized the benefits of Microsoft Search in Bing, interviewed companies retired their former search solutions, thus eliminating significant software and labor costs associated with the search capability.

Projected Benefits

  • Circular arrow representing productivity savings

    Productivity savings from using Microsoft Search in Bing:
    $13.6M to $40.7M

  • Image of multiple documents representing savings from reducing help desk burden

    Savings from reducing burden on help desk:
    $262K to $1M

  • Downward icon representing savings from retiring former enterprise search solutions

    Savings from retiring former enterprise search solutions:
    $1.1 million

“This is one of those small IT projects that can have big wins. It does not take a lot of resources to get started with Microsoft Search in Bing, but you get that big bang for your buck.”

Chief architect for collaboration services, professional services

Key Findings

  • ROI icon

    Projected benefits PV
    $15 million to $43 million

  • Benefits PV icon

    Projected NPV
    $14.9 million to $42.8 million

  • NPV icon

    Total costs
    $41,379

Quantified benefits.The following risk-adjusted present value (PV) quantified benefits are representative of those experienced by the companies interviewed:

  • Productivity growth enabled by Microsoft Search in Bing leads to $13.6 to $40.7 million to savings in labor costs in three years.

    Prior to using Microsoft Search in Bing, employees did not have a way to search the entire Microsoft 365 ecosystem. They had to initiate search from the right product to get a result, and even if they did, they often had to scroll or preview files to get to the exact one. It was not uncommon to abandon the effort in frustration and reach out for help or recreate the document. Microsoft Search in Bing transformed the search experience by not only finding the right files, but also by doing so in fewer clicks. The 50% to 75% reduction in average time spent on work-related search allowed employees to cut the time that was wasted looking for information and to use that time for productive work.

  • Improved ability to find answers reduces the number of certain support tickets by 30% to 60%.

    Prior to Microsoft Search in Bing, employees frequently contacted help desk, HR, or other dedicated resources for guidance on simple tasks. Microsoft Search in Bing makes relevant answers easy to find, which helps save $262,000 to $1 million over three years.

  • Built-in search eliminates the costs of former search tools of $1.1 million.

    Historically, organizations relied on an assortment of tools to manage content and make it easier to find. Supporting these tools was labor-intensive and yielded limited results. With the adoption of Microsoft Search in Bing, organizations stopped paying software licenses for former tools and reassigned the dedicated engineering resources to their core functions.

Unquantified benefits. The interviewed organizations experienced the following benefits, which are not quantified for this study:

  • Improved employee experience.

    Before Microsoft Search in Bing, enterprise search frequently returned irrelevant results which wasted employees’ time and incited frustration. With its reliance on AI and Microsoft Graph, Microsoft Search in Bing understands the user’s intent and returns personalized and relevant results.

  • Consistency across environments.

    Using Microsoft Search in Bing enables users to search the entire Microsoft 365 ecosystem from a single location, ensuring convenience and consistent results from any device.

  • Stronger security and compliance.

    Protection of organizations’ information is top priority for IT and security teams. Using Microsoft Search in Bing helped companies identify existing data security vulnerabilities and address them before any damage was done, i.e., such as a SharePoint site or a OneDrive file containing personally identifiable information that is open to everyone.

  • Better business outcomes and collaboration.

    Microsoft Search in Bing enables employees to quickly identify content that’s relevant to their work, which could lead to more collaboration across teams, business units, and geographies, and ultimately, better business outcomes.

Costs. The interviewed organizations experienced the following risk-adjusted PV costs:

  • Implementation, ongoing adminstration, and change management.

    Every Microsoft customer has Microsoft Search in Bing, and using it successfully requires both the customization of features like Bookmarks, Acronyms, or Q&A, and training the organization to take advantage of the tool. Internal team members enabled Microsoft Search in Bing over the course of one month. At the same time, a change management group ensured that users were familiar with the new tool. Their time sums to a risk-adjusted, three-year PV of $41,379.

Forrester modeled a range of projected low, medium, and high impact outcomes based on evaluated risk. This financial analysis projects that the composite organization accrues the following three-year net present value (NPV) for each scenario by enabling Microsoft Search in Bing:

  • Projected high impact of a $42.8 million NPV.

  • Projected medium impact of a $27.3 million NPV.

  • Projected low impact of a $14.9 million NPV.

TEI Framework And Methodology

The New Tech TEI methodology helps companies demonstrate and justify the projected tangible initiatives to both senior management and other key business stakeholders.

From the information provided in the interviews, Forrester has constructed a Total Economic Impact™ (TEI) framework for those organizations considering implementing Microsoft Search in Bing.

The objective of the framework is to identify the cost, benefit, flexibility, and risk factors that affect the investment decision. Forrester took a multistep approach to evaluate the impact that Microsoft Search in Bing can have on an organization:

  • Chat icon
    DUE DILIGENCE

    Interviewed Microsoft stakeholders and Forrester analysts to gather data relative to Microsoft Search in Bing.

  • Customer icon
    EARLY-IMPLEMENTATION CUSTOMER INTERVIEWS

    Interviewed seven organizations using Microsoft Search in Bing in a pilot or beta stage to obtain data with respect to projected costs, benefits, and risks.

  • Composite Organization
    COMPOSITE ORGANIZATION

    Designed a composite organization based on characteristics of the interviewed organizations.

  • Framework icon
    PROJECTED FINANCIAL MODEL FRAMEWORK

    Constructed a financial model projection representative of the interviews using the New Tech TEI methodology and risk-adjusted the financial model based on issues and concerns of the interviewed organizations.

  • Casestudy icon
    CASE STUDY

    Employed four fundamental elements of New Tech TEI in modeling Microsoft Search in Bing’s potential impact: benefits, costs, flexibility, and risks. Given the increasing sophistication that enterprises have regarding ROI analyses related to IT investments, Forrester’s TEI methodology serves to project a complete picture of the total economic impact of purchase decisions. Please see Appendix A for additional information on the New Tech TEI methodology.

DISCLOSURES

Readers should be aware of the following:

This study is commissioned by Microsoft and delivered by Forrester Consulting. It is not meant to be used as competitive analysis.

Forrester makes no assumptions as to the potential ROI that other organizations will receive. Forrester strongly advises that readers use their own estimates within the framework provided in the report to determine the appropriateness of an investment in Microsoft Search in Bing.

Microsoft reviewed and provided feedback to Forrester, but Forrester maintains editorial control over the study and its findings and does not accept changes to the study that contradict Forrester’s findings or obscure the meaning of the study.

Microsoft provided the customer names for the interviews but did not participate in the interviews.


The Microsoft Search In Bing Customer Journey

BEFORE AND AFTER THE SEARCH IN BING INVESTMENT

Interviewed Organizations

For this study, Forrester conducted seven interviews with Microsoft Search in Bing customers. Interviewed customers include the following:

INDUSTRY REGION INTERVIEWEE NUMBER OF EMPLOYEES
Consumer goods Global Office 365 architect 130,000 employees
Professional services Global Solutions architect 30,000 employees
Healthcare Global Collaboration analyst 80,000 employees
Oil and mining Global – Core productivity platforms manager
– Office 365 architect engineer
100,000 employees
IT services Global IT director, corporate digital services 120,000 employees
Education North America Digital workplace architect 7,000 employees,
75,000 students
Professional services Global Chief architect 300,000 employees

Key Challenges

“There are 250,000 SharePoint sites in our environment, which equals to 117 terabytes of data. On OneDrive, we have billions of files and almost two petabytes worth of data. [Our previous solution] was not indexing it. None of that was being indexed. And this is a significant amount of data not available to search. Once we switched over to Microsoft Search in Bing, all of that is being indexed. And it’s being indexed at no cost.”

Core productivity platforms manager, oil and mining

Interviewees told Forrester that, prior to using Microsoft Search in Bing, their organizations were expecting the following challenges.

  • Inefficient content management.

    Enterprise content management poses a significant challenge for companies today.3 The Microsoft customers interviewed for this study are no exception. While all companies attempted to find ways of organizing data to make it discoverable and usable across teams and geographies, they did not succeed. The digital workplace architect at an education company said: “All of our previous efforts were never enough. I was always concerned that we were not enabling an average user to get a quality result.”

  • Loss of productivity.

    More than half (54%) of global information workers are interrupted from their work a few times or more per month to spend time looking for or trying to get access to information, insights, and answers.4 This is also true for the interviewed customers: inefficient search slowed employees down and, when they could not find something, frequently led to duplicating work. The collaboration analyst at a healthcare company explained, “Our employees told us that when they went to search for something, they hadn’t known the exact keyword, or they would have to scroll through several pages of results before they found the right document.”

Key Results

The interviews revealed that key results from the Search in Bing investment include:

“While tagging works for some organizations, it did not for us. But with Microsoft Search in Bing the relevance is based off the graph and AI, and it is a more effective way for us to get better search results without so much administrative effort as implementing the tags which took a lot of work.”

Collaboration analyst, healthcare

  • Microsoft Search in Bing solves the IT quest to make enterprise data discoverable.

    As modern organizations continue to grow the amount of data they store, it is critical for IT to make this data searchable. Former attempts of tagging and implementing taxonomies demanded significant investment and labor, yet yielded little to no results. Microsoft Search in Bing, on the contrary, relies heavily on AI and Microsoft Graph, requires little management, and delivers results quickly, at no cost. The chief architect at a professional services company said: “Even if you have another search solution, it is not going to give you what Microsoft Search in Bing can provide. You'll put in significant effort setting it up, you'll have to pay for it, but it won’t have native integrations with all of Office 365, and it won't provide the best user experience. Switching to Microsoft Search in Bing is one of those projects that is low hanging fruit for IT, but it’s a big win for the organization.”

  • Efficient search improves employees' productive time, enhances collaboration, and reduces frustration.

    Microsoft Search in Bing enables employees to find relevant work-related information within the Office 365 ecosystem faster and on any device, saving each employee, on average, tens of hours per year. One of the key Microsoft Search in Bing features, People search, helps employees find others across the company who work on similar projects or possess the expertise on a subject, thus streamlining collaboration. For newer employees, the ability to look up acronyms that are used within the organization or their specific group can reduce stress and make the transition process much smoother.

  • Microsoft Search in Bing shows potential in reducing pressure on support systems.

    Many interviewed organizations are just starting to set up Q&A in Microsoft Search in Bing, but they already see positive results from enabling employees to find answers instead of asking for help. Organizations expect to see a reduction in number of help-desk tickets and, possibly, HR tickets, as users start to treat enterprise search as web search and use it more heavily to answer questions like “how to reset my password” or “how to submit my timesheets” instead of going to a corresponding team for help.

"We occasionally go out to the user communities and talk to them about the technology they use. When we showed Microsoft Search in Bing to them, the reaction was: 'Oh my god. This is great! I didn't know we could do this.'"

Core productivity platforms manager, oil and mining


Composite Organization

Based on the interviews, Forrester constructed a TEI framework, a composite company, and an associated ROI analysis that illustrates the areas financially affected. The composite organization is representative of the five companies that Forrester interviewed and is used to present the aggregate financial analysis in the next section. The composite organization that Forrester synthesized from the customer interviews has the following characteristics:

  • Description of composite.

    This global organization has a strong brand and a large customer base. The composite organization employs office, remote, and field workers. Most employees rely on access to enterprise files to successfully perform their daily tasks.

  • Deployment characteristics.

    The organization is a Microsoft 365 customer and relies on SharePoint and OneDrive to store enterprise data. From conducting regular employee satisfaction surveys, the organization knew that employees found search for existing information and data within the organization difficult and inefficient. To improve search, the organization invested in several search tools, including a tagging and a taxonomy solution, but it found the tools difficult to implement and inadequate, failing to enhance employee experience and productivity. After learning about Microsoft Search in Bing, the organization decided to gradually enable Microsoft Search in Bing for all employees. At the time of the study, employees use multiple browsers, including Internet Explorer, Edge, Chrome, and Firefox. For Internet Explorer and Edge, Microsoft Search in Bing is a default search engine and is automatically used when users type a request in their task bars. In other browsers employees need to set Bing as the default web browser in order to use Microsoft Search in Bing through the task bar, or they can access it at www.bing.com once they log in with their work account.

Key assumptions:
  • $10B revenue
  • 50,000 employees
  • Worldwide operations
  • 5.28% of each day spent searching for information

Analysis Of Benefits

QUANTIFIED BENEFIT DATA AS APPLIED TO THE COMPOSITE

Total Projected Benefits

Projected Benefits Year 1 Year 2 Year 3 Total Present Value
Total projected benefits (low) $2,904,900 $5,402,778 $10,448,492 $18,756,170 $14,956,031
Total project benefits (mid) $5,101,161 $9,839,225 $19,410,115 $34,350,501 $27,352,117
Total projected benefits (high) $7,849,200 $15,390,264 $30,623,213 $53,862,677 $42,862,537

The table above shows the total of all benefits across the areas listed below, as well as present values (PVs) discounted at 10%. Over three years, the composite organization expects risk-adjusted total benefits to have a projected PV range of $17 million to $46.5 million.


Productivity Savings From Using Microsoft Search In Bing

Digital watch icon representing multiple hours employees earn back with Microsoft Search in Bing
Employees earn back on average 55 to 82 hours per year with Microsoft Search in Bing.

All interviewed organizations were looking to make it easier for employees to succeed in their daily work; by providing easy access to information they need to complete their tasks.5 Prior to Microsoft Search in Bing, enterprise search was not a tool that employees could rely on to quickly find work-related information, including files, people, acronyms, or answers to specific questions. Typically, users would conduct search within a certain product, such as OneDrive or Outlook, and give up if they could not find what they are looking for within the first few results. Not finding answers could lead to more extensive search within other products, reaching out to colleagues for help, or giving up on search entirely. For employees who tried Microsoft Search in Bing, the new enterprise search proved capable of delivering the answers in fewer clicks. A collaboration analyst at a healthcare company stated, “With Microsoft Search in Bing, we are reducing the number of steps it takes to get to the information you need, reducing the number of clicks, and eliminating the need to scour webpages.”

  • Microsoft Search in Bing works across the entire Microsoft 365 ecosystem and is powered by Microsoft Graph. The users are able to sufficiently find what they are looking for by seeing the most relevant results first, which reduces the number of clicks, time to answers, and unnecessary rework. The chief architect at a professional services company said, “One of my senior directors told me that Microsoft Search in Bing found something that had been elusive previously.”

  • At another professional services company, employees frequently look for relevant research that others within the organization have done on a certain topic. The solutions architect explained: “Previously, we would rely on Yammer and broadly ask people around a certain topic within a specific community, and hope that someone replies. But with Microsoft Search in Bing, it’s easy to just go and find this information.”

  • At a healthcare company, reducing the number of clicks to find answers was top priority. “Clinical workers’ time is very valuable; they don’t have time to be digging through pages of search results to find the right policy when they need it. Where in the past they had to do four clicks, it’s one click now because of Microsoft Search in Bing and its Bookmarks capability,” said the collaboration analyst.

  • A consumer goods company found that, on average, employees spent a couple of hours per week searching for information internally. The interviewee estimated that time spent on search could be reduced by 50% to 75% with Microsoft Search in Bing. The organization heavily relied on the use of acronyms, and as stated by the Office 365 architect, “It could be very challenging for somebody new in the organization to figure out what people are talking about.” The organization set up more than 2,000 acronyms in Microsoft Search in Bing to make finding the right definitions easier for users, and as stated by the Office 365 architect, “People are impressed with how well it works.”

Based on the customer interviews, Forrester modeled the financial impact for the composite organization with the following estimates:

  • The composite organization has 50,000 employees.

  • When the IT organization announces Microsoft Search in Bing availability in Year 1, its adoption varies between 5% and 10%. Each year thereafter, adoption doubles as the word of mouth spreads and the organization educates employees on Microsoft Search in Bing’s availability and capabilities.

  • Based on Forrester research, an average employee dedicates 5.28% of its typical workday to searching for relevant work information.

  • Using Microsoft Search in Bing allows employees to reduce time spent on search by 50% to 75%.

  • An average business user's hourly burdened rate is $35.

  • Fifty percent of the total time saved per user is applied directly back to revenue-generating tasks and is therefore included in the benefit calculation. Individual users may apply additional time savings toward professional development, networking, and work-life activities, which are not included in the benefit.

Magnifying glass icon represents Microsoft Search in Bing features
Microsoft Search in Bing features that were frequently mentioned by interviewees:
  • Curated bookmarks help users navigate quickly to internal sites and tools.
  • Q&As give users quick answers without additional clicks.
  • People search helps find someone's role within the company, what they've been working on, org charts, and ways to contact them – even when searched by their nickname.

This yields a three-year projected PV ranging from $13.6 million to $40.7 million

ADDITIONAL BENEFIT CONSIDERATIONS

Microsoft customers discussed the following benefits of using Microsoft Search in Bing to enhance employee productivity. However, these benefits were not quantified as part of the analysis.

  • At a professional services company, Microsoft Search in Bing enabled users to find previously created decks, allowing employees to reuse the existing files, rather than recreate them. As the chief architect explained to Forrester: “When you think about what it is that we do, we’re building decks, we’re building proposals, we’re creating SOWs. We’re doing research and finding preexisting documents. Using Bing could save each user up to 10 hours per week.”

  • Interviewees repeatedly described People search as a feature that saves employees’ time and makes it easier to find the right people. The collaboration analyst at a healthcare company said: “Maybe you know a person’s name but you don’t know their last name, Microsoft Search in Bing is pretty good at finding the right person who is most relevant to you, you can quickly see what their title is and the org chart. Just having the org chart built into the search result is really useful to see how a person fits or into the organization, who they report to, who is on their team.”

"We have lots of folks that will search people and I think that adds the most value. If I’m at an office location, and I can't find someone, I can quickly pull them up, send them a chat message, send an email, or call them directly from the search results."

Core productivity platforms manager, oil and mining

Productivity Savings From Using Microsoft Search In Bing: Calculation Table

Ref. Metric Calc. Year 1 Year 2 Year 3
A1 Number of employees Includes 1% YoY growth 50,000 50,500 51,005
A2Low
A3Mild
A2Hi

Microsoft Search in Bing adoption

Doubles each year
5%
7.5%
10%
10%
15%
20%
20%
30%
40%
A3 Average number of work hours per day 8 8 8
A4 Percentage of time spent on search for relevant information during a workday Forrester data 5.28% 5.28% 5.28%
A5 Hours spent per user per year by every employee A3*260 days*A4 110 110 110
A6Low
A6Mid
A6Hi

Reduction in time spent on search with Microsoft Search in Bing
50%
63%
75%
50%
63%
75%
50%
63%
75%
A7Low
A7Mid
A7Hi

Subtotal: hours saved per employee with Microsoft Search in Bing

A5*A6

55
69
82
55
69
82
55
69
82
A8 Average business user hourly burdened rate $35 $35 $35
A9 Productivity recapture 50% 50% 50%
AtLow
AtMid
AtHi

Productivity savings from using Microsoft Search in Bing

A1*A2*A7*A8*A9
$2,402,400
$4,540,536
$7,207,200
$4,852,848
$9,171,883
$14,558,544
$9,802,753
$18,527,203
$29,408,259

Impact risk is the risk that the business or technology needs of the organization may not be met by the investment, resulting in lower overall total benefits. The greater the uncertainty, the wider the potential range of outcomes for benefit estimates.


Savings From Reducing Burden On The Help Desk

All interviewed organizations saw an opportunity to improve the way they handled employee questions after they activated Microsoft Search in Bing. In most companies, employees were not historically accustomed to relying on enterprise search to find answers to work-related questions, such as finding a corporate holiday calendar or guidance on how to fill out timecards. Instead, employees frequently called the help desk, HR, or other dedicated resources for answers. The Office 365 architect for the consumer goods company said, “Microsoft Search in Bing offers a huge potential to cut down on a support calls.”

  • An oil and mining company is using Microsoft Search in Bing to shift employees away from an ask-first to a search-first mentality, as stated by the core productivity platforms manager, “If we can push people to search first, instead of call first, it can eliminate a lot of requests to the service desk.” The Office 365 architect told Forrester that users frequently submitted simple requests, such as ones for password resets. They used instant messages, calls, or walked in, which still required help-desk engineers’ time. The core productivity platforms manager explained, “In the ideal world, we could eliminate up to 60% of the simple tickets as long as we have the Q&A set up within Microsoft Search in Bing.”

  • For a healthcare organization, at least 10% of the calls received by the help desk were password-related. With Q&A set up with Microsoft Search in Bing, the company is looking to significantly reduce the amount of calls, as stated by the collaboration analyst, “As people around the organization see how easy it is to quickly find information using Microsoft Search in Bing, I think it could really lower the amount of calls we receive.”

  • Similarly, a consumer goods company saw Microsoft Search in Bing as a ticket reduction opportunity. Using Microsoft Search in Bing makes it easy for employees to find information “without having to open up a ticket and wait for some to get back to them,” said the Office 365 architect.

“With Microsoft Search in Bing we are trying to bring the consumer search experience to enterprise search, so that if you just have a few moments to look something up, you just type in the word “[portal]”, you get to our portal and then you’re able to see your timesheet, hours, and benefits.”

Collaboration analyst, healthcare

Based on the customer interviews, Forrester modeled the financial impact for the composite organization with the following estimates:

  • While the use of Microsoft Search in Bing can help eliminate multiple types of help-desk requests, at the time of these interviews the organizations primarily saw a reduction in the number of password reset tickets; therefore, the model relies only on this type of ticket. Readers are encouraged to use other types of support requests in their analysis.

  • Based on research conducted by Forrester, each employee contacts the help desk, on average, twice per year for guidance on how to reset its password.

  • With Microsoft Search in Bing, the organization set up easy-to-find Q&As on how to reset passwords, which eliminates 30% to 60% of the password reset calls and tickets that would normally be directed to the help desk.

  • The average cost for the help desk to reset a password is $31.

Icon of a human representing employee time saved by using Bing for password reset tickets
Employees' reliance on Microsoft Search in Bing to find answers can reduce the number of password reset tickets by 30% to 60%.

This yields a three-year projected PV ranging from $262,454 to $1 million.

ADDITIONAL BENEFIT CONSIDERATION

Several organizations expected employees to submit fewer HR requests as companies ramp up their Q&As and Bookmarks, which would lead to additional reduction in personnel time to answer these questions. However, this benefit was not quantified as part of the analysis.

Impact risk is the risk that the business or technology needs of the organization may not be met by the investment, resulting in lower overall total benefits. The greater the uncertainty, the wider the potential range of outcomes for benefit estimates.

Savings From Reducing Burden On The Help Desk: Calculation Table

Ref. Metric Calc. Year 1 Year 2 Year 3
B1 Number of employees 50,000 50,500 51,005
B2Low
B2Mid
B2Hi

Microsoft Search in Bing adoption
5%
8%
10%
10%
15%
20%
20%
30%
40%
B3 Number of times an average employee contacts the help desk for a password reset Forrester data 2 2 2
B4Low
B4Mid
B4Hi

Reduction to the number of times employees contact the help desk due to Microsoft Search in Bing
30%
45%
60%
30%
45%
60%
30%
45%
60%
B5Low
B5Mid
B5Hi

Subtotal: number of calls to the help desk eliminated with Microsoft Search in Bing

B1*B2*B3*B4
1,500
3,375
6,000
3,030
6,818
12,120
6,121
13,771
24,482
B6 Average fully loaded cost of a help-desk call to reset a password Forrester data $31 $31 $31
BtLow
BtMid
BtHi

Savings from reducing burden on the help desk

B5*B6
$46,500
$104,625
$186,000
$93,930
$211,343
$375,720
$189,739
$426,912
$758,954

Savings From Retiring Former Enterprise Search Solutions

“Traditional search solutions require a lot of effort and oversight from really skilled and knowledgeable resources. But with the power of the Graph, a lot of that heavy lifting is taken care of. We can have a big pile of random documents, and the Graph is able to discern and reveal relationships among them. That makes for a more exact, more relevant, and more personalized experience for the end user.”

Digital workplace architect, education

Most organizations were not new to managing enterprise search by the time they became familiar with Microsoft Search in Bing. Across industries, interviewees told Forrester that they struggled to manage the wealth of content that their organizations created in a way that it could consistently be found and used, and employees expressed their frustration with search in the form of annual feedback.

  • Both an education and a healthcare company invested in multiple tools to improve search, including taxonomy and tagging products. However, the results proved disappointing, as stated by the collaboration analyst at a healthcare company, “We interviewed a leader from each department to ask which words they would use to tag their documents; and each struggled to define the metadata for their own documents.” If the used metadata did not match the search terms used by employees, the information remained undiscoverable.

  • An oil and mining company used a single enterprise search solution. In addition to an annual license fee, the organization paid for the amount of content indexed. The core productivity platforms manager said, “If we would have indexed all the data we had in SharePoint, that would have brought us somewhere in the range of almost $5 million per year.” To manage costs, the organization chose to avoid indexing SharePoint, which meant that one of the primary data storages could not be searched.

  • Regardless of the tools, managing enterprise search required dedicated resources. Interviewed companies assigned one to two FTEs or hired contractors to manage search.

Transitioning to Microsoft Search in Bing enabled the interviewed organizations to stop paying licensing fees on one or more enterprise search tools and to reassign engineering resources back to their core functions.

For the composite organization, Forrester assumes:

  • Prior to switching to Microsoft Search in Bing, the organization paid a total of $300,000 per year in license fees for several tools, including products for taxonomy and tagging.

  • The company assigned 1.5 IT FTEs to manage enterprise search.

  • An IT FTE fully burdened annal salary is $120,000.

Icon of gears representing the cost elimination for licenses and management of legacy search tools
Transition to Microsoft Search in Bing eliminates license and managing costs of legacy search tools.

This yields a three-year projected PV ranging from $13.6 million to $40.7 million.

Savings From Retiring Former Enterprise Search Solutions: Calculation Table

Ref. Metric Calc. Year 1 Year 2 Year 3
C1 License fees paid for search solutions or tools prior to Microsoft Search in Bing 300,000 300,000 300,000
C2 IT FTEs required to manage enterprise search before Microsoft Search in Bing 1.5 1.5 1.5
C3 IT FTE fully burdened annual salary $120,000 $120,000 $120,000
Ct Savings from retiring former enterprise search solutions C1+(C2*C3) $480,000 $480,000 $480,000
Risk adjustment ↓5%
Ctr Savings from retiring former enterprise search solutions (risk-adjusted) $456,000 $456,000 $456,000

Unquantified Benefits

Through the interviews with Microsoft Search in Bing customers, Forrester identified the following unquantified benefits:

"Our field employees can access Microsoft Search in Bing on the go on their mobile devices and look up something to immediately use it to help the customer. So, our consultants in the field are now more efficient, because they’re able to find information quickly and easily.”

Chief architect, professional services company

  • Improved employee experience.

    Prior to leveraging Microsoft Search in Bing, IT organizations struggled to define the right metadata to tag files. On the other hand, even with proper tagging, only a subset of files would be truly relevant to an employee within a certain team, geography, or role. For example, an employee in the UK might search for vacation policy and the search would yield US holidays. Sorting through search results that did not directly apply to an employee took extra time and added frustration. With its reliance on AI and Microsoft Graph, Microsoft Search in Bing was able to address the issue both for IT and the end user. The digital workplace architect for the educational institution explained: “What intrigued me personally about Microsoft Search in Bing, is its ability to use the details about the documents and the relationships to people searching for them through the Microsoft Graph. There is immediate understanding of the intent of the searcher, and the results will be automatically filtered, promoted, and personalized for that user. The whole idea is fascinating.”

  • Consistency across environments.

    Prior to Microsoft Search in Bing, enterprise users had to know where to start searching for a file or data for their search to be fruitful. The solutions architect at a professional services organization said, “In the past, you go to all these different tools, and what you get back depends upon the product you are in. Having the same capabilities across all the tools where people are working changes the experience.” As a search available in a browser, Microsoft Search in Bing delivers the same search experience across devices, making field employees more productive.

  • Stronger security and compliance.

    Protection of organizations’ information is top priority for IT and security teams. Using Microsoft Search in Bing helped companies identify existing data security vulnerabilities and address them before any damage was done. The core productivity platforms manager at an oil company said: “Our data governance team is always on the lookout for things like a SharePoint site or a OneDrive file containing personally identifiable information that is open to everyone within the company. They are regularly using Microsoft Search in Bing to go out and perform random searches to find that type of information. If they find it, we can take action upon this, whereas before, we couldn’t do that.”

  • Better business outcomes and collaboration.

    The solutions architect for a professional services company shared: “When people are searching on specific technologies, or how to do something, they’ll get those results back internally pointing them to experts and what they’ve done in the past for other clients.” In the past, this overlap would have gone unnoticed. Microsoft Search in Bing enables employees to quickly identify content that’s relevant to their work, which could lead to more cross-team knowledge share, better collaboration across teams, business units, and geographies, and ultimately, better outcomes.


“We want to make the wealth of IT knowledge within our ticketing system easily available to IT. But for an employee, if you put in a request and you don’t hear anything for a day, you may want to check the status. You probably won’t remember the ticket number, but you should be able to quickly find the ticket and see the status in the system.”

Core productivity platforms manager, oil and mining

Flexibility

The value of flexibility is clearly unique to each customer, and the measure of its value varies from organization to organization. There are multiple scenarios in which a customer might choose to implement Microsoft Search in Bing and later realize additional uses and business opportunities, including:

  • Driving further employee adoption.

    All interviewees agree that high adoption levels are key to seeing Microsoft Search in Bing’s full potential. Early attempts to inform users about Microsoft Search in Bing via intranet and email announcements drove early adopters to use the new search, however, more effort would be required to achieve a critical mass of users. Some companies plan to make Microsoft Search in Bing a default search engine, regardless of the browser, which would enforce usage. Others are considering more creative ways of convincing users to try Microsoft Search in Bing, such as short entertaining videos highlighting the benefits of the new search.

  • Expanding use with connectors.

    Enabling Microsoft Search in Bing within learning and knowledge management systems or other data sources, such as ticket management, is the next step for most interviewees. Making these new types of information searchable could have a big impact on utilizing the existing knowledge within each organization.

Flexibility would also be quantified when evaluated as part of a specific project (described in more detail in Appendix A).

Flexibility, as defined by TEI, represents an investment in additional capacity or capability that could be turned into business benefit for a future additional investment. This provides an organization with the “right” or the ability to engage in future initiatives but not the obligation to do so.


Analysis Of Costs

QUANTIFIED COST DATA AS APPLIED TO THE COMPOSITE

Total Costs

Ref. Cost Initial Year 1 Year 2 Year 3 Total Present Value
Dtr Implementation, management, and promotion $23,673 $7,362 $6,981 $6,981 $44,996 $41,379
Total costs (risk-adjusted) $23,673 $7,362 $6,981 $6,981 $44,996 $41,379

The table above shows the total of all costs across the areas listed below, as well as present values (PVs) discounted at 10%. Over three years, the composite organization expects risk-adjusted total costs to be a PV of $41,379.


Implementation, Management, And Promotion

Dollar sign representing minimal effort to enable, manage and drive adoption of Microsoft Search in Bing
Microsoft Search in Bing is available at no charge to Microsoft 365 customers and requires minimal effort to enable, manage, and drive adoption.

Interviewed organizations noted the costs of implementing, managing and promoting Microsoft Search in Bing.

Based on the customer interviews and the composite organization's deployment size, Forrester modeled the financial impact for the composite organization with the following estimates:

  • Two IT FTEs will be involved in setting up Microsoft Search in Bing for one month. Microsoft Search in Bing is instantly available to all Microsoft 365 customers and will function without any input from IT, however, the organization chose to spend time upfront performing security testing and setting up Bookmarks, Q&As, and Acronyms for enhanced performance.

  • After the initial setup, the same two IT FTEs dedicated 1 hour per week each to review usage statistics and make changes to the content as needed.

  • In addition to IT, one FTE from a change management group spent a week preparing for the Microsoft Search in Bing launch to ensure awareness among employees. The organization posted on Yammer, sent out an email announcing the new capability, and created an intranet page dedicated to Microsoft Search in Bing.

  • As the organization prepares to make Microsoft Search in Bing its default search, no significant informational campaigns are planned beyond implementation, but the change management group will still monitor adoption and design periodic promotional material to drive adoption. In Year 1, the effort will require 1 hour per month, then half an hour per month in Years 2 and 3.

This cost will vary based on:

  • The complexity of organizing and preparing the transition to Microsoft Search in Bing.

  • The number and salaries of FTEs dedicated to implementation, management, and driving adoption.

To account for these risks, Forrester adjusted this cost upward by 10%, yielding a three-year, risk-adjusted total PV of $41,379.

Implementation risk is the risk that a proposed investment may deviate from the original or expected requirements, resulting in higher costs than anticipated. The greater the uncertainty, the wider the potential range of outcomes for cost estimates.

Implementation, Management, And Promotion: Calculation Table

Ref. Metric Calc. Initial Year 1 Year 2 Year 3
D1 Number of IT FTEs involved in setting up Microsoft Search in Bing 2 2 2 2
D2 Time needed to prepare Microsoft Search in Bing for enterprise launch (months) 1 - - -
D3 Time needed to manage Microsoft Search in Bing (hours) 1 hour*52 weeks - 52 52 52
D4 IT FTE fully burdened annual salary $120,000 $120,000 $120,000 $120,000
D5 Subtotal: IT cost of implementing and managing Microsoft Search in Bing Time dedicated*# of FTEs*cost of resource $20,000 $6,000 $6,000 $6,000
D6 Number of change management FTEs involved in Microsoft Search in Bing adoption effort 1 1 1 1
D7 Time dedicated by change management FTEs to Microsoft Search in Bing adoption effort (months) 0.25 - - -
D8 Time dedicated by change management FTEs to Microsoft Search in Bing adoption (hours) An hour per month in Y1, then .5 hour per month - 12 6 6
D9 Change management FTE fully burdened annual salary $73,000 $73,000 $73,000 $73,000
D10 Subtotal: cost of driving Microsoft Search in Bing adoption Time dedicated* # of FTEs* cost of resource $1,521 $692 $346 $346
Dt Implementation, management, and promotion D5+D10 $21,521 $6,692 $6,346 $6,346
Risk adjustment ↑10%
Dtr Implementation, management, and promotion (risk-adjusted) $23,673 $7,362 $6,981 $6,981
ADDITIONAL PRICING CONSIDERATIONS

Microsoft Search in Bing is available at no charge to Microsoft 365 customers. Therefore, the composite organization does not incur an additional license fee. As one customer put it: “It doesn’t cost us anything. Regardless of what tools you are using to enable search today, they are not going to provide your users with the search experience Microsoft Search in Bing delivers for free.”


Financial Summary

CONSOLIDATED THREE-YEAR RISK-ADJUSTED METRICS
  • Circular representing recalculation

    The risk-adjusted projected NPV values are determined by applying risk-adjustment factors to the unadjusted results in each Benefit and Cost section.

Cash Flow Chart (Risk-Adjusted)

Cash flow chart shows three-year risk-adjusted financial analysis for the composite organization

Cash Flow Table (Risk-Adjusted)

Initial Year 1 Year 2 Year 3 Total Present Value
Total costs ($23,673) ($7,362) ($6,981) ($6,981) ($44,996) ($41,379)
Total benefits (low) $0 $2,904,900 $5,402,778 $10,448,492 $18,756,170 $14,956,031
Total benefits (mid) $0 $5,101,161 $9,839,225 $19,410,115 $34,350,501 $27,352,117
Total Benefits (hi) $0 $7,849,200 $15,390,264 $30,623,213 $53,862,677 $42,862,537
Net benefits (low) ($23,673) $2,897,538 $5,395,797 $10,441,511 $18,711,174 $14,914,652
Net benefits (mid) ($23,673) $5,093,799 $9,832,244 $19,403,134 $34,305,505 $27,310,738
Net benefits (hi) ($23,673) $7,841,838 $15,383,283 $30,616,233 $53,817,681 $42,821,158

The financial results calculated in the Benefits and Costs sections can be used to determine the PROI, NPV, and payback period for the composite organization’s investment. Forrester assumes a yearly discount rate of 10% for this analysis.

logo

Microsoft Search in Bing: Overview

The following information is provided by Microsoft. Forrester has not validated any claims and does not endorse Microsoft or its offerings.

Search your workplace just like you would the web.

Organizations are creating and storing more data, information, and content than ever before and it’s only accelerating. This creates knowledge-sharing challenges that search is uniquely positioned to solve. When you need answers, you can use Microsoft Search in Bing to save time to find people, files, org charts, internal sites, and more.

Microsoft Search in Bing uses insights from the Microsoft Graph to search across your Office 365 content and show the most relevant results to each user. This means each user might see different results, even if they search for the same words, and users only see results they already have access to. More on how Microsoft Search in Bing protects company data can be found here.

Best of all, for Microsoft 365 customers, Microsoft Search in Bing is on by default. No initial setup is required and there’s no additional cost. Just sign in with your work account (AAD) and use Bing to search things like “my team.” You can enhance the search results by adding content such as bookmarks, Q&As, acronyms, building locations, and floor plans. For more information, see: How to prepare your organization for Microsoft Search in Bing.

When you choose to set Bing as the default search engine for your organization, you’ll have Microsoft Search in Bing integrated directly in the address bar where you already search. Now, they’ll find the right answers, people, and content, and your organization can realize more time savings and reduced burden on IT than with other search engines. See: How to set Bing as your default search engine.

We’re excited about the impact Microsoft Search in Bing is having and believe it’s only just scratching the surface. We’ll be continuing to add to our list of connectors so you can access your other data sources. We’re also expanding our feature set to make even more information discoverable. Organizations need new ways to improve knowledge sharing and are placing Microsoft Search in Bing at the center. Try it today and transform your company culture from “ask first” to “search first”.

To contact , please click here.

Microsoft Search in Bing: Overview (continued)

The following information is provided by Microsoft. Forrester has not validated any claims and does not endorse Microsoft or its offerings.

Microsoft Search In Bing Features & Definitions

Bookmarks Discover internal sites and tools more easily with bookmarks that are triggered by a set of several keywords ("my benefits", "IT help","time off").
Acronyms Discover meanings to acronyms in your company by searching “what is __.” Both mined and editorial acronyms are available.
Floorplans Easily locate places and people by searching geography, title, or job function when you're traveling or away from your desk.
Locations Easily find addresses and locate your organization's buildings by providing accurate locations for offices, campuses, and buildings, along with directions and navigation.
Book a room Search "book a room" and available rooms in your location can reserved with one click.
Enterprise Vertical Bing.com now has a dedicated place to search for your enterprise content.
MSB on mobile Microsoft Search in Bing features and results are available in the Bing mobile and Edge mobile apps with your AAD sign-in.
People search with attributes Look up people in your org by their title, their location, or their discipline, even if you only have a first name, powered by the Microsoft Graph.
People search — Who Knows What Surface relevant people for a topic or skill, based on personal, organization-wide, or world content to easily discover SMEs within your company (coming soon, requires Project Cortex E5 addon).
New Microsoft Edge integration (NTP/QF) Start typing and query formulation returns results personalized to you. The graph continually collects and analyzes signals while you’re working in Office 365 and surfaces the content most relevant to you.
Connectors 130+ connectors available can find your enterprise content from Microsoft and beyond.
Turing Ask any question via search and deep learning models and machine reading comprehension will understand what you’re looking for and provide relevant answers.
Q&A Similar to creating a bookmark, admins can create Q&As to provide an immediate answer triggered by users' questions rather than just providing a link to a webpage.
Power BI Surface specific Power BI reports and results with a single search.
Bing enterprise homepage The Bing enterprise homepage centralizes information from your company like popular bookmarks—as well as your documents from Office—along with fresh news related to your industry.
Enterprise news Enterprise news from Bing in an industry news email serves relevant, fresh, and high-value news about your company, your industry, and your career in a trusted and compliant way.
Analytics Provides IT admins with search usage through the M365 Admin Portal.

This study was conducted by Forrester’s Custom Content Consulting Practice. Get in touch to discuss your custom content needs and goals. Learn more about all of Forrester's consulting capabilities.

logo

Appendix A: New Technology:Projected Total Economic Impact

New Technology: Projected Total Economic Impact (New Tech TEI) is a methodology developed by Forrester Research that enhances a company’s technology decision-making processes and assists vendors in communicating the value proposition of their products and services to clients. The New Tech TEI methodology helps companies demonstrate and justify the projected tangible value of IT initiatives to both senior management and other key business stakeholders.

Total Economic Impact Approach

  • Benefits icon

    Projected Benefits represent the projected value to be delivered to the business by the product. The New Tech TEI methodology places equal weight on the measure of projected benefits and the measure of projected costs, allowing for a full examination of the effect of the technology on the entire organization.

  • Costs icon

    Projected Costs consider all expenses necessary to deliver the proposed value, or benefits, of the product. The projected cost category within New Tech TEI captures incremental costs over the existing environment for ongoing costs associated with the solution.

  • Flexibility icon

    Flexibility represents the strategic value that can be obtained for some future additional investment building on top of the initial investment already made. Having the ability to capture that benefit has a PV that can be estimated.

  • Risks icon

    Risks measure the uncertainty of benefit and cost estimates given: 1) the likelihood that estimates will meet original projections and 2) the likelihood that estimates will be tracked over time.

  • Benefits PV icon
    PRESENT VALUE (PV)

    The present or current value of (discounted) cost and benefit estimates given at an interest rate (the discount rate). The PV of costs and benefits feed into the total NPV of cash flows.

  • NPV icon
    NET PRESENT VALUE (NPV)

    The present or current value of (discounted) future net cash flows given an interest rate (the discount rate). A positive project NPV normally indicates that the investment should be made, unless other projects have higher NPVs.

  • ROI icon
    PROJECTED RETURN ON INVESTMENT (PROI)

    A project’s expected return in percentage terms. PROI is calculated by dividing net projected benefits (projected benefits less costs) by projected costs.

  • Discount Rate icon
    DISCOUNT RATE

    The interest rate used in cash flow analysis to take into account the time value of money. Organizations typically use discount rates between 8% and 16%. A 10% discount rate is used for this analysis.

The initial investment column contains costs incurred at “time 0” or at the beginning of Year 1 that are not discounted. All other cash flows are discounted using the discount rate at the end of the year. PV calculations are calculated for each total cost and benefit estimate. NPV calculations in the summary tables are the sum of the initial investment and the discounted cash flows in each year. Sums and present value calculations of the Total Benefits, Total Costs, and Cash Flow tables may not exactly add up, as some rounding may occur.


Appendix B: Endnotes

1 Source: “The Forrester Wave™: Cognitive Search And Knowledge Discovery Solutions, Q2 2017 (Updated),” Forrester Research, Inc., June 26, 2018.

2 Source: “The CIO’s Guide To Employee Experience,” Forrester Research, Inc., April 8, 2019.

3 More than 61% of large enterprises report storing more than 100 terabytes of data. Source: “The Forrester Wave™: File Analytics Providers, Q2 2018,” Forrester Research, Inc., June 13, 2018.

4 Source: “The Forrester Wave™: Cognitive Search And Knowledge Discovery Solutions, Q2 2017 (Updated),” Forrester Research, Inc., June 26, 2018.

5 Source: “The Employee Experience Technology Ecosystem,” Forrester Research, Inc., February 14, 2019.