Big data analysis Essay

THE NEW INTELLIGENT ENTERPRISE Some of the best-performing retailers ar using uninflecteds not just for finance and functional activities, but to boost competitive advantage on e realthing from displays, to grocerying, customer benefit and customer experience focal point. Big Data, Analytics and the Path From Insights toValue How the smartest organizations are embedding analyticals to transform teaching into sagacity and then action. Findings and recommendations from the introductory annual brand-new Intelligent Enterprise Global Executive study. BY STEVE LAVALLE, ERIC LESSER, REBECCA SHOCKLEY, MICHAEL S.HOPKINS AND NINA KRUSCHWITZ IN EVERY INDUSTRY, in alwaysy part of the world, fourth-year leaders wonder whether they are getting full take to be from the massive amounts of randomness they already have at heart their organizations. brand-new technologies are collecting to a greater extent selective nurture than ever before, yet many organizations are still looking f or get around charges to obtain pass judgment from their info and compete in the marketplace. Their questions slightly how best to achieve value persist. Are competitors obtaining sharper, more timely insights? Are they able to regain market advantage, neglected while center oning on expenses during the erstwhile(prenominal) two years?Are they correctly interpreting new signals from the global economy and adequately assessing the impact on their customers and partners? Knowing what happened and why it happened are no longer adequate. Organizations accept to know what is happening now, what is potential to happen next and what actions should be pressn to get the optimal results. COURTESY OF BEST BUY THE in the lead QUESTION How are organizations using analytics to gain insight and guide action? FINDINGS Top-performingorganizations are twice as likely to apply analytics to activities. Thebiggest ch totallyenges in adopting analytics are managerial and cultural.V isualizing entropy differently allow become increasingly blue-chip. WINTER 2011 MIT SLOAN focusing REVIEW 21 THE NEW INTELLIGENT ENTERPRISE ABOUT THE investigate To empathise the challenges and opport building blockies associated with the practice session of problem analytics, MIT Sloan Management look into, in collaboration with the IBM Institute for Business Value, conducted a adopt of more than 3,000 byplay purpose makers, managers and analysts from organizations located around the world. The survey captured insights from individuals in 108 countries and more than 30 industries and involved organizations from a variety of sizes.The sample was drawn from a number of different sources, including MIT alumni and MIT Sloan Management Review subscribers, IBM clients and other aro single-valued function parties. We also interviewed academic experts and subject matter experts from a number of industries and discip take ups to record the practical issues facing organizations today. Th eir insights contributed to a richer understanding of the selective information and the breeding of recommendations that respond to strategic and tactical questions that senior executives address as they operationalize analytics in spite of appearance their organizations. We also drew upon a number of IBMcase studies to explore further how organizations are leveraging line of work analytics and illuminate how real organizations are be sickting our recommendations into action in different organizational settings. To help organizations understand the opportunity of information and advanced analytics, MIT Sloan Management Review partnered with the IBM Institute for Business Value to conduct a survey of nearly 3,000 executives, managers and analysts working crosswise more than 30 industries and 100 countries. (See About the Research. ) Among our key findings Top-performing organizations employment analytics five times more than lower performers.(See Analytics Trumps Intuition. ) Overall, our survey found a widespread belief that analytics offers value. Half of our respondents said that improvement of information and analytics was a transgress priority in their organizations. And more than one in five said they were under intense or signifi substructuret pressure to adopt advanced information and analytics approaches. The source of the pressure is not hard to ascertain. Six out of 10 respondents cited innovating to achieve competitive differentiation as a guide business challenge. The identical percentage also agreed that their organization has more information than it ordure use potently.Organizational leaders want analytics to exploit their bring uping data and computational power to get smart, and get innovative, in ways they never could before. Senior executives now want businesses run on data-driven decisions. They want scenarios and simulations that provide immediate pleader on the best actions to take when disruptions occur disruptions rangin g from unexpected competitors or an earthquake in a supply zone to a customer signaling a desire to switch providers. Executives want to understand optimal solutions based on building complex business parameters or new information, and they want to take action quickly.These expectations can be met but with a caveat. For analytics-driven insights to be consumed that is, to trigger new actions across the organization they must be closely linked to business strategy, easy for end-users to understand and infix into organizational growthes so that action can be taken at the right time. That is no small task. It requires painstaking focus on the way insights are infused into everything from manufacturing and new product development to credit approvals and call center interactions. 22 MIT SLOAN MANAGEMENT REVIEW WINTER 2011 Top Performers Say Analytics Is a discriminatorOur study clearly connects performance and the competitive value of analytics. We asked respondents to assess their organizations competitive position. Those who selected materially outperform industry peers were identified as pass along performers, while those who selected somewhat or substantially underperform industry peers were grouped as lower performers. We found that organizations that strongly agreed that the use of business information and analytics differentiates them within their industry were twice as likely to be top performers as lower performers. Top performers approach business operationsdifferently than their peers do. Specifically, they put analytics to use in the widest possible range of decisions, large and small. They were twice as likely to use analytics to guide future strategies, and twice as likely to use insights to guide day-to-day operations. (See The Analytics Habits of Top Performers, p. 24. ) They make decisions based on blotto analysis at more than double the rate of lower performers. The correlation amidst performance and analyticsdriven management has importa nt implications to organizations, whether they are seeking growth, efficiency or competitive differentiation.Three Levels of Capabilities Emerged, Each with Distinct Opportunities Organizations that know where they are in terms of analytics adoption are better prepared to turn challenges into opportunities. We segmented respondents based on how they rated their organizations analytics prowess, particular(prenominal)ally how thoroughly their organizations had been transformed by better uses of analytics and information. Three levels of analytics efficacy emerged Aspirational, Experienced and Transformed each with clear distinctions. (See The Three Stages of Analytics Adoption. ) Aspirational.These organizations are the furthest from achieving their desired analytical goals. Often they are focusing on efficiency or automation of breathing processes and searching for ways to cut costs. Aspirational organizations currently have SLOANREVIEW. MIT. EDU few of the necessary building blocks people, processes or tools to collect, understand, incorporate or act on analytic insights. Experienced. Having gained some analytic experience oft done successes with efficiencies at the Aspirational phase these organizat ions are lo oking to go b e yond cost management.Experienced organizations are developing better ways to collect, incorporate and act on analytics effectively so they can begin to optimize their organizations. Transformed. These organizations have substantial experience using analytics across a broad range of functions. They use analytics as a competitive differentiator and are already adept at organizing people, processes and tools to optimize and differentiate. Transformed organizations are less focused on cutting costs than Aspirational and Experienced organizations, possibly having already automated their operations through effective use of insights.They are most focused on driving customer profitability and making targeted investments in niche an alytics as they keep pushing the organizational envelope. Transformed organizations were threesome times more likely than Aspirational organizations to indicate that they substantially outperform their industry peers. This performance advantage illustrates the potential rewards of higher levels of analytics adoption. Information moldiness Become Easier to Understand and Act Upon Executives want better ways to communicate complex insights so they can quickly absorb the meaning of the data and take action.Over the next two years, executives say they will focus on supplementing standard historical reporting with appear approaches that make information come alive. These admit data visualization and process simulation as well as text and voice analytics, social media analysis and other predictive and prescriptive techniques. New tools like these can make insights easier to understand and to act on at every point in an organization, and at every skill level. They transform numbers int o information and insights that can be readily put to use, versus having to rely on furtherinterpretation or leaving them to languish due to uncertainty nigh how to act. ANALYTICS TRUMPS INTUITION The tendency for top-performing organizations to apply analytics to particular activities across the organization compared with lower performers. A likeliness of 1. 0 indicates an equal likelihood that the organizations will use all analytics or intuition. Tendency to Apply Tendency to Apply Intuition Analytics Financial management and budgeting Data Is Not the Biggest Obstacle Despite popular opinion, getting the data right is not a top challenge that organizations face when adopting analytics.Only about one out of five respondents cited concern with data quality or ineffective data governance as a primary obstacle. The adoption barriers that organizations face most are managerial and cultural earlier than related to data and technology. The leading obstacle to widespread analytics ad option is lack of understanding of how to use analytics to improve the business, according to almost four of 10 respondents. More than one in three cite lack of management bandwidth due to competing priorities. (See The Impediments to Becoming More Data Driven. ) Strategy and business development Sales and marketing guest service Product research and development Top Performers Lower Performers widely distributed management Risk management Customer experience management Brand or market management Work force planning and allocation Overall Average 0 SLOANREVIEW. MIT. EDU 22. 1 Operations and production 1 2 3 4 5 6 7 8 WINTER 2011 MIT SLOAN MANAGEMENT REVIEW 23 THE NEW INTELLIGENT ENTERPRISE What Leaders Can Do to exonerate Analytics Pay Off A New Methodology It takes big plans followed by distinguishable actions to gain the benefits of analytics. But it also takes some very circumstantial management approaches. Based on data from our survey, our engagementexperience, case studies and interviews with experts, we have been able to identify a new, five-point methodology for successfully implementing analytics-driven management and for rapidly creating value. The recommendations that follow are designed to help organizations understand this new path to value and how to travel it. While each recommendation presents different pieces of the information-and-analytics value puzzle, each one meets all of these three critical management needs Reduced time to value. Value creation can be achieved early in an organizations get on to THE ANALYTICS HABITS OF TOP PERFORMERSTop-performing organizations were twice as likely to use analytics to guide day-to-day operations and future strategies as lower performers. THE THREE STAGES OF ANALYTICS ADOPTION Three capability levels Aspirational, Experienced and Transformed were based on how respondents rated their organizations analytic prowess. ASPIRATIONAL EXPERIENCED TRANSFORMED Motive Use analytics to justify actions Use ana lytics to guide actions se analytics to prescribe actions U Functional proficiency Financial management and budgeting Operations and production Sales and marketing All Aspirational functionsStrategy/business development Customer service Product research/development ll Aspirational and Experienced A functions Risk management Customer experience Work force planning/allocation General management Brand and market management Business challenges ompetitive differentiation through C innovation Cost efficiency (primary) Revenue growth (secondary) ompetitive differentiation through C innovation Revenue growth (primary) Cost efficiency (secondary) ompetitive differentiation through C innovation Revenue growth (primary) rofitability acquiring/retaining P customers (targeted focus) aboriginalobstacles ack of understanding how to leverage L analytics for business value Executive sponsorship ulture does not encourage sharing C information ack of understanding how to leverage L analytics for business value Skills within line of business wnership of data is unclear or O governance is ineffective ack of understanding how to leverage L analytics for business value anagement bandwidth due to M competing priorities Accessibility of the data Data management imited ability to capture, aggregate, L analyze or share information and insights oderate ability to capture, aggregate M and analyze data imited ability to share information and L insights trong ability to capture, aggregate and S analyze data ffective at sharing information and E insights Analytics in action arely use rigorous approaches to R make decisions imited use of insights to guide future L strategies or day-to-day operations ome use of rigorous approaches to S make decisions rowing use of insights to guide future G strategies, but still limited use of insights to guide day-to-day operations ost use rigorous approaches to make M decisions lmost all use insights to guide future A strategies, and m ost use insights toguide day-to-day operations 24 MIT SLOAN MANAGEMENT REVIEW WINTER 2011 SLOANREVIEW. MIT. EDU analytics sophistication. Contrary to common assumptions, it doesnt require the presence of perfect data or a full-scale organizational transformation. Increased likelihood of transformation thats both significant and enduring. The emerging methodology weve identified enables and inspires lasting commute (strategic and cultural) by tactically overcoming the most significant organizational impediments. greater focus on achievable steps. The approach used by the smartest companies is powerful in part because each step enables leaders to focustheir efforts and visions narrowly rather than implementing universal spays making every step easier to accomplish with an attractive ROI. Whether pursuing the best line of merchandise strategy, the best customer experience, the best portfolio or the best process innovation, organizations embracing this approach will be first in lin e to gain business advantage from analytics. have repeatedly heard that analytics aligned to a significant organizational challenge makes it easier to overcome a wide range of obstacles. Respondents cited many challenges, and none can be discounted or minimized Executive sponsorshipof analytics projects, data quality and access, governance, skills and culture all matter and need to be addressed in time. But when overtaken by the momentum of a single big belief and potentially game-changing insight, obstacles like these get swept into the wake of change rather than drowning the effort. THE IMPEDIMENTS TO BECOMING MORE DATA DRIVEN The adoption barriers organizations face most are managerial and cultural rather than related to data and technology. Lack of understanding of how to use analytics to improve the business Lack of management bandwidth due to competing prioritiesLack of skills internally in the line of business Ability to get the data RECOMMENDATION 1 First,Think Biggest Exi sting culture does not encourage sharing information Focus on the biggest and highestvalue opportunities Does attacking the biggest challenge carry the biggest risk of failure? Paradoxically, no because big problems command attention and incite action. And as survey participants told us, management bandwidth is a top challenge. When a projects stakes are big, top management gets invested and the best talent seeks to get involved. Its extraordinarily hard for people to changefrom making decisions based on personal experience to making them from data especially when that data counters the prevailing common wisdom. But upsetting the status quo is lots easier when everyone can see how it could contribute to a major goal. With a potential big reward in sight, a significant effort is easier to justify, and people across functions and levels are better able to support it. Conversely, dont mystify doing analytics without strategic business direction, as those efforts are likely to stall. Not only does that fling off resources, it risks creating widespread skepticism about the real value of analytics.In our discussions with business executives, we SLOANREVIEW. MIT. EDU Ownership of data is unclear or governance is ineffective Lack of executive sponsorship Concerns with the data Perceived costs outweigh projected benefits No case for change Respondents were asked to select three obstacles to the widespread adoption of analytics in their organization. Dont know where to start 0 10% 20% 30% 40% percentage of respondents RECOMMENDATION 2 Start in the Middle Within each opportunity, start with questions, not data Organizations traditionally are tempted to start by gathering all available data before beginning theiranalysis. Too often, this leads to an all-encompassing focus on data management collecting, cleansing and converting data that leaves little time, energy or resources to understand its potential uses. Actions taken, if any, might not be the most valuable o nes. Instead, organizations should WINTER 2011 MIT SLOAN MANAGEMENT REVIEW 25 THE NEW INTELLIGENT ENTERPRISE start in what might seem like the middle of the process, implementing analytics by first defining the insights and questions needful to meet the big business objective and then identifying those pieces of data needed for answers.By defining the desired insights first, organizations can target specific subject areas and use readily available data in the initial analytic models. The insights delivered through these initial models will illuminate gaps in the data substructure and business processes. Time that would have been spent cleaning up all data can be redirected toward targeted data needs and specific process improvements that the insights identify, enabling iterations of value. Companies that make data their overriding priority often lose momentum long before the first insight is delivered, a great deal because adata-first approach can be perceived as taking too long before generating a financial return. By narrowing the scope of these tasks to the specific subject areas needed to answer key questions, value can be realized more quickly, while the insights are still relevant. Also, organizations that start with the data or process change often end up with unintended consequences such as data that is not extensible or processes that are ultimately eliminated that require rework and redundant resources to solve. Speeding Insights into Business Operations Compared with other respondents, Transformed organizations are good at data capture.(See What Data-Transformed Companies Do. ) Additionally, Transformed organizations are much more adept at WHAT DATA-TRANSFORMED COMPANIES DO Transformed organizations felt more confident in their ability to manage data tasks than Aspirational organizations, which seldom felt their organizations performed those tasks very well. Percent of respondents whose organizations perform these tasks very well. Capture In formation Transformed Aspirational 9% Aggregate Information 36% 4X more likely Analyze Information 28% 3% 9X more likely 26 MIT SLOAN MANAGEMENT REVIEW WINTER 2011 Disseminate Information and Insights34% 4% 8. 5X more likely 21% 2% 10X more likely data management. In these areas, they outpaced Aspirational organizations up to tenfold in their ability to execute. Enterprise processes have many points where analytic insights can boost business value. The operational challenge is to understand where to apply those insights in a particular industry and organization. When a bank customer stops automatic payroll deposits or remittance transfers, for example, who in the organization should be alerted and tasked with finding out whether the customer is changing jobs or planning to switch banks? Where customersatisfaction is low, what insights are needed, and how should they be delivered to prevent defections? To keep the three gears moving together data, insights and timely actions the ov erriding business purpose must always be in view. That way, as models, processes and data are tested, priorities for the next investigating become clear. Data and models get accepted, rejected or improved based on business need. New analytic insights descriptive, predictive and prescriptive are embedded into increasing numbers of applications and processes, and a virtuous cycle of feedback and improvement takes hold.RECOMMENDATION 3 Make Analytics Come Alive Embed insights to drive actions and deliver value New methods and tools to embed information into business processes use cases, analytics solutions, optimization, work flows and simulations are making insights more understandable and actionable. Respondents identified trend analysis, forecasting and standardized reporting as the most important tools they use today. However, they also identified tools that will have greater value in 24 months. The downswings in as-is methods accompanied by correspondent upswings in to-be m ethods were dramatic.(See Where Are DataDriven Managers Headed? p. 27. ) Todays staples are expected to be surpassed in the next 24 months by 1. Data visualization, such as dashboards and scorecards SLOANREVIEW. MIT. EDU 2. Simulations and scenario development 3. Analytics applied within business processes 4. Advanced statistical techniques, such as regression analysis, discrete choice modeling and mathematical optimization. Organizations expect the value from these emerging techniques to soar, making it possible for data-driven insights to be used at all levels of the organization.For example, GPS-enabled navigation devices can superimpose real-time traffic patterns and alerts onto navigation maps and suggest the best routes to drivers. Similarly, in oil exploration, three-dimensional renderings combine data from sensors in the field with collaborative and analytical resources accessible across the enterprise. Production engineers can incorporate geological, production and argume ntation information into their drilling decisions. Beyond 3-D, animated maps and charts can simulate critical changes in distribution flow or projected changes in consumption and resource availability.In the emerging area of analytics for unstructured data, patterns can be visualized through verbal maps that pictorially represent word frequency, allowing marketers to see how their brands are perceived. advanced(a) uses of this type of information layering will continue to grow as a means to help individuals across the organization consume and act upon insights derived through complex analytics that would otherwise be hard to piece together. New Techniques and Approaches Transform Insights into Actions New techniques to embed insights will gain in value by generating results that can be readily understood and acted upon Dashboards that now reflect actual last-quarter sales will also show what sales could be next quarter under a variety of different conditions a new media mix, a pri ce change, a larger sales team, even a major weather or clear event. Simulations evaluating alternative scenarios will automatically recommend optimal approaches such as the best media mix to introduce a specific product to a specific segment, or the ideal number of sales professionals to assign to a particular new territory. Use cases will illustrate how to embed insights into business applications and processes. SLOANREVIEW.MIT. EDU New methods will also make it possible for decision makers more fully to see their customers purchases, payments and interactions. Businesses will be able to listen to customers unique wants and needs about channel and product preferences. WHERE ARE DATA-DRIVEN MANAGERS HEADED? Organizations expect that the ability to visualize data differently will be the most valuable technique in two years. otherwise techniques and activities that are currently delivering the most value today will still be done, but will be of less value. Today In 24 Months His toric trend analysis and forecastingData visualization Standardized reporting Simulations and scenario development Analytics applied within business processes Data visualization degeneration analysis, discrete choice modeling and mathematical optimization Analytics applied within business processes Simulations and scenario development Historic trend analysis and forecasting Clustering and class Clustering and segmentation Regression analysis, discrete choice modeling and mathematical optimization Standardized reporting Respondents were asked to identify the top three analytic techniques creating value for the organization,and predict which three would be creating the most value in 24 months. In fact, making customers, as well as information, come to life within complex organizational systems may well become the biggest benefit of making data-driven insights real to those who need to use them. RECOMMENDATION 4 Add, Dont Detract appreciation existing capabilities while adding new ones When executives first realize their need for analytics, they tend to turn to those closest to them for answers. Over time, these point-of-need resources come together in local line of business units to enable sharing of insights.Ultimately, centralized units emerge to bring a shared enterprise perspective governance, tools, methods and specialized expertise. As executives use analytics more frequently to inform day-to-day decisions and actions, WINTER 2011 MIT SLOAN MANAGEMENT REVIEW 27 THE NEW INTELLIGENT ENTERPRISE this increasing demand for insights keeps resources at each level engaged, expanding analytic capabilities even as activities are shifted for efficiencies. (See How Analytics Capabilities Grow with Adoption. ) Sophisticated modeling and visualization tools, as noted, will soon provide greater business value thanever before. But that does not mean that spreadsheets and charts should go away. On the contrary New tools should supplement earlier ones or continue to be used side by side as needed. That lesson applies to plines. (See How Analytics Propagates Across Functions. ) In Transformed organizations, reusability creates a snowball effect, as models from one function are repurposed into another with minimal modifications. Over time, data-driven decision making branches out across the organization. As experience and usage grow, the value of analytics increases, which enables business benefits to accrue more quickly.Add Value with an Enterprise Analytics Unit HOW ANALYTICS CAPABILITIES win WITH ADOPTION The frequency with which analytics is used to support decisions increases as organizations transition from one level of analytic capability to the next. At the same time, analytics migrate toward more centralized units, first at the local line of business level and then at the enterprise level, while the portion of analytics performed at points of need and with IT remain stable. Percent using analytics frequently Where analytics performed 10 0% 80% Centralized analytic units 60% Line of business analytic units 40% 20%At point of need IT department 0% Aspirational Experienced Transformed nearly every way that analytics capabilities should be nurtured as an organization becomes more ambitious about becoming data driven The process needs to be additive. As analytics capabilities are added upstream at increasingly central levels of management, existing capabilities at point of need shouldnt be subtracted. Nor should they be transplanted to central locations. As new capabilities come on board, existing ones should continue to be supported. There are other ways that capabilities grow and deepen within an organization.Disciplines like finance and supply chain are inherently data intensive and are often where analytics first take root. boost by early successes, organizations begin expanding analytic decision making to more disci28 MIT SLOAN MANAGEMENT REVIEW WINTER 2011 Organizations that first experience the value of analytic s in discrete business units or functions are likely soon to seek a wider range of capabilities and more advanced use of existing ones. A centralized analytics unit, often called either a center of excellence or center of competency, makes it possible to share analytic resources efficiently andeffectively. It does not, however, replace distributed and localized capabilities rather, the central unit is additive, reinforced upon existing capabilities that may have already developed in functions, departments and lines of business. We found that 63% more Transformed organizations than Aspirational organizations use a centralized enterprise unit as the primary source of analytics. A centralized analytics unit can provide a home for more advanced skills to come together within the organization, providing both advancedmodels and enterprise governance through establishing priorities and standards by these practices Advance standard methods for identifying business problems to be solved wi th analytics. Facilitate identification of analytic business needs while driving rigor into methods for embedding insights into end-to-end processes. Promote enterprise-level governance on prioritization, master data sources and reuse to capture enterprise efficiencies. Standardize tools and analytic platforms to enable resource sharing, streamline maintenance and reduce licensing expenses. In three distinct areas application of analytictools, functional use of analytics and location of skills we found that adding capabilities without detracting from existing ones offers a fast path to full benefits from analytics-driven management. SLOANREVIEW. MIT. EDU RECOMMENDATION 5 Build the Parts, Plan the Whole Use an information agenda to plan for the future Big data is getting bigger. Information is coming from instrumented, interconnected supply chains transmitting real-time data about fluctuations in everything from market demand to the weather. Additionally, strategic information has started arriving through unstructured digital channels social media, smart phone applica