The implementation and use of innovative and disruptive technologies with a great impact on society are usually accompanied by two important phenomena:
1. Significant returns for the first entrepreneurs who bet on new products and/or methods brought by this technology, even before they are widely used.
2. Major challenges in exploring the potential of new solutions due to the threats and risks brought by change, as well as a lack of knowledge, resources, and foresight to understand and advance new ideas. The scenario is no different when it comes to big data, and this topic was addressed in a previous article published in this space.
One of the main obstacles for organizations to leverage analytical solutions and big data is not related to the technical knowledge of the tools and methods linked to their implementation, but rather to the ability of analysts and executives to formulate relevant questions for their work performance that can be answered through sophisticated data analysis processes.
According to the study titled Big data: The next frontier for innovation, competition, and productivity, published in 2011 by the McKinsey Global Institute (MGI), the U.S. market will face serious problems in meeting the growing demand for managers with analytical skills. According to this study, if radical measures are not taken to train a new generation of analysts and executives, by 2018 there will be a deficit of 1.5 million professionals to work in various business areas whose essential function will be to formulate questions that explore the potential of increasingly powerful and important big data resources for companies and institutions of all sizes and segments.
The scenario anticipated by MGI tends to be even more catastrophic for organizations operating in countries or regions with low competitiveness levels and/or recognized difficulties in hiring human resources with an analytical profile and knowledge of the benefits and applications of big data in their business. In a low-competitiveness scenario, companies have little incentive to implement new management models and innovative technology to support decision-making.
Even where there is an environment conducive to innovation and entrepreneurship, the absence or difficulty in hiring qualified professionals can result in frustrations in implementing analytical solutions. Regardless of the type of obstacle encountered by organizations in carrying out their data analysis projects and activities, those that do not effectively overcome these challenges run serious risks of accelerated loss of competitiveness that can even compromise their economic viability under new market conditions.
Surviving and standing out in an increasingly integrated and competitive market is directly linked to the quality and availability of consistent data that can help clearly answer all relevant questions to improve the performance of companies and public institutions. Asking the right questions and obtaining their answers in the form, content, and timeframe required by an institution's business processes is the starting point for improving the quality and assertiveness of corporate decisions.
Big data is a huge step towards providing these answers, but the responsibility for the relevance and quality of the questions is still essentially a function of the professionals involved in the use and application of analytical solutions in these organizations. In this sense, Professor Andreas Weigend, from Stanford University, aptly states that big data is useless without good questions, which leads us to reflect on the most relevant and useful questions that can be answered with the support of this technology and the maturity level of organizations to formulate and seek answers to these questions.
The Gartner Group, a world leader in information technology research and consulting, defines four stages or maturity levels related to the use of analytical solutions in general within an organization. At the most elementary level, known as descriptive analytics, companies seek to answer questions related to the past (question: what happened?). Even at this stage, it is surprising the difficulty many companies still have in quickly, consistently, and effectively answering this type of question, which is part of the daily demands of executives and managers at all levels of the administrative structure.
Certainly, to be competitive and grow sustainably, companies need answers to questions that go far beyond descriptive analytics. At a second level of maturity, companies must be able to diagnose any aspect related to the results of their operations (question: why did it happen?). This type of analysis is essential for improving the performance of an organization's processes, and there are various tools available on the market that allow obtaining answers to these questions with high productivity and low cost.
At the highest and most complex levels of analysis, according to Gartner Group's classification, are predictive analytics processes (question: what will happen?) and prescriptive analytics (question: what should I do?). In fact, much of society's attention and interest in big data is related to extremely successful experiences of applying predictive analytics in private and governmental institutions operating in areas as varied as public health, retail, financial institutions, electoral marketing, etc.
If having a clear view of the past and accurately diagnosing to avoid errors and replicate positive experiences is fundamental for efficient management of any business, anticipating the future represents an extraordinary gain in competitiveness for the vast majority of organizations, considering the current stage of evolution of analytical solutions and the availability of data from the most varied sources.
At the most advanced stage, prescriptive analytics seeks to automate decision-making processes and make them more effective and immune to negative external influences that might eventually negate the reason and logic established by data analysis at all previous stages. Few companies have achieved excellence levels in all these stages, especially the more complex ones with higher return on investment. However, demand is growing rapidly, and opportunities are enormous for companies and professionals.
To benefit from these opportunities and ensure a greater return on big data investment, it is necessary that the essential questions for each type of analysis are linked to a specific business context and well-defined objectives. It is very likely that the answers to some questions will bring insights or relevant information for decision-making processes that motivate the formulation of new questions, leading to a virtuous cycle of knowledge building known as data discovery. Thus, having good questions to start this cycle is fundamental.
And good questions don't arise by chance. They are formulated directly in proportion to business knowledge and the potential of analytical solutions and big data to improve organizational results, the level of executive training, and are also influenced by an internal and external environment that values competitiveness and meritocracy. As answers become increasingly accessible, the great challenge for professionals will be to formulate the right questions to solve problems that lead to excellence in the management and execution of corporate processes. Overcoming this challenge has invaluable worth for all companies.