Harmonization and consolidation of Sell Out data: a major and current issue for consumer goods companies
Consumer goods, luxury goods and retail players are faced with an ever-increasing and heterogeneous amount of transactional data: it is becoming more accurate, the sources are multiplying and the use cases are exponential!
These data are collected via internal channels such as D2C (eCommerce…), via the company’s finance teams (sell-in data) or via market estimates according to sometimes very subjective rules. They also come from third party companies such as panelists (Nielsen, Kantar, Iri etc…), retailers/distributors (B2B, B2B2C). But new sources of data, via innovative players (AI, web scraping data, more granular and factual data, new consumer information, DN/DV…) are shaking up the data landscape even more, requiring a real organization and mastery of the data in order to extract as many levers as possible for the business.
If the quantity (and quality!) of data is increasing, the diversification of distribution channels – with in particular the acceleration of e-commerce, induced by the Covid 19 pandemic (see illustration below) – thus imposes a new challenge: how to harmonize, model, consolidate and, ultimately, make use of this heterogeneous and scattered data in order to carry out more targeted, better documented and therefore more ROI focused business actions?
The growth of e-commerce makes it all the more necessary to collect and report sales data on this channel
Indeed, most multinational groups own as many data sources as they have subsidiaries/local offices. As a result, the structure, coverage and granularity of sales data sources vary greatly within the company’s databases.
Finally, the heterogeneity and difference in granularity of sales data sources must be imperatively considered for the development of a holistic vision of sales!
Thus, there are 4 main challenges for companies :
- Make all company transactional data available and usable instantly to maximize sales monitoring and decision making regarding targeted actions to be implemented quickly.
- Be able to assess and compare data across different entities, countries, brands or categories by creating a similar hierarchical structure and typology for all data.
- Empower employees in their use of data by implementing data-centric governance that promotes data as a service.
- And finally, build more efficient data pipelines by simplifying existing data architectures within the company information system.
A technological and organizational answer to these challenges: Building a Single Source of Truth (SSOT)
What is the Single Source Of Truth (SSOT)? A data source that meets the following four conditions:
- A unique data source. It gathers, in a single datalake, all the internal and external data sources of the company related to customers and to D2C and B2B sales. It is a precise, homogeneous, complete and granular source ensuring a partition of the data, also referred as omnichannel data.
- A data source open to each employee, in accordance with the access rules defined before the project. This is a crucial point, as because of data partition, an employee spends an average of nearly 20% of his working week looking for internal information or colleagues who can help him with specific tasks. [3]
- An undeniable source of truth, ensuring data standardization, data update and quality. To ensure this, the data is deduplicated, frequently updated and corresponds to a defined quality standard. A SSOT allows to overcome having data existing under different values thus making more difficult the identification of the “good” data to use.
- A standardized and unified data source. The definition of a common data format allows to create a unique language between the different entities of a company, its departments and its brands. No more arguing about market definitions!
With an SSOT, a company can:
The reporting tools and dashboards are connected to the SSOT, all the data from the datalake can now be automatically incremented in the sales reports.
Usually, a company has one data source per sales channel. In comparison, a SSOT allows to compare sales on each channel (Ecommerce, brick & mortar, B2B2C, B2C…) and in each country.
Indeed, the consolidation of a company’s and its competitors’ data allows to rebuild markets and their segments on a local, regional and global scale.
In each country, up to EAN/SKU code in some cases.
And other factors that contribute to a company’s success to help define and drive growth factors.
By providing immediate access to SSOT data through enterprise data visualization tools (Tableau, Power BI, Looker, etc.).
Between teams to address more substantive issues.
Access to high quality and high volume of data are two conditions to train machine learning programs for the purpose of predicting future sales.
Summary of the data flows feeding SSOT and the main resulting use cases
Finally, a single source of truth allows teams to make easier, quicker, and more effective business decisions, at the global and local level.
Research time and effort is reduced, therefore employees are focused on how to use the data, not how to collect it.
The main challenges in harmonizing and consolidating Sell Out data sources
During a harmonization and consolidation project of data sources, 3 major challenges will be encountered.
The first challenge lies in the heterogeneity of the different data sources. This has three main origins:
- The data may come from external providers who use their own methodologies and taxonomies. Thus, the structuring of this data often appears to be a real black box
- Each data source has a unique story: indeed, a data source is initially developed/structured for a specific use case. As a result, the business rules governing the different data sources are by nature not identical.
- The granularity from one data source to another often differs, so in order to consolidate them, the common granular denominator must be determined.
For instance, one of the main heterogeneity factors for panel data sources covering different markets is the “local knowledge” of the data. Indeed, the build of reporting panel data is the result of a close collaboration between the consumer/customer market insight departments covering a specific geographical zone and the data agencies that bring their knowledge of the local market.
Thus, because of these data heterogeneities, the consolidation of an aggregated market view cannot be done by simply concatenating data sources that each cover part of the original market.
To consolidate and harmonize data sources, it is therefore necessary to transform the original data sources. This process, also called mapping, is a key step in the harmonization towards a target architecture shared by all the data that we aim to aggregate.
Then, the result of mapping is a common data structure that constitutes a new data repository.
Mapping example of heterogeneous spreads databases
Thus, the second challenge of a harmonization project lies in the legitimacy of the common architecture choices that will be adopted during mapping. Indeed, the original data sources are duplicated and then restructured towards the target architecture. As a result, some market segment definitions can be reworked, thus overturning traditional sales reporting.
Taking back the example of the above visual, we understand that the sales reports’ structure for spreads market in Germany will be different from the (consolidated) European sales reports.
It is therefore a real change and a new vision that must be legitimized to drive the adoption of the consolidated and unified vision by data sources’ users…
Finally, once the data sources have been consolidated, a last challenge arises: the potential overlap of the different data sources.
Since the data sources come have different origins and corresponds to specific use cases, it is common to have data sources that overlap, at least partially.
The main challenges of overlapping data lie in:
- The identification of overlapping data segments
- Then to be able to isolate these segments in each concerned data source
- And finally, to establish what should be the only source of truth for these two segments…
Our recommendations for the success of such a transformation
Conclusion
The benefits of having a single source of truth (SSOT), easily available, built on real business use cases with well-chosen KPIS are no longer to be listed. Thus, for several companies in the Consumer Goods, Luxury Goods and Retail sectors, SSOT is the goal to master the accuracy of sales figures and thus make better business decisions and directly impact sales performance.
The path to get there has been mapped out, but to do so, it is important to know the maturity of your company (quality and accessibility of Sell Out data sources, IT architecture and capacity, availability of IT leaders and developers, dedicated or non-dedicated data organization) and to size your ambitions according to your resources to be able to effectively deploy the best suited solution to your company.
To that end, they can rely on our experts’ network to support them both in consolidating their data strategy and in deploying it while considering their specific challenges and IT ecosystem.
Sources :
[1] – Statista: Retail e-commerce sales worldwide from 2014 to 2025
[2] – Statista: Consumer Market Outlook
[3] – McKinsey: The social economy: Unlocking value and productivity through social technologies
Leandra de Barba
Jeanne Bouaziz
Julien Miniconi
Alessia Corsi
Xavier Baudouin
Marina Soldi
Thibaud Verdickt