Making the most of commerce data using a composable versus a monolithic tech stack
Experts discuss data at organizational and technological level: decentralized versus centralized data strategy, composable versus monolithic commerce.
Data is a key business driver in today's e-commerce, where buyers expect customized offers and processes, and knowing your customers' behavior is critical to success in keeping them engaged and outperforming your competition.
“Data is a kind of currency in the commerce world - knowing what customers do and what motivates them is gold.” - Dom Selvon, CTO at e2x
In a very insightful MACH talk, Stefan Schmidt, CPO/CTO at Emporix and Dom Selvon, CTO at e2x discussed data in a MACH vs. monolithic tech setup. We summarize key aspects below and provide some context.
You can watch the full talk here:
TL;DR
Properly implemented, both data strategies, centralized data lake or warehouse and decentralized mesh, have their raison d'être in commerce businesses.
The challenge with decentralized data networks is to balance consistency, availability, and failure tolerance (CAP theorem).
Composable commerce allows data services and storage distributed across the enterprise to be integrated without locking them into a monolithic warehouse.
In many commerce businesses, multiple sources of data exist, first-party and third-party. Accordingly, decentralized data landscapes where data is stored and managed in many different buckets have grown over time. Data ownership and responsibility are also decentralized.
The challenge of decentralized data architectures is to orchestrate multiple sources, data models, and responsibilities to make sense of data for the end-to-end commerce process. According to the CAP theorem, data consistency, availability, and failure tolerance can hardly be achieved equally in distributed systems, so businesses seek ways to centralize.
In response to the challenges of decentralized data landscapes, the concept of the data lake or warehouse aims to create a central repository for storing, processing and securing large volumes of structured, semi-structured or unstructured data.
Both approaches, decentralized and centralized data, have advantages and disadvantages that cannot be discussed in detail here. Which one is more suitable for your business cannot be answered in a generalized way, but must be aligned with individual business objectives.
Since both business and legal requirements for storing and handling data in companies are constantly changing, it should be possible to adapt data architecture and processes to new requirements at any time. Adaptive technology stack plays an essential role here.
Assuming that decentralized data spread over multiple domains results in a decentralized process and technology landscape, then centralized data results in a centralized, monolithic tech stack that ideally bundles data, capabilities and processes into one single entity.
The challenge in both approaches is to have a commerce technology stack that best supports orchestrating data from different sources, with different owners and responsibilities to make sense of the data and fuel your commerce business.
A composable tech stack allows assembling all the business capabilities you need to store, analyze, and leverage data as needed, like Lego bricks, and reorganize them at any time as new business requirements arise. This is a major advantage over monolithic systems.
“Composable Commerce helps reduce data fragmentation and lay the foundation for the seamless migration of multiple tools and data models.” - Stefan Schmidt, CPO/CTO at Emporix
Composable commerce platforms such as Emporix allow data services and storage distributed across the enterprise to be "captured" and integrated without locking them into a monolithic warehouse or data lake that may be difficult to adapt to future requirements.
A truly composable commerce system doesn't immediately force you to decide whether to pursue a centralized or decentralized data strategy. You can start with a decentralized representation of your data architecture and gradually centralize it, or the other way around.
Brief note on the CAP theorem mentioned above: Composable commerce architectures are designed to integrate decentralized data and capabilities into one modular environment, helping data teams balance consistency, availability, and fault tolerance in a data mesh.