This is the first in a series that explains how innovative and successful organizations transform from data nascent organizations into data-driven organizations, outperforming competitors in the process.

When we say “data-driven,” we don’t mean companies that simply run management reports, perform analytics, or even implement complex statistical or self-calibrating machine learning models. We mean companies that have completely redesigned their business processes around sharing and using data as a fundamental part of their business strategy. They have fundamentally transformed their culture, technology, and data governance procedures to leverage as much internal and external data as possible, even the bits they didn’t originally think were going to be of value. Data becomes an enterprise asset, the centrifugal force around sustainable competitive advantage.

So what does this look like to the company and its customers?

Let’s explore the customer angle. For customers, it will feel like the company knows their needs and provides solutions and products that fit those needs exactly. I am sure, as a customer, you can think of companies that do this better than others. Which companies provide you with selections and choices that closely (or exactly) match your needs? A simple example would be Netflix. Before Netflix, you might go to a video store like Blockbuster (now I am dating myself). At the video store, you would search for titles that interested you or you heard about. The store didn’t have any suggestions or recommendations based on “knowing you” that would guide you to titles you might be interested in.

Netflix changed that.

With data about you, the titles and about other peoples’ choices, Netflix is able to provide you with recommendations which you may have never considered. And the success of those recommendations feeds back into the data to continuously improve the product (e.g., the recommendations). This is just one rather simple example. I am sure you can come up with others based on your own experiences of how data-driven organizations have changed (and hopefully improved) the customer experience. What do these companies have in common? They have access to a vast and diverse set of quality data and they harness this data as an enterprise (and valued) asset to drive their business (and their competitive advantage).

Over the course of this series, we will explore many of the critical dimensions of the data-driven transformation. What does it mean to transform? Why do it? What technologies enable this transformation, how does this affect the governance of data, and where do you even begin?

In this first article, we will explore the business reasons and implications for a shift to a data-driven organization. Topics will include:

• Why you need to care about transforming into a data-driven organization.
• The characteristics of a data-driven organization.
• The business changes and cultural changes required.
• Implications for technology.

The next article in the series will describe the new technologies and data architecture patterns that have emerged to make transformation possible. The third article will provide a point of view on what data-driven transformation means for data governance, and how managing data as a collaborative, shared asset will fundamentally change how we think about ownership and control. And finally, the series will conclude with a discussion on how to begin the transformation — how to sell it within the organization, finance it, and take the first step.

 Why Become a Data-Driven Organization?

Business dynamics are rapidly changing. There will be winners and losers. New methods are replacing the traditional ways of interacting with your customers, your vendors, your suppliers — even your own company and products. Data is at the core of this transition.

If you are already convinced that you need to become a data-driven organization, then feel free to move to the next section. If not, let’s take an example from the banking industry, where the dynamics of how we interact with customers have drastically changed. Traditionally, you would know your customer because they would come into the branch. The customer would know the teller and maybe even the branch manager. The teller would know the customer. They would strike up casual conversations, which would yield important information to help the bank service the customer. For instance, the bank might find that the customer has an imbalance between the dates they get paid and their bills. Maybe a credit line would help. They may find that a child will attend college soon, so maybe a loan may be necessary.

Interactions with customers today are very different. Customers may on occasion still visit a branch, but they also interact with the bank digitally. Banks have better data as it relates to customers’ transactions, accounts, and balances, both currently and in the past. Banks can also access additional customer information they can use to better understand customers’ needs. These may include products that they have looked at but not transacted, life events, relationships inside and outside the family, and so on. The list is practically endless and ever-expanding. The more data the organization has about the customer, the economic environment — such as interest rates, world events, and so on — the better informed the organization is, and the better it can service the customer.

In the past, the bank would segment customers and service each segment with certain products, but a data-driven organization no longer needs to lump customers into segments. Instead, it can be customized to each individual customer. This is sometimes called a “segment of one.” In addition to providing the customer with better-suited and potentially more products, such data processes also lead to improved customer retention.

That is just one example of how to know your customers better and, as a result, understand and potentially anticipate their needs. But the value of data doesn’t stop with customers. It actually infiltrates all aspects of the business — products, locations, employees, and so on. The organization can better understand how products are performing and make adjustments to better fit customer needs. The business can not only recommend more suitable products but potentially customize these products to better fit each customer’s need, assuming operational systems can support these customizations.

The list of benefits of a data-driven organization are nearly countless, so I will just provide one more potentially obvious area.

Data-driven organizations are able to bring the whole firm to the customer. What does that mean? In most cases today, organizations tend to be siloed by business line and/or geography. The ability to offer the customer the full set of products and services across the entire enterprise is very difficult, if not impossible. Data-driven organizations can provide their customers the full range of products and services irrespective of business line or geography, as regulatory laws permit. Business and geography silos are eliminated.

Hopefully, you will agree that becoming a data-driven organization needs to be a core focus. If you are not yet convinced, I will leave this section with one more thought. Competitive differentiation used to come from being able to execute your business processes better or more cost efficiently, whether through the leverage of globalization, automation or digitization. But that is now basic table stakes. Out-executing your competition can only achieve ever-diminishing marginal returns. To gain a consistent and sustainable competitive advantage, the winners in this new environment will outsmart their competitors through maximum leverage of data. Your competitors will be moving toward this objective. Will you be able to compete in the new economy with a traditional business model when your competitors are leveraging the value of data for their customers and businesses? Not to be overly melodramatic, but it can mean the difference between survival and prosperity.

What Are the Characteristics of a Data-Driven Organization?

The case put forward in the previous section must have some merit because most organizations are already on their journey from a traditional business model to becoming data-driven organizations. Few, however, have fully achieved this transformation. Even the ones who have claimed victory — while further along than their competitors — haven’t attained their goal. There is actually a continuum you can map each organization against. To simplify it, I have divided the continuum into four significant phases as shown in Figure 1.

Most mature organizations were initially focused on business processes (the Data Aware phase). Their main concern was to standardize and automate these processes to improve control and efficiency. This resulted in the creation of business (process) silos and resulted in data being locked into those silos with significant (bespoke) effort required to provision data across the organization or to other business functions. 

The move from Data-Aware to Data-Informed started with the introduction of management information (MI) and business information (BI) tools. Multiple data warehouses were created to support different uses of the data, usually by different groups such as risk, finance, etc. Data was moved and transformed for each data warehouse or use case. 

Then, over the last 10 years, with the advent of big data and a heavy emphasis on analytics, many organizations moved forward on the continuum from data-informed to data-enabled. Global data lakes were created and analytics were used to enable some capabilities for the business and, in some cases, the customer. While this was an improvement over the data warehouses, as described before, it is no longer sufficient as it still requires copies (sometimes multiple copies) of the data with specialized Extract, Transform, Load (ETL). The winners will take the extra step to become data-driven.

So, what does a data-driven organization look like? Simply put, data-driven organizations have easy access to high-quality data across the entire organization and utilize nearly all of it as part of their normal business processes. With that tenet in mind, there are four characteristics that distinguish a data-driven organization. They can:

• Find the Data. The data you are looking for is easy to find, usually through a robust data catalog.

• Know what it means. It is clear what the data represents or means, usually through a robust data dictionary.

• Know the quality. The data is of known quality and can be trusted

• Get access. Access to the data is quick and seamless to any region, product or function. Data is also stored securely and provisioned in line with legal and regulatory restrictions.

Some might think that this sounds relatively straightforward and easy to accomplish, but there are two important transformations required — a business transformation and a technology transformation. We will examine each of them next (and in more detail) in subsequent articles. Mature companies in particular need to shift their business, processes, technology and people —their very culture. New companies, on the other hand, can start their business without the baggage of the past. Hence, we have seen some really impressive results from some of these smaller, newer companies that large mature organizations are trying to mimic.

Business Transformation Required

One of the keys to transformation in a large, diversified organization is a shift in each line of business’s model. Every business in an organization needs to run itself, meaning it needs to be accountable for its own strategy, revenues, expenses, and so on. However, it can’t run in isolation. It needs to operate in the context of the enterprise. If it does not, it may as well be a separate company, as it is not realizing any benefit of the larger, more diversified organization. Thus the business needs to share both its processes and data with the rest of the organization, as well as consume processes and data from the rest of the organization. We move from a producer or consumer model to a model where everyone is both a producer and consumer. This will stretch the organizational boundaries of trust and control. The companies that succeed will reap the rewards.

The recognition of the whole as greater than the sum of its parts requires that all data naturally be shared, meaning that the default is to openly share data across the enterprise. The only exception would be due to privacy or data protection concerns, but there are solutions for this too that will be discussed in the second paper. Cross-organizational sharing will reduce the friction that currently exists when a data consumer in one part of the enterprise needs to access data produced by another part of the enterprise. Business leaders will have easy access to all data across the enterprise to include in their business plans and customer acquisition and retention strategies. This will allow these business leaders to leverage the strength of the entire enterprise (e.g a macro-view) as they develop and execute their business strategy and not be limited to the micro-view within their business.

That is harder than it sounds. The initial business model of most organizations was set up with a product or location focus. Teams developed processes and systems specific to this focus. As organizations evolved, or matured, they had wider business objectives to solve (e.g. multi-product, multi-location). Prior solutions were not set up for this new, broader objective. For example, in the early days of financial services, a bank’s primary objective would have been to optimize individual products. But as time marched on, the emphasis shifted to offer customers additional products or an entire portfolio of products. Data processes and systems were not optimized for this. Because data was not widely shared, cross-product capabilities required significant bespoke effort as the data could not easily be found and utilized. The result is the business and data silos that we see in most organizations today.

The idea of data sharing by default is not only a problem of silos but it is also a cultural problem. Most think that data is not to be shared unless explicitly required. Accordingly, we have built mechanisms to make it difficult to share data. This is because the business model and the architecture built around it identify the business and/or function as the owner of the data rather than the enterprise. If the enterprise was the owner of the data, then the default emphasis would be on sharing data enterprise-wide. Mechanisms would make it easy, rather than hard, to share data (and yes, we will have the ability to limit sharing of sensitive data — stay tuned).

Another important aspect to discuss is how people become data capable. As we move toward a data-driven organization, data becomes an essential driver of the business. Ultimately, data becomes the business. We need to ensure that our people have data skills. Just as people need business skills to be successful, they must now also be informed, active contributors and consumers of data. They will have new responsibilities — ensuring their data is well-defined in a data dictionary, is of good quality, and is easily accessible. They must be active participants in the exchange of data, with the enterprise as both receiver and provider. I could go deeper into the evolving role of the CDO, their responsibilities in the data-driven organization and what the data organization would look like, but I will leave that for the third article.

Some may think that with my focus on the enterprise, I am advocating for a centralized business-and-data organization. The contrary is true. I have found that driving from the center, or the enterprise, is very difficult and usually ends in failure. In fact, it needs to be a federated model where every part of the organization does their part for the whole. Think of it as crowdsourcing or a federated organization, where each part of the organization has their areas of expertise (responsible for its processes, data, costs, etc.), is an active member of the enterprise and functions according to rules defined by the enterprise.

In this way, each part operates effectively and efficiently as an individual business but also provides capability and data to the rest of the enterprise. This will ensure that each part of the organization is focused on its own area of expertise yet also provides the collaboration required to share, but not replicate, data across the enterprise. The impact on data governance processes and organization will be explored in the third article.

One final point that I want to make is that all of this is reliant on corporations building trust with their customers. Customers need to feel confident that the company secures their data, respects their privacy, and is using their data for their benefit. Clearly, this requires sound data security capabilities (including numerous techniques, e.g., masking, anonymization, etc.); clear ‘contacts’ (terms of business) with the customer on how their data will be used and a data ethics program to ensure that all data is used within the guidelines that the customer would be comfortable with. In short, if a corporation is data-driven, it needs to maintain a good relationship (trust) with the data providers (e.g., customers, et al).

Technology Transformation

In this section, I want to lightly touch on the technology transformation required to become a data-driven organization. We will go into more detail in the second article, so I will just introduce the topic here.

In the new economy, technology is usually a differentiating factor toward becoming a data-driven organization, meaning a technology transformation is required. Most systems were initially developed with a focus on function, such as a trading application, a confirmation system, or a settlement system. Each of these managed the data necessary to perform its own specific function. When a sequence of functions needed to be strung together to complete a process, data was sent from function to function — system to system — leading to the function-based architectures we see in most financial services firms today. The emphasis was on the function and, hence, we needed to move the data to the function. Systems became optimized, but data exchanges became duplicated and complex. While some functions have moved to a pub/sub or API-based mechanisms, the focus is still on the function as the center of the architecture. We will talk more about this later.

Next comes the move from a function-first architecture to a data-first architecture. As stated earlier, the priority of our infrastructure has been the function and the data has been a consequence of the function. The function produces and uses the data. If we flip to a data-first architecture, then the data is the primary focus and the functions are processes that act on the data. This may seem somewhat subtle.

One way to think about it is that the data is permanent and the function is a temporal process that has a beginning and an end. Inverting the focus significantly changes the way we build systems and treat data. Instead of functions owning data and moving the data around from system to system, we would leave the data in place and have the functions act on the same data. Another way of saying this is, “Instead of moving the data to the function, move the function to the data.” This would reduce or eliminate the number of copies of data while making data sharing the bedrock of the architecture. We would now optimize around the data, not the function, as was done previously. In the next article, we will look deeper into data-centric technology architecture.

Lastly, I want to talk about the technology toolset, which will be explored in more detail in the second paper. New tools are evolving that will be essential for the move to a seamless data-driven organization. The capabilities are built around the ability to actively manage metadata and bridge the gap between business metadata, for example through a data dictionary, and technical metadata, such as a data catalog. That is, to build a bridge between the way the business knows and uses data and the way the data exists in the technology.

Once we are able to actively and seamlessly manage the metadata, all of the tools will “sit on top” of this foundation. The data across the enterprise will be opened up to the enterprise as a critical component of its strategy, enabling it to finally become a data-driven organization. This data-driven business model will be “data hungry.” Those that are successful in this transformation will continually require more and more data (different types of data, from different sources), so the key will be to reduce the amount of time it takes to acquire data and push it through a process to identify it, assess it, catalog it and make it available. The only way to do this is through automation of the full end-to-end data process (i.e., a robust data architecture and toolset).

Summary

The move to a data-driven organization requires a business model shift and a new set of responsibilities. Each business must act as part of a larger organization. In doing so, it also benefits from the other businesses across the enterprise that also play by the same rules. Every business will benefit from enterprise-wide data for its own customer purposes. This will more than outweigh the effort required.

Even though a data-driven organization can be achieved and benefits realized at the business level, if achieved at the enterprise level, the whole organization will benefit. uplift. Customers will see and feel the difference. Just imagine the customer experience if a business is able to fully understand the entirety of a customer’s activities, interactions, experiences, and aspirations without the friction of silos and provide the customer seamless access to all of the organization’s products across all of its lines of businesses globally. Why does the customer have to bear the friction caused by these artificial barriers? The organization that conquers and breaks down these silos will reap the benefits.

So, where to begin? In the next article, we will discuss the enabling technology that makes data-driven transformation possible. It will describe some of the terminology and jargon behind the innovations in data processing and describe what it is about existing data technology that traps data and keeps it from being fully leveraged in the company.

About the Author

Peter Serenita is the Chairman of the Enterprise Data Management Council, a trade organization advancing data management globally across industries. One of the first Chief Data Officers (CDOs) in financial services, the 28-year JPMorgan veteran held several key company positions in business and information technology, including Chief Data Officer of the Worldwide Securities division. Subsequently, Peter became HSBC’s first Group Chief Data Officer,focusing on establishing a global data organization and capability to improve data consistency across the firm.

More recently, Peter was the Enterprise Chief Data Officer for Scotiabank, focused on defining and implementing a data management capability to improve data quality and leverage data for business enablement and improve the customer experience.

His awards include OnConferences’ 2021 Data & Analytics Professional of the Year and Waters’ 2009 Reference Data Executive of the Year.