Data management overhaul needed to migrate to cloud, report says

A new report based on a survey commissioned by financial firm Capital One explores data management in the cloud, concluding that new models are needed in the face of exponential data growth.

According to “New data management models are essential for operating in the cloudreport for which research firm Forrester Consulting surveyed 157 data management decision makers in North America earlier this year. Although primarily known for banking and credit cards, Capital One has recently entered the industry B2B software by launching Capital One Software, whose first product is Slingshot, a data management solution for Snowflake Data Cloud users.

“Data has become the currency companies use to create value and build strong relationships with their customers,” the report says. “Companies that leverage data at scale are in the best position to win, serve and delight customers. As companies move more data to the cloud, they will need new data management models to manage their quantity. and their exponentially growing variety of data New models that deliver improved self-service, integration, security and cost clarity will help businesses gain efficiencies and overcome the myriad of challenges data management they currently face.

Data management offers many benefits, Capital One has Explainincluding:

  • A more productive workforce
  • Data consistency across the enterprise
  • Faster delivery of new products and services
  • Better access to data for all
  • Better control over infrastructure and processing costs

As for what the term “data management” entails, the list is long:

  • Data governance establishes common standards and policies for data protection that align with government regulations.
  • Data Architecture describes the structure of an organization’s data assets and how data moves from one point in the system to another.
  • Integration of dates involves combining data from multiple sources and presenting it in a unified view.
  • Data Stewardship are the monitoring practices that ensure the quality and suitability of data assets for business users.
  • Data quality measures the suitability of a data asset for its intended use through factors such as consistency, accuracy and timeliness.
  • Data lineage provides visibility into how data flows through an organization, including its origin, transformations, and endpoints.
  • Data Correction is the organization, cleaning, moving, deletion and archiving of data to ensure data quality and the correction of any errors or errors.
  • Analysis of sensitive data identifies sensitive and confidential data, its location and degree of security.
  • Extract, transform, load (ETL) is a set of processes for moving data from its original source to a data warehouse.
  • Master data management ensures that a shared set of critical data exists for an organization that is accurate and up-to-date.
  • Data security is the practice of protecting data from harmful, undesirable and corrupting forces and breaches.
  • Metadata management is an agreement on how to identify, classify and describe data in order to improve the way users access and use the data.
  • Data lifecycle management ensures the proper management of a data asset from its creation to its archiving.

Capital One highlighted three key takeaways from the study:

Data estate models vary and evolve as the cloud journey progresses
“Nearly three-quarters of data management decision makers have yet to manage most of their organizations’ data in the cloud,” the report said. “As they continue their journey to the cloud, they need vendors to support organizational models that require decentralized and federated data support.

Data estate models vary and evolve as the cloud journey progresses
[Click on image for larger view.] Data estate models vary and evolve as the cloud journey progresses (source: Capital One).

“More than half of decision makers (56%) note that their organizations maintain a centralized data estate that brings together most or all organizational data. However, a small percentage (15%) are federated, which means there is a central set of data policies and platforms, but lines of business manage data independently via self-service A federated model supports the inevitable expansion of a data estate as data volumes, sources and tools are proliferating, eliminating potential bottlenecks in the enterprise-service, data assets must still remain well managed.”

Market demands integrated experience and combination of capabilities
“Eighty-two percent of data management decision makers find controlling and predicting data costs difficult, making ecosystem management tools essential,” the report said. Further: “Today, more than eight in ten data management decision makers (82%) cite cost forecasting and control as a challenge for the data ecosystem. Usage-based pricing models coinciding with cloud computing introduce month-to-month cost variability. was once meticulously planned and budgeted on site, is now unpredictable. As the volume and use of data increases, opacity hampers both general accounting and opportunities to optimize costs. Without the ability to forecast and manage cloud costs, decision makers can quickly find themselves off budget.”

Talent shortages and misaligned tools plague businesses and exacerbate challenges
“Talent challenges are the biggest barriers to data management excellence,” the report says. “Decision makers don’t have enough people, internal expertise, or collaboration across teams and tools to achieve their results.” Additionally: “Decision makers surveyed cite talent challenges as the biggest impediment to overcoming barriers, followed by tool issues. The ‘Great Resignation’ has made it difficult to retain and retain staff. While employees Skilled workers seek new opportunities, internal expertise moves with them, and barriers to integration and collaboration across teams and tools often lead to delays and inefficiencies.

Talent shortages and misaligned tools plague businesses and exacerbate challenges
[Click on image for larger view.] Talent shortages and misaligned tools plague businesses and exacerbate challenges (source: Capital One).

“Decision makers need tools that are aligned to meet changing business needs and meet a variety of roles. Integration and collaboration challenges are compounded when tools lack a simple user interface for different skill sets and areas of responsibility, and are not designed to work partners who can fill staffing, coordination and knowledge gaps, as well as make technology recommendations, are very attractive.”

In conclusion, Capital One said data management decision makers face key challenges in understanding what their organizations have, what they need, and how they share and exchange data to realize business value while continuing their journey to the cloud.

“As organizations progress in their cloud journey, they must adjust how they balance the use of global and local data with a federated approach to ensure that governance requirements are met and that data access, quality and observability are optimal,” the report said. “Decision makers need to understand how operating groups can work together seamlessly and close skills gaps. Organizations cannot extricate themselves from the skills shortage that so many businesses face. Suppliers and partners can fill expertise and best practice gaps left by employees through automation and no-code/low-code self-service.”

About the Author

David Ramel is an editor and writer for Converge360.