Data Management

[nigatpolish]

Data Management

What is data management?
Data is essential to how a business operates and functions. Businesses must make sense of data and find relevancy in the noise that’s created by diverse systems and technologies supporting today’s highly connected global economies. In this regard, data takes center stage. On its own, data is useless – companies need an effective strategy, governance, and data management model to leverage all forms of data for practical and efficient use across supply chains, employee networks, customer and partner ecosystems … and much more.

So what is data management? Data management is the practice of collecting, organizing, and accessing data to support productivity, efficiency, and decision-making. Given the pivotal role data plays in business today, a solid data management strategy and a modern data management system are essential for every company – regardless of size or industry.
The key elements of data management
The data management process includes a wide range of tasks and procedures, such as:
• Collecting, processing, validating, and storing data
• Integrating different types of data from disparate sources, including structured and unstructured data
• Ensuring high data availability and disaster recovery
• Governing how data is used and accessed by people and apps
• Protecting and securing data and ensuring data privacy
Why is data management important?
Every application, analytics solution, and algorithm used in a business (the rules and associated process that allow computers to solve problems and complete tasks) depends on seamless access to data. At its core, a data management system helps ensure data is secure, available, and accurate. But the benefits of data management don’t end there.
Turning Big Data into a high-value business asset

Too much data can be overwhelming – and useless – if not managed properly. But with the right tools, Big Data can be harnessed to empower companies with deeper-than-ever insights and more accurate predictions. It can give companies a better understanding of what customers want and help companies deliver exceptional customer experiences based on the learning data provides. It can also help drive new data-driven business models – such as service offerings based on real-time Internet of Things (IoT) and sensor data – that wouldn’t be evident or obvious without the ability to analyze and interpret big data.
Big Data are extremely large data sets often characterized by the five V’s: the sheer volume of data collected, the variety of data types, the velocity at which the data is generated, the veracity of the data, and the value of it.
It’s no secret that data-driven organizations have a major competitive advantage. With advanced tools, companies are able to manage more data from more sources than ever before. They can also leverage many different types of data, structured and unstructured, in real time – including IoT device data, video and audio files, Internet clickstream data, and social media comments – opening up more opportunities to monetize data and use it as an asset.

Laying the data foundation for digital transformation

It’s often said that data is the lifeblood of digital transformation – and it’s true. Artificial intelligence (AI), machine learning, Industry 4.0, advanced analytics, the Internet of Things, and intelligent automation all require lots and lots of timely, accurate, and secure data to do what they do.
The importance of data and data-driven technologies has only increased since the outbreak of COVID-19. Many businesses are feeling intense pressure to make better use of their data now – and use it to predict future events, pivot rapidly, and build resiliency into plans and business models.
Machine learning, for example, needs very large and diverse data sets to “learn,” identify complex patterns, solve problems, and keep its models and algorithms up to date and running effectively. Advanced analytics (which often leverage machine learning) also depend on vast quantities of high-quality data in order to produce relevant and actionable insights that can be acted on with confidence. And the IoT and Industrial IoT run on a steady stream of machine and sensor data, flowing at a million miles a minute.
The common denominator in any digital transformation project is data. Before businesses can transform processes, take advantage of new technologies, and become intelligent enterprises, they need a solid data foundation. In short, they need a modern data management system.
The continued survival of any business will depend upon an agile, data-centric architecture that responds to the constant rate of change.
Donald Feinberg, vice president at Gartner
Ensuring compliance with data privacy laws

Good data management is also essential for ensuring compliance with national and international data privacy laws – like the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act in the U.S. – as well as industry-specific privacy and security requirements. And when those protections are to be proven or audited, having solid data management policies and procedures in place is essential.
Data management systems and components
Data management systems are built on data management platforms and include a range of components and processes that work together to help you extract value from your data. These can include database management systems, data warehouses and lakes, data integration tools, analytics, and more.

Database management systems (DBMS)

There are many different kinds of database management systems. The most common ones include relational database management systems (RDBMS), object-oriented database management systems (OODMBS), in-memory databases, and columnar databases.
Different data management systems
• Relational database management system (RDBMS): An RDBMS is a database management system that contains data definitions so that programs and retrieval systems can reference data items by name, rather than describing the structure and location of the data each time. Based on the relational model, RDBMS systems also maintain relationships between data items that enhance access and avoid duplication. An item’s basic definition and characteristics, for example, are stored once and linked to customer order detail lines and pricing tables.
• Object-oriented database management system (OODBMS): An OODBMS is a different approach to data definition and storage, developed and used by object-oriented programming system (OOPS) developers. Data is stored as objects, self-contained and self-described entities, rather than in tables as in an RDBMS.
In-memory database: An in-memory database (IMDB) stores data in a computer’s main memory (RAM), instead of on a disk drive. Retrieval from memory is much faster than retrieval from a disk-based system, so in-memory databases are commonly used by applications that require rapid response times. For example, what once took days to compile into a report can now be accessed and analyzed in minutes, if not seconds.
• Columnar database: A columnar database stores groups of related data (a “column” of information) together for quick access. It is used in modern in-memory business applications and for many standalone data warehouse applications where retrieval speed (of a limited range of data) is important.

Data warehouses and lakes
• Data warehouse: A data warehouse is a central repository of data accumulated from many different sources for the purpose of reporting and analysis.
• Data lake: A data lake is a vast pool of data stored in its raw or natural format. Data lakes are typically used to store Big Data, including structured, unstructured, and semi-structured data.

Master data management (MDM)

Master data management is the discipline of creating one trusted master reference (a single version of the truth) for all important business data, such as product data, customer data, asset data, finance data, and more. MDM helps ensure businesses don’t use multiple, potentially inconsistent versions of data in different parts of business, including processes, operations, and analytics and reporting. The three key pillars to effective MDM include: data consolidation, data governance, and data quality management.
A technology-enabled discipline in which business and the IT organization work together to ensure the uniformity, accuracy, stewardship, semantic consistency, and accountability of the enterprise’s official, shared master data assets.
Gartner definition of MDM
Big Data management

New types of databases and tools have been developed to manage Big Data – massive volumes of structured, unstructured, and semi-structured data inundating businesses today. In addition to highly efficient processing techniques and cloud-based facilities to handle the volume and velocity, new approaches to interpreting and managing the data variety have been created. In order for the data management tools to be able to understand and work with different kinds of unstructured data, for example, new pre-processing processes are used to identify and classify data items to facilitate storage and retrieval.

Data integration

Data integration is the practice of ingesting, transforming, combining, and provisioning data, where and when it’s needed. This integration takes place in the enterprise and beyond – across partners as well as third-party data sources and use cases – to meet the data consumption requirements of all applications and business processes. Techniques include bulk/batch data movement, extract, transform, load (ETL), change data capture, data replication, data virtualization, streaming data integration, data orchestration, and more.

Data governance, security, and compliance

Data governance is a collection of rules and responsibilities for ensuring data availability, quality, compliance, and security across the organization. Data governance establishes the infrastructure and names the individuals (or positions) within an organization that have both the authority and the responsibility for the handling and safeguarding of specific kinds and types of data. Data governance is a key part of compliance. The systems will take care of the mechanics of storage, handling, and security – it is the people side, the governance side, that ensures that the data is accurate to begin with and is handled and protected properly before being entered into the system, while being used, and when retrieved from the system for use or storage elsewhere. Governance specifies how responsible individuals use processes and technologies to manage and protect data.
Of course, data security is a major concern in today’s world of hackers, viruses, cyberattacks, and data breaches. While security is built into systems and applications, data governance is there to ensure that those systems are properly set up and administered to protect the data, and that procedures and responsibilities are enforced to protect the data outside of the systems and database.
Business intelligence and analytics

Most, if not all, data management systems include basic data retrieval and reporting tools, and many incorporate or are bundled with powerful retrieval, analysis, and reporting applications. Reporting and analytics applications are also available from third-party developers and almost certainly will be included in the application bundle as a standard feature or as an optional add-on module for more advanced functionality.
The power of today’s data management systems lies, to a great extent, in the ad-hoc retrieval tools that allow users with a minimum amount of training to create their own on-screen data retrievals and print-out reports with surprising flexibility in formatting, calculations, sorts, and summaries. In addition, professionals can use these same tools or more sophisticated analytics tool sets to do even more in the way of calculations, comparisons, higher math, and formatting. New analytics applications are able to bridge across traditional databases, data warehouses, and data lakes to allow incorporation of Big Data with business application data for better forecasting, analysis, and planning.
What is an enterprise data strategy and why should you have one?
Many companies have been passive in their approach to data strategy: accepting whatever their business application supplier has built into their systems. But now, that is not good enough. With today’s explosion of data, and its importance to the operation of every enterprise, it is increasingly necessary to take a more proactive and comprehensive approach to data management. What that means, from a practical standpoint, is preparation, setting a data strategy that:
• Identifies the specific types of data your company will need and use,
• Assigns responsibility for each type of data, and
• Establishes procedures to govern the acquisition, collection, and handling of that data.
One of the key benefits of a corporate data management strategy and infrastructure is that it brings the organization together – coordinating all activities and decisions in support of the enterprise’s purpose, which is to effectively and efficiently deliver quality products and services to customers. Having an all-encompassing data strategy and seamless data integration eliminates silos of information. This allows each department, manager, and employee to see and understand their individual contribution to company success – and keep their decisions and actions aligned with those goals.

The evolution of data management
Effective data management has been critical to business success for well over 50 years – from helping companies improve the accuracy of information reporting, spot trends, and make better decisions to fueling digital transformation and powering new technologies and business models today. Data has become a new kind of capital, and forward-thinking organizations are always on the lookout for new and better ways to use data to their advantage. Here are the latest trends in modern data management that are important to keep an eye on and explore their relevance to your business and industry:
• Data fabric: Most organizations today have a variety of types of data deployed on premise and in the cloud – and they use multiple database management systems, processing technologies, and tools. A data fabric, which is a custom combination of architecture and technology, uses dynamic data integration and orchestration to enable frictionless access to and sharing of data across a distributed environment.
• Data management in the cloud: Many companies are moving some or all of their data management platform to the cloud. Cloud data management takes advantage of all the benefits cloud has to offer – including scalability, advanced data security, improved data access, automated backups and disaster recovery, cost savings, and more. Cloud databases and database-as-a-service (DBaaS) solutions, cloud data warehouses, and cloud data lakes are all growing in popularity.
• Augmented data management: One of the newer trends is called “augmented data management.” Identified by Gartner as having significant disruptive potential by 2022, augmented data management uses AI and machine learning to make data management processes self-configuring and self-tuning. Augmented data management is automating everything from data quality and master data management to data integration – freeing up skilled technical staff to focus on higher value.
Through 2022, data management manual tasks will be reduced by 45% through the addition of machine learning and automated service-level management.
– Gartner
• Augmented analytics: Augmented analytics – another top technology trend identified by Gartner – is already here. Augmented analytics uses artificial intelligence, machine learning, and natural language processing (NLP) to not only find the most important insights automatically, but to democratize access to advanced analytics so everyone, not just data scientists, can ask questions of their data and get answers in a natural, conversational way.
Summary
We know that information is derived from data. And if information is power, then effectively managing and capitalizing on your data could very well be your company’s super power. As such, data management responsibilities and the role of database analysts (DBAs) are evolving to become change agents – in driving cloud adoption, leveraging new trends and technologies, and delivering strategic value to the business.

1 Reply Last reply