My Communities

Data management questions entire revolve what data to collect and to what extent, when is the right time to collect, who will collect such data, what to do with the data and who will benefit from such data and knowledge gained.

Data management questions entire revolve what data to collect and to what extent, when is the right time to collect, who will collect such data, what to do with the data and who will benefit from such data and knowledge gained.

A data flow map is a valuable tool to help you visualize your data processes, as well as define clear roles. It can help you set expectations for what data products need to be produced and by whom, and what steps need to be taken to get to those final products.

A data flow map is a valuable tool to help you visualize your data processes, as well as define clear roles. It can help you set expectations for what data products need to be produced and by whom, and what steps need to be taken to get to those final products.

Pour la qualité des données une analyse sera faite et les résultats obtenus seront testés à l'aide des tests statistique

Pour la qualité des données une analyse sera faite et les résultats obtenus seront testés à l'aide des tests statistique

1. What kind of data do I need to analyze?
2. Is the majority of your company's data transactional?
3. What level of data security do I need?
4. Who has access to sensitive customer information/data?
5. What is the single identifier that helps you link all customer data?
6. How many separate databases do you use to hold and analyze customer data?
7. Do you have a single person in charge of data management?
8. Do you have a process or platform that enables data sharing across your organization?
9. What is a centralized data management strategy and how can I achieve it?
10. How do you define and assign data roles?
11. How can improved data quality enable to make better decisions?

Why data management is important
Data management is a crucial first step to employing effective data analysis at scale, which leads to important insights that add value to your customers and improve your bottom line. With effective data management, people across an organization can find and access trusted data for their queries. Some benefits of an effective data management solution include:

Visibility
Data management can increase the visibility of your organization’s data assets, making it easier for people to quickly and confidently find the right data for their analysis. Data visibility allows your company to be more organized and productive, allowing employees to find the data they need to better do their jobs.

Reliability
Data management helps minimize potential errors by establishing processes and policies for usage and building trust in the data being used to make decisions across your organization. With reliable, up-to-date data, companies can respond more efficiently to market changes and customer needs.

Security
Data management protects your organization and its employees from data losses, thefts, and breaches with authentication and encryption tools. Strong data security ensures that vital company information is backed up and retrievable should the primary source become unavailable. Additionally, security becomes more and more important if your data contains any personally identifiable information that needs to be carefully managed to comply with consumer protection laws.

Scalability
Data management allows organizations to effectively scale data and usage occasions with repeatable processes to keep data and metadata up to date. When processes are easy to repeat, your organization can avoid the unnecessary costs of duplication, such as employees conducting the same research over and over again or re-running costly queries unnecessarily.

The data flow maps is the true visualizer and planner of any data management process.

Pretty intersting.

it is the collection, collation, storing and use of data

Data also can lead to teams not relying on their own knowledge and experience to come up with the best solutions. Researchers found in a study that 60% of radiologists asked to analyze a routine chest x-ray failed to detect that a collarbone was missing – because they were so familiar with data that trained them to expect to see one.
Experts say that data science is just another tool, and it’s designed to provide probabilities, not absolute answers. In addition, companies must understand that those who build the predictive models may have flawed assumptions or be mistaken about what data is most important to a company’s objective or strategy.
twenty question that can be discussed.

1 What is the subject discipline (domain, field) to which your research data relates?

2 What is the exact nature (range, scope) of your research data?

3 When will your research data be collected?

4 When will your research data be processed and analysed?

5 Who owns the data arising from your research, and the intellectual property rights relating to them?

6 How will your research datasets be described, i.e. with what metadata or accompanying interpretive information will they be accompanied, and how will these metadata be created?

7 Where, and in what format(s), will you store your data in the short term after acquisition?

8 Who is responsible for the immediate day-to-day management, storage and backup of the data arising from your research?

9 How frequently and where will your research data be backed up for short-term data security?

10 With whom will you share your research data in the short term, before publication of any papers arising from their interpretation?

11 Why is access to your research data to be restricted in the short term (if indeed it is)?

12 To whom will you provide access to your research data in the long term, with what limitations as to re-use, and under what license arrangements.

13 Why is access to your research data to be restricted in the long term (if indeed it is)?

14 How (i.e. by what physical or electronic method) are your research datasets to be transferred from short-term storage under the local care of yourself or your research group to their long-term archival and Web publication destination under the curatorial care of a separate third-party, e.g. a data repository?

15 Where will your research data be archived for long-term preservation?

16 When will your research data be moved from your own local storage to a secure archive for long-term preservation (e.g. your institutional library’s data repository)?

17 Who has authority to decide which of your research data are NOT worth preserving and will be deleted?

18 Where will your research data be published for others to see?

19 When will your research data be published in this manner?

20 To whom will responsibility for the long-term preservation of your research data devolve, once you have left your present research group?

Data management is the practice of managing data as a valuable resource to unlock its potential for an organization. Managing data effectively requires having a data strategy and reliable methods to access, integrate, cleanse, govern, store and prepare data for analytics.
A "data question" is something you ask for smart data and that it is expected to be answered by a table of data. In practice a data question is described by a plain English sentence and a set of expected answer fields: "Zip codes for all Italian administrative areas" (municipality, zip, areacode")
Data management questions to ask solution providers is an annual sneak peak at the top-of-mind concepts to consider during product evaluation. For five key data management questions to ask yourself, consult our Buyer’s Guide for Data Management Solutions.

Cue the process of seeking out, evaluating, choosing, purchasing, and deploying a data management solution. There’s no such thing as a one-size-fits-all approach when it comes to big data. Solutions come in a variety of flavors—ranging from data management solutions for analytics to operational database management systems. Each features a particular set of capabilities, strengths, and drawbacks. Choosing the right vendor and solution is a complicated process—one that requires in-depth research and often comes down to more than just the solution and its technical capabilities.

To help you evaluate prospective data management solutions, these are five data management questions to ask solution providers during product evaluation. If you find these questions helpful, check out our Buyer’s Guide which features five more questions to ask yourself, a comprehensive overview of the market, and full, one-page profiles of the top-28 offerings, as well as our ‘Bottom Line’ analysis.

The essence of Data flow map helps us in identifying roles and role plays in the project. Also ,it helps in getting rid of unnecessary positions in managing the staffing process in the organisation.

i wish you all the best

It is important to ask questions during data management because it's only when we ask questions that we can establish gaps that need to be filled. So during data management,it is imperative that questions,alot of questions need to be asked to ensure a perfect management of data.

yes i can

Data management is the processes involved from data collection, entry, collation, analysis and reporting. It involves a team of field staff, M&E team, project manager and director. It varies depending on project or organisation

Peer educators can play a similar role with field officers.

Data management is the process of looking for, ingesting, storing, organizing and maintaining the data created and collected by an organization. It is important to keep a proper data management because the inconsistent data sets , the data quality problems or the lack of finding can limit or create problems to the projects of the organization.

I would say to split it in different batches and it is important to have a good quality check

I have an inquiry about where in the data management process does the monitoring and evaluation concept apply.

When thinking about the different data management steps it will be important to consider various questions while looking at each step. For data collection you want to think about for each data collection tool - who is responsible for data collection and ensuring quality assurance. For data entry and collation think about who will enter the data, who will collate it and where will the data be entered. For data analysis, verification and storage - who will analyze the data, how often should it be verified and who will conduct the verification process. Who will archive the data and when should it be archived. Once data has been collected and analyzed it is important to understand how the data will be used - who will create and distribute reports? Who will prepare other products and who will review the data to help determine future projects? It is important that all team members involved understanding the data management process as well as have defined roles to ensure the data is managed effectively for the project duration.

One of the top questions for data management is security. How can you securely store data? Do you need to de-identify it?

in a small organization without M&E officer who else can take charge of the data

In found this process of drawing data flow map very helpful. It's easy to know the roles of each person/office though it may happen that two people can have one role. It's a good way of dividing roles and responsibilities and help people to focus on them.

In found this process of drawing data flow map very helpful. It's easy to know the roles of each person/office though it may happen that two people can have one role. It's a good way of dividing roles and responsibilities and help people to focus on them.

Data management minimizes the risks and costs of regulatory non-compliance, legal complications, and security breaches. It also provides access to accurate data when and where it is needed, without ambiguity or conflict, thereby avoiding miscommunication.

very good explanation indeed

Data management seems very complex and requires more attention than I imagined at the beginning of the course.

Data management is a set of practices for handling data collected or created by a company so that it can be used to make informed business decisions. The core idea behind the entire process is to treat data as a valuable asset.
Data management is the process of ingesting, storing, organising and maintaining the data created and collected by an organisation.

If well designed, it has the following advantages:
Overall productivity improvement
Cost efficiency, no duplication of processes or hiring un necessarily
Ability to rapidly respond to change
Enhanced accuracy of decisions

Hi @peacekale, excel and google sheets are some of the tools that help with data analysis and can be used to store data as well. But these can only be efficient with small data. With large/big data, Database Management Software like MySQL, PostgreSQL, Oracle and advanced analytical tools like R, Python, SPSS come in handy.

Data management is imperative aspect of project monitoring and evaluation as it helps to provide an insight on how data is collected, stored, analyzed and disseminated to a number of projects stake holders. Additionally, data management helps the directors and project managers to generate precise reports to the donors and other projects stakeholders, as well as facilitation of rational decision making towards attainment of projects objectives.

in a small organization without M&E officer who else can take charge of the data

Discussion: Data Management Questions
Clock 5 minutes

What are the most critical questions those should be answered during data management process?

The data management is a key aspect of an M&E plan, as it would have an impact on the daily job of everyone involved in the process, on the quality of the results and, consequently, on the satisfaction of people involved (donors, office staff, beneficiaries, etc,). It is important to spend some time to clearly set a data flow chart, in order to better oganise the data management.

is it consistent that all the data collected must be sent to government agencies.

For data that should be used internally, there is always a consistent data flow map.

the most important aspect of the data flow map considering the MEH situation is that there is a clear role and responsibilities of every staff at every stage of the data flow.

What are some tools for effective data management?

Some data management tools can be computer software programmes that assist you to insert individual data and indicators, keep track of information and create visual analytics to help you interpret and analyze the data and come to effective solutions.

Data is nothing without good management. It is important to create and follow a flow that will ensure effective and efficient collection and use of data

I think one of the most important aspects of data management is the validity and transparency of the data collected. At times, some data is collected and entered on time and then the data is processed and analyzed; therefore, there is missing information that could have affected the analysis of the data. Therefore, to avoid this, I think there always has to be an assigned staff member to follow up with the people who are supposed to collect and enter the data.

Data quality is a practice of collecting, keeping and using data efficiently and cost effectively

Data Management is key because it lingers on even after the project is over. Data should be well tracked and managed

I agree totally with you. weldone

Tthank you for this exposition

1. What level of data security do I need?

Some vendors offer expansive data protection capabilities. Of course, these add-ons come with a price, and it would be helpful to know upfront whether or not securing your stored data is a priority to the degree that paying for it represents. We think it is, but some organizations like to maintain those protocols in-house. Many solution providers even go a step further, offering services that ensure certain data types remain compliant with ever growing regulations, which leads us to our next question.

2. What kind of data do I need to analyze?

Is the majority of your company’s data transactional? Is it all structured? If so, a traditional or “legacy” tool may be the best fit for your use case. If the bulk of your data streams to your data lake in real-time via CRM, cloud applications, and customer feedback, then a solution that can integrate with the likes of Hadoop, Spark, and NoSQL repositories is likely appropriate. Be sure to take into account the types of data that run through your business and then match that up with the appropriate provider.

3. Cloud, on-prem, or both?

A hybrid approach is a growing trend in the enterprise market as it provides organizations the ability to execute integration in both on-prem and cloud environments. Thus, organizations are able to interchange data to and from either framework as a way to gain business agility, manage cloud delivery, and address the need for data sharing between environments. On-prem data management is certainly not dead, but a hybrid approach will set your organization up nicely for the future, even if cloud exposure is currently limited.

4. Which use cases do I need to focus on? What will the impact look like?

In other words, what does the deployment of a data management tool help you do differently? Focusing on specific use cases helps to ensure that the implementation of this technology helps move the needle along the desired path. The impact should be measurable, but it will also require collaboration amongst users. The expansion of data volumes and velocities should always result in an end goal of expanded business value.

5. Will data management help me maintain compliance?

Organizations that place a hefty emphasis on data are increasingly realizing they are not in compliance with industry regulations. In many cases, companies are flocking to data management for this reason – to automate the process of regulatory compliance and ensure that they are following the law. Compliance is vital in any vertical where personal records are shared. Healthcare and government are just two of the major players. If your organization resides in a highly regulated industry, it becomes important to choose a tool that will help you remain up-to-date.

1. What level of data security do I need?

Some vendors offer expansive data protection capabilities. Of course, these add-ons come with a price, and it would be helpful to know upfront whether or not securing your stored data is a priority to the degree that paying for it represents. We think it is, but some organizations like to maintain those protocols in-house. Many solution providers even go a step further, offering services that ensure certain data types remain compliant with ever growing regulations, which leads us to our next question.

2. What kind of data do I need to analyze?

Is the majority of your company’s data transactional? Is it all structured? If so, a traditional or “legacy” tool may be the best fit for your use case. If the bulk of your data streams to your data lake in real-time via CRM, cloud applications, and customer feedback, then a solution that can integrate with the likes of Hadoop, Spark, and NoSQL repositories is likely appropriate. Be sure to take into account the types of data that run through your business and then match that up with the appropriate provider.

3. Cloud, on-prem, or both?

A hybrid approach is a growing trend in the enterprise market as it provides organizations the ability to execute integration in both on-prem and cloud environments. Thus, organizations are able to interchange data to and from either framework as a way to gain business agility, manage cloud delivery, and address the need for data sharing between environments. On-prem data management is certainly not dead, but a hybrid approach will set your organization up nicely for the future, even if cloud exposure is currently limited.

4. Which use cases do I need to focus on? What will the impact look like?

In other words, what does the deployment of a data management tool help you do differently? Focusing on specific use cases helps to ensure that the implementation of this technology helps move the needle along the desired path. The impact should be measurable, but it will also require collaboration amongst users. The expansion of data volumes and velocities should always result in an end goal of expanded business value.

5. Will data management help me maintain compliance?

Organizations that place a hefty emphasis on data are increasingly realizing they are not in compliance with industry regulations. In many cases, companies are flocking to data management for this reason – to automate the process of regulatory compliance and ensure that they are following the law. Compliance is vital in any vertical where personal records are shared. Healthcare and government are just two of the major players. If your organization resides in a highly regulated industry, it becomes important to choose a tool that will help you remain up-to-date.

In my opinion, I think that data management is a vital part of M&E. All program staff are playing key role in the process unknown or knowing. Mostly I used to think that Project Officers, Field Workers etc are not part of M&E. Now I understand that I was wrong.

The Data Flow Map is really useful for this process. It makes it more visual and practical to tie up loose ends in the flow.

An intern can be employed and trained on basic M&E processes; preferably one with a background in numerical and analytical processes

I totally agree with this!

If they are one of the key stakeholders, it might be necessary to do so.

Data management is the practice of collecting. keeping and using data securely, efficiently, and cost effectively

It is crucial for the team leader of a project to distribute responsibilities considering the capacity, ability and knowledge of all of the persons who intervene in the M&E process for the correct organization, collation, analyzing, storing (archiving) and transformation of data and give DATA the power, credibility and confidence for decision making.

Yes... I have worked with them many times and they are really interested in doing field work. Helping with the surveys and gathering other types of information.

Data management is a important part of any project

Two things that have helped transparency of the projects especially when I work with government agencies are:

1. Create a community monitoring committee where the organization does not participate (only by providing the instruments and training to monitor the processes) but local authorities and beneficiaries do.
2. Designate an internal auditor who is in charge of monitoring that each M&E actor carries out their activities effectively. Verifies the information gathered and systemized. That way I ensure credibility in the data.

Need Help. How can we attach the final project with all the M&E Plan. I cant find where to send it.

Yes me too. It gives a complete picture of what we need to consider for the project as well as the roles and responsibilities.

Proper data management is essential for the project cycle helping strengthen the decisions made

Proper data management is essential for the project cycle helping strengthen the decisions made

It is very important to know how to manage data

Data Management is all about the process of collecting, Storing, accessing, organizing, analyzing and using data.
It is a process, data needs to be collected, stored, analyzed , verified and then used for decision making and for future reference. and used in the reporting.
data collected needs to be kept confidential.

Data management questionnaire helps the data collectors to access massive participants to collect valid and accurate data that can be helpful for project to build its decision on it

The data flow mapping process looks quite complicated. I've never used it before. Anyone care to share experiences on its efficacy and how it helped their organisation's processes?

Data management is the practice of collecting, keeping, and using data securely, efficiently, and cost-effectively.

What does data verification refer to and why is it important?

Its good module, almost there

I think one person can be chosen trained and be entrusted with the data

Data verification is a process in which different types of data are checked for accuracy and inconsistencies after data migration is done. It helps to determine whether data was accurately translated when data is transferred from one source to another, is complete, and supports processes in the new system.

Data verification is a process in which different types of data are checked for accuracy and inconsistencies after data migration is done. It helps to determine whether data was accurately translated when data is transferred from one source to another, is complete, and supports processes in the new system.

Read this, it will help you get a clue

Data management processes actually encompasses; data entry, data analysis, data storage, data verification and data use. This process is important in ensuring the success of a project. Knowledgeable monitoring and evaluation staff should be assigned these roles each.

I have indeed appreciated this aspect of data flow map since it gives us a clear and fast way of knowing which role is performed by which officer.

data verification is the process of checking the presented data to ensure that they are devoid of errors and are in the right form. This process helps to ensure that the data are correct and right decisions are drawn from them.

Data management is one of the final steps with a very important role in the monitoring process

I agree. I am considering taking extra courses to learn data analysis specifically

Why is data management important?

Q:

What were your practices for backing up and storing media for your previous employer?
A:

A data manager is responsible for maintaining all confidential files generated by your company. The candidate must understand the importance of backing up data on servers and workstations used throughout your organization. Security practices must be followed to prevent unauthorized access to the data, and disaster recovery plans must be followed to protect your company against data loss. The data manager must follow proper backup and security protocol for your company’s files.

What to look for in an answer:

Experience backing up servers and securing media
Data security skills and IT standards experience
Knowledge of disaster and recovery practices
Example:
“I created backup media for all servers and workstations daily, allowing the backups to capture all data that was changed throughout the workday. All backup media was stored in an off-site and secure location.”

Q:

What development and implementation procedures do you follow for new data systems?
A:

A data manager develops and implements new data systems when the information system is upgraded or changed. He or she follows current IT standards and regulations for the new systems and ensures that the products remain compliant with federal laws for storing confidential records and information. The candidate must have experience in designing new systems, evaluating the integration with your existing infrastructure and following all necessary security standards for storing the data.

What to look for in an answer:

Experience developing and implementing new data systems
Familiarity with managing and securing data storage systems and devices
Knowledge of all IT standards, regulations and laws
Example:
“I followed all IT standards for developing new data systems for storing and protecting data while conducting implementation protocols to ensure compliance with current regulations.”

Q:

How did you manage proper data sharing practices for your previous employer?
A:

A data manager follows strict protocols to prevent workers from sharing data with unauthorized users. He or she must adhere to standards and ensure that workers stick to strict guidelines for transmitting confidential files or information between departments and to outside sources. The candidate must have experience tracking access to the data systems and blocking unauthorized workers from opening or sharing files illegally. He or she must enforce strict sharing practices and lower the risk of data loss.

What to look for in an answer:

Knowledge of using data sharing protocols and enforcing standards
Experience in creating credentials for authorized workers
IT skills for tracking and monitoring access to data systems
Example:
“Working closely with the network and systems administrators, I enforced authorization and authentication practices for data sharing between departments and outside or remote users.”

Q:

As a data manager, you recommend new technological changes. How do you arrive at your recommendations?
A:

A data manager is responsible for evaluating how current systems, software, hardware and data storage devices perform. The services must meet current IT standards and federal laws as well. He or she recommends alterations when the current systems aren’t serving your company at top levels or fail to comply with standards through upgrades. The applicant creates a full report conveying their recommendations for the changes and presents feasibility studies showing why the updates are necessary.

What to look for in an answer:

Experience with IT systems and new integration recommendations
Strong analytical and research skills
An understanding of budgetary constraints and feasibility studies
Example:
“I conducted research for the latest software, hardware and data storage options for the company through IT seminars and reports explaining the benefits of the investments.”

Q:

Why is a disaster recovery plan vital for all companies using data systems?
A:

A data manager devises disaster recovery plans for data storage systems. The protocol involves backing up files as soon as changes are made and removing the data storage media from the property daily. The backup media is used to restore the files if an attack happens that causes your company to lose all of its data or if the information becomes corrupted. He or she uses the same recovery plan if your business property is destroyed.

What to look for in an answer:

Experience implementing a disaster recovery plan
Ability to follow protocols when disaster recovery is needed
An understanding of the importance of protecting all data
Example:
“The disaster recovery plan mitigates the risk of data loss and helps companies protect their data from attacks that lead to data corruption and potential identity theft liabilities.”

Q:

What are the first steps you would take in the event of a security breach within your company database?
A:

Data managers are responsible for protecting the security of the data they collect. They should not only know how to prevent security breaches, they should know how to limit their negative impact if they occur. Interviewers should ask this question to assess whether a candidate understands best practices for emergency response to a breach. Strong candidates will have a general plan for determining if sensitive data was affected by a security issue and preventing similar problems in the future.

Look for these elements of a candidate's response:

An example action plan
Confidence
Learning from experience
Here is one good response:

Example:
"The first priority is to repair the issues that caused the breach to prevent further data leaks and take additional security measures to identify other vulnerabilities. I'd begin researching the extent of the breach to determine what kinds of information was released and the potential consequences of the security problem. This would help me be able to approach the people affected by the leak with potential solutions, showing accountability for our role in learning from new forms of cyber attacks

Yes very key

What level of data security do I need?
What kind of data do I need to analyze?
Cloud, on-prem, or both?
Which use cases do I need to focus on? What will the impact look like?
Will data management help me maintain compliance?

Data flow map is very essential to identifying who does what and assigning such responsibilities appropriately.

Data management is used by different stakeholders for different purposes. The goal of a knowledge based management system is to generate and share usable knowledge based on this data. Data needs to be properly stored, processed and shared, either physically or electronically.

When data are properly managed with well articulated data flow map, there is greater chances that the ultimate goal of the project is achieved.

What would be the best tip for best practices on data management on a large scale data collection (nationwide)?

On the agricultural sector in many provinces what would be the best structure?

What would be the best tip for best practices on data management on a large scale data collection (nationwide)?

On the agricultural sector in many provinces what would be the best structure?

I think data management is the most crucial part in project evaluation

It is very important to identify the roles each of your stakeholders can play then assign the necessary roles to them.

Data management is a very key and integral part for the M&E function. Data being a very sensitive resource, should handled through a process that is free from biases and errors.
There are a number of questions that any M&E team on any project should consider key throughout the data management process.

1. What are the sources of the data?
2. What type of data do we need from these sources?
3. How much detail do we need to obtain?
4. How shall we collect the data?
5. How often will the data be collected and who will collect the data?
6. How will the data be cleaned and analysed?
7. Who will analyse the data?
8. Who will write the reports?
9. Who are the end users of the data/reports and whats their level of interest?
10. How will the data be disseminated to the end users?
    Once these questions are answered, thenthe data management process becomes easy and simple.

When data are properly managed with well articulated data flow map, there is greater chances that the ultimate goal of the project is achieved.

In a situation where, for reasons that border on scarcity of personnel in an organization, where can the data management role access adequate capacity building in data analysis, seeing that this is a very important step in the data management process?

What kind of data will you need to collect to measure your indicator? Detailed qualitative data will be collected very differently than quantitative data. For example, surveys, laboratory measurements, and document reviews work well for collecting large amounts of quantitative data, while interviews and focus groups are often the best choices for collecting detailed qualitative data.

What can be the problems in the case where one person plays the roles from data collection up to reporting?

What level of data security do I need?
What kind of data do I need to analyze?
Which use cases do I need to focus on? What will the impact look like?
Will data management help me maintain compliance?

Please, what is actually data security? I am not an expert in this field. Does it implies confidentiality considering that there are different actors involved and at different levels? Could it be about preventing from damage in storage media? I am imagining data being blocked in failed hard disk or corrupted by viruses. It will be very good for me to understand this data security through your experiences.

Data management is a process of data collection, data entry and collation, data analysis, verification, storage or archiving and finally data use in drawing conclusions and decision making.

data flow mapping helps visualization of data management process in all its complexity making it easy to understand and assign roles and responsibilities to the human interface in the data management cycle.

A simple project which has limited measurable indicators and if one has the prerequisite capacity one can perform all the data processing stages from data collection to date analysis, verification and storage. case example of a laboratory research.