1.What are the responsibilities of a Data Analyst?
- Collects and analyzes data using statistical techniques and reports the results accordingly.
- Interpret and analyze trends or patterns in complex data sets.
- Establishing business needs together with business teams or management teams.
- Find opportunities for improvement in existing processes or areas.
- Data set commissioning and decommissioning.
- Follow guidelines when processing confidential data or information.
- Examine the changes and updates that have been made to the source production systems.
- Provide end-users with training on new reports and dashboards.
- Assist in the data storage structure, data mining, and data cleansing.
2. Write some key skills usually required for a data analyst.
Some of the key skills required for a data analyst include:
- Ability to analyze, organize, collect, and disseminate big data accurately and efficiently.
- The ability to design databases, construct data models, perform data mining, and segment data.
- Good understanding of statistical packages for analyzing large datasets (SAS, SPSS, Microsoft Excel, etc.).
- Effective Problem-Solving, Teamwork, and Written and Verbal Communication Skills.
- Excellent at writing queries, reports, and presentations.
- Understanding of data visualization software including Tableau and Qlik.
- The ability to create and apply the most accurate algorithms to datasets for finding solutions.
3. What is the data analysis process?
Data analysis generally refers to the process of assembling, cleaning, interpreting, transforming, and modeling data to gain insights or conclusions and generate reports to help businesses become more profitable. The following diagram illustrates the various steps involved in the process:
- Collect Data: The data is collected from a variety of sources and is then stored to be cleaned and prepared. This step involves removing all missing values and outliers.
- Analyse Data: As soon as the data is prepared, the next step is to analyze it. Improvements are made by running a model repeatedly. Following that, the model is validated to ensure that it is meeting the requirements.
- Create Reports: In the end, the model is implemented, and reports are generated as well as distributed to stakeholders.
4. What are the different challenges one faces during data analysis?
While analyzing data, a Data Analyst can encounter the following issues:
- Duplicate entries and spelling errors. Data quality can be hampered and reduced by these errors.
- The representation of data obtained from multiple sources may differ. It may cause a delay in the analysis process if the collected data are combined after being cleaned and organized.
- Another major challenge in data analysis is incomplete data. This would invariably lead to errors or faulty results.
- You would have to spend a lot of time cleaning the data if you are extracting data from a poor source.
- Business stakeholders’ unrealistic timelines and expectations
- Data blending/ integration from multiple sources is a challenge, particularly if there are no consistent parameters and conventions
- Insufficient data architecture and tools to achieve the analytics goals on time.
5. Explain data cleansing.
Data cleaning, also known as data cleansing or data scrubbing or wrangling, is basically a process of identifying and then modifying, replacing, or deleting the incorrect, incomplete, inaccurate, irrelevant, or missing portions of the data as the need arises. This fundamental element of data science ensures data is correct, consistent, and usable.
6. What are the tools useful for data analysis?
Some of the tools useful for data analysis include:
- Google Search Operators
- Google Fusion Tables
- Wolfram Alpha
- Tableau, etc.
7. Write the difference between data mining and data profiling.
Data mining Process: It generally involves analyzing data to find relations that were not previously discovered. In this case, the emphasis is on finding unusual records, detecting dependencies, and analyzing clusters. It also involves analyzing large datasets to determine trends and patterns in them.
Data Profiling Process: It generally involves analyzing that data’s individual attributes. In this case, the emphasis is on providing useful information on data attributes such as data type, frequency, etc. Additionally, it also facilitates the discovery and evaluation of enterprise metadata.
|Data Mining||Data Profiling|
|It involves analyzing a pre-built database to identify patterns.||It involves analyses of raw data from existing datasets.|
|It also analyzes existing databases and large datasets to convert raw data into useful information.||In this, statistical or informative summaries of the data are collected.|
|It usually involves finding hidden patterns and seeking out new, useful, and non-trivial data to generate useful information.||It usually involves the evaluation of data sets to ensure consistency, uniqueness, and logic.|
|Data mining is incapable of identifying inaccurate or incorrect data values.||In data profiling, erroneous data is identified during the initial stage of analysis.|
|Classification, regression, clustering, summarization, estimation, and description are some primary data mining tasks that are needed to be performed.||This process involves using discoveries and analytical methods to gather statistics or summaries about the data.|
8. Which validation methods are employed by data analysts?
In the process of data validation, it is important to determine the accuracy of the information as well as the quality of the source. Datasets can be validated in many ways. Methods of data validation commonly used by Data Analysts include:
- Field Level Validation: This method validates data as and when it is entered into the field. The errors can be corrected as you go.
- Form Level Validation: This type of validation is performed after the user submits the form. A data entry form is checked at once, every field is validated, and highlights the errors (if present) so that the user can fix them.
- Data Saving Validation: This technique validates data when a file or database record is saved. The process is commonly employed when several data entry forms must be validated.
- Search Criteria Validation: It effectively validates the user’s search criteria in order to provide the user with accurate and related results. Its main purpose is to ensure that the search results returned by a user’s query are highly relevant.
9. Explain Outlier.
In a dataset, Outliers are values that differ significantly from the mean of characteristic features of a dataset. With the help of an outlier, we can determine either variability in the measurement or an experimental error. There are two kinds of outliers i.e., Univariate and Multivariate. The graph depicted below shows there are four outliers in the dataset.
10. What are the ways to detect outliers? Explain different ways to deal with it.
Outliers are detected using two methods:
- Box Plot Method: According to this method, the value is considered an outlier if it exceeds or falls below 1.5*IQR (interquartile range), that is, if it lies above the top quartile (Q3) or below the bottom quartile (Q1).
- Standard Deviation Method: According to this method, an outlier is defined as a value that is greater or lower than the mean ± (3*standard deviation).
11. Write difference between data analysis and data mining.
Data Analysis: It generally involves extracting, cleansing, transforming, modeling, and visualizing data in order to obtain useful and important information that may contribute towards determining conclusions and deciding what to do next. Analyzing data has been in use since the 1960s.
Data Mining: In data mining, also known as knowledge discovery in the database, huge quantities of knowledge are explored and analyzed to find patterns and rules. Since the 1990s, it has been a buzzword.
|Data Analysis||Data Mining|
|Analyzing data provides insight or tests hypotheses.||A hidden pattern is identified and discovered in large datasets.|
|It consists of collecting, preparing, and modeling data in order to extract meaning or insights.||This is considered as one of the activities in Data Analysis.|
|Data-driven decisions can be taken using this way.||Data usability is the main objective.|
|Data visualization is certainly required.||Visualization is generally not necessary.|
|It is an interdisciplinary field that requires knowledge of computer science, statistics, mathematics, and machine learning.||Databases, machine learning, and statistics are usually combined in this field.|
|Here the dataset can be large, medium, or small, and it can be structured, semi-structured, and unstructured.||In this case, datasets are typically large and structured.|
12. Explain the KNN imputation method.
A KNN (K-nearest neighbor) model is usually considered one of the most common techniques for imputation. It allows a point in multidimensional space to be matched with its closest k neighbors. By using the distance function, two attribute values are compared. Using this approach, the closest attribute values to the missing values are used to impute these missing values.
13. Explain Normal Distribution.
Known as the bell curve or the Gauss distribution, the Normal Distribution plays a key role in statistics and is the basis of Machine Learning. It generally defines and measures how the values of a variable differ in their means and standard deviations, that is, how their values are distributed.
The above image illustrates how data usually tend to be distributed around a central value with no bias on either side. In addition, the random variables are distributed according to symmetrical bell-shaped curves.
14. What do you mean by data visualization?
The term data visualization refers to a graphical representation of information and data. Data visualization tools enable users to easily see and understand trends, outliers, and patterns in data through the use of visual elements like charts, graphs, and maps. Data can be viewed and analyzed in a smarter way, and it can be converted into diagrams and charts with the use of this technology.
15. How does data visualization help you?
Data visualization has grown rapidly in popularity due to its ease of viewing and understanding complex data in the form of charts and graphs. In addition to providing data in a format that is easier to understand, it highlights trends and outliers. The best visualizations illuminate meaningful information while removing noise from data.
16. Mention some of the python libraries used in data analysis.
Several Python libraries that can be used on data analysis include:
- SciKit, etc.
17. Explain a hash table.
Hash tables are usually defined as data structures that store data in an associative manner. In this, data is generally stored in array format, which allows each data value to have a unique index value. Using the hash technique, a hash table generates an index into an array of slots from which we can retrieve the desired value.
18. What do you mean by collisions in a hash table? Explain the ways to avoid it.
Hash table collisions are typically caused when two keys have the same index. Collisions, thus, result in a problem because two elements cannot share the same slot in an array. The following methods can be used to avoid such hash collisions:
- Separate chaining technique: This method involves storing numerous items hashing to a common slot using the data structure.
- Open addressing technique: This technique locates unfilled slots and stores the item in the first unfilled slot it finds
19. Write characteristics of a good data model.
An effective data model must possess the following characteristics in order to be considered good and developed:
- Provides predictability performance, so the outcomes can be estimated as precisely as possible or almost as accurately as possible.
- As business demands change, it should be adaptable and responsive to accommodate those changes as needed.
- The model should scale proportionally to the change in data.
- Clients/customers should be able to reap tangible and profitable benefits from it.
20. Write disadvantages of Data analysis.
The following are some disadvantages of data analysis:
- Data Analytics may put customer privacy at risk and result in compromising transactions, purchases, and subscriptions.
- Tools can be complex and require previous training.
- Choosing the right analytics tool every time requires a lot of skills and expertise.
- It is possible to misuse the information obtained with data analytics by targeting people with certain political beliefs or ethnicities.
21. Explain Collaborative Filtering.
Based on user behavioral data, collaborative filtering (CF) creates a recommendation system. By analyzing data from other users and their interactions with the system, it filters out information. This method assumes that people who agree in their evaluation of particular items will likely agree again in the future. Collaborative filtering has three major components: users- items- interests.