Data cleaning is the important phase in impala. Because this dataset contains some errors and
outliers. Some kind of dataset in only depend on the type of quantitative but it is not qualitative.
And more over this dataset contains some discrepancies in their codes or names. And it lacks our
attributes of interest for analyzing process.
The data preparation is most important task. Because it is used to correct the inconsistency of
data in the dataset. And it is also used for smooth the noise data in that dataset and fill the
missing values or missing attributes in that dataset. Sometimes dataset contains duplicate records
or inconsistent data or errors in random manner. These kind of issues will be solved in the part of
data preparation or cleaning. The main advantage for doing this data cleaning was to provide
data quality. In impala various noisy data’s are find out manually or cleaned manually by this
technique. Missing values will be cleaned by either removing those rows or filling values in that
row.
The basics steps for cleaning or preparing the data:
Editing: It is used to correct the inconsistent data, incomplete data illegible data that are stored
in the impala table for processing.
Cleaning: It is used to clean the data from inconsistent data. If any fault logics or queries will be
applied it will removed using cleaning. It is used to remove the extreme values in the table.
Questionnaire checking: It is used to eliminate the unwanted questionnaires. If the
questionnaires may be little variant from another, instructions could not be followed or it may be
incomplete.
Coding: It is used to verify if the code is typically assigns a numeric codes to answer. And check
that if the statistical techniques will be applied or not.
Transcribing: It is used to transfer the data for easy accessing by people.
Strategy selection for analysis: choose the appropriate data analysis strategy for further
processing.
Statistical adjustments: It was the most important step in data cleaning. It is used to require the
correct scale transformations weighting for the analyzed data.
5

Assignment on Big data PDF

Contribute Materials

Related Documents

APACHE HADOOP VERSUS APACHE SPARK.

+13062052269

info@desklib.com