Explore topic-wise InterviewSolutions in Current Affairs.

Criteria for a good data model includes:

• It can be easily consumed
• Large data CHANGES in a good model should be scalable
• It should PROVIDE predictable performance
• A good model can ADAPT to changes in REQUIREMENTS.

Criteria for a good data model includes:

N-gram:

An n-gram is a CONTIGUOUS sequence of n ITEMS from a given sequence of TEXT or speech. It is a type of probabilistic LANGUAGE model for PREDICTING the next item in such a sequence in the form of a (n-1).

N-gram:

An n-gram is a contiguous sequence of n items from a given sequence of text or speech. It is a type of probabilistic language model for predicting the next item in such a sequence in the form of a (n-1).

Although single imputation is widely USED, it does not reflect the UNCERTAINTY created by missing data at random. So, MULTIPLE imputation is more favorable then single imputation in case of data missing at random.

Although single imputation is widely used, it does not reflect the uncertainty created by missing data at random. So, multiple imputation is more favorable then single imputation in case of data missing at random.

During IMPUTATION we replace missing data with substituted values.

The types of imputation techniques INVOLVE are:

Single Imputation

Hot-deck imputation: A missing value is imputed from a randomly SELECTED similar record by the help of punch card

Cold deck imputation: It works same as hot deck imputation, but it is more ADVANCED and selects donors from another datasets

Mean imputation: It involves replacing missing value with the mean of that variable for all other cases

Regression imputation: It involves replacing missing value with the predicted values of a variable based on other variables

Stochastic regression: It is same as regression imputation, but it adds the average regression VARIANCE to regression imputation

Multiple Imputation:

Unlike single imputation, multiple imputation estimates the values multiple times

During imputation we replace missing data with substituted values.

The types of imputation techniques involve are:

Single Imputation

Hot-deck imputation: A missing value is imputed from a randomly selected similar record by the help of punch card

Cold deck imputation: It works same as hot deck imputation, but it is more advanced and selects donors from another datasets

Mean imputation: It involves replacing missing value with the mean of that variable for all other cases

Regression imputation: It involves replacing missing value with the predicted values of a variable based on other variables

Stochastic regression: It is same as regression imputation, but it adds the average regression variance to regression imputation

Multiple Imputation:

Unlike single imputation, multiple imputation estimates the values multiple times

A hash table COLLISION happens when two different keys hash to the same value. Two DATA cannot be stored in the same slot in array.

To avoid hash table collision there are many techniques, here we list out two:

Separate Chaining:

It uses the data structure to store multiple ITEMS that hash to the same slot.

Open addressing:

It SEARCHES for other slots USING a second function and store item in first empty slot that is found

A hash table collision happens when two different keys hash to the same value. Two data cannot be stored in the same slot in array.

To avoid hash table collision there are many techniques, here we list out two:

Separate Chaining:

It uses the data structure to store multiple items that hash to the same slot.

Open addressing:

It searches for other slots using a second function and store item in first empty slot that is found

In computing, a hash table is a map of KEYS to values. It is a data structure used to implement an associative ARRAY. It uses a hash function to compute an INDEX into an array of SLOTS, from which DESIRED value can be fetched.

In computing, a hash table is a map of keys to values. It is a data structure used to implement an associative array. It uses a hash function to compute an index into an array of slots, from which desired value can be fetched.

A correlogram analysis is the common FORM of SPATIAL analysis in geography. It consists of a series of estimated autocorrelation coefficients CALCULATED for a different spatial relationship. It can be used to construct a correlogram for distance-based data, when the raw data is EXPRESSED as distance RATHER than values at individual points.

A correlogram analysis is the common form of spatial analysis in geography. It consists of a series of estimated autocorrelation coefficients calculated for a different spatial relationship. It can be used to construct a correlogram for distance-based data, when the raw data is expressed as distance rather than values at individual points.

TIME series analysis can be done in two domains, frequency DOMAIN and the time domain. In Time series analysis the output of a particular process can be forecast by analyzing the previous data by the HELP of various METHODS like EXPONENTIAL smoothening, log-linear regression method, etc.

Time series analysis can be done in two domains, frequency domain and the time domain. In Time series analysis the output of a particular process can be forecast by analyzing the previous data by the help of various methods like exponential smoothening, log-linear regression method, etc.

Statistical methods that are useful for data SCIENTIST are:

• BAYESIAN method
• Markov process
• SPATIAL and cluster PROCESSES
• Rank statistics, percentile, outliers DETECTION
• Imputation techniques, etc.
• Simplex algorithm
• Mathematical optimization

Statistical methods that are useful for data scientist are:

Clustering is a classification METHOD that is applied to data. Clustering algorithm DIVIDES a data set into natural groups or clusters.

Properties for clustering algorithm are:

• Hierarchical or FLAT
• Iterative
• HARD and soft
• Disjunctive

Clustering is a classification method that is applied to data. Clustering algorithm divides a data set into natural groups or clusters.

Properties for clustering algorithm are:

Map-reduce is a framework to process large DATA SETS, splitting them into subsets, processing each subset on a DIFFERENT SERVER and then blending results obtained on each.

Map-reduce is a framework to process large data sets, splitting them into subsets, processing each subset on a different server and then blending results obtained on each.

KPI: It stands for Key Performance Indicator, it is a metric that CONSISTS of any combination of spreadsheets, reports or charts about business process

Design of experiments: It is the initial process used to split your DATA, sample and SET up of a data for statistical analysis

80/20 rules: It means that 80 percent of your income comes from 20 percent of your clients.

KPI: It stands for Key Performance Indicator, it is a metric that consists of any combination of spreadsheets, reports or charts about business process

Design of experiments: It is the initial process used to split your data, sample and set up of a data for statistical analysis

80/20 rules: It means that 80 percent of your income comes from 20 percent of your clients.

Collaborative filtering is a simple ALGORITHM to CREATE a recommendation system based on user behavioral data. The most important components of collaborative filtering are users- items- INTEREST.

A good example of collaborative filtering is when you SEE a statement LIKE “recommended for you” on online shopping sites that’s pops out based on your browsing history.

Collaborative filtering is a simple algorithm to create a recommendation system based on user behavioral data. The most important components of collaborative filtering are users- items- interest.

A good example of collaborative filtering is when you see a statement like “recommended for you” on online shopping sites that’s pops out based on your browsing history.

A data scientist must have the following skills:

Database knowledge

• Database management
• Data BLENDING
• QUERYING
• Data manipulation

Predictive Analytics

• Basic descriptive statistics
• Predictive modeling
• Advanced analytics

Big Data Knowledge

• Big data analytics
• Unstructured data analysis
• Machine learning

Presentation SKILL

• Data VISUALIZATION
• Insight presentation
• Report design

A data scientist must have the following skills:

Database knowledge

Predictive Analytics

Big Data Knowledge

Presentation skill

K mean is a famous partitioning METHOD. Objects are classified as BELONGING to ONE of K groups, k chosen a priori.

In K-mean algorithm:

• The clusters are spherical: the DATA points in a cluster are centered around that cluster
• The variance/spread of the clusters is similar: Each data point belongs to the closest cluster.

K mean is a famous partitioning method. Objects are classified as belonging to one of K groups, k chosen a priori.

In K-mean algorithm:

Hierarchical clustering algorithm combines and DIVIDES existing groups, CREATING a hierarchical STRUCTURE that showcase the order in which groups are DIVIDED or merged.

Hierarchical clustering algorithm combines and divides existing groups, creating a hierarchical structure that showcase the order in which groups are divided or merged.

The outlier is a commonly used TERMS by analysts REFERRED for a value that appears far away and DIVERGES from an overall pattern in a sample.

There are two types of Outliers:

1. UNIVARIATE
2. Multivariate

The outlier is a commonly used terms by analysts referred for a value that appears far away and diverges from an overall pattern in a sample.

There are two types of Outliers:

To DEAL the multi-source problems:

• Restructuring of schemas to accomplish a schema integration
• Identify similar RECORDS and merge them into single record containing all relevant ATTRIBUTES without redundancy.

To deal the multi-source problems:

• Prepare a validation report that gives information of all suspected data. It should give information like validation criteria that it failed and the date and TIME of OCCURRENCE
• Experience PERSONNEL should examine the suspicious data to determine their acceptability
• Invalid data should be assigned and replaced with a validation code
• To work on missing data use the best analysis strategy like deletion method, single imputation methods, model based methods, etc.

Usually, METHODS used by DATA analyst for data VALIDATION are:

1. Data screening
2. Data verification

Usually, methods used by data analyst for data validation are:

In KNN imputation, the missing attribute VALUES are imputed by USING the attributes value that are most SIMILAR to the attribute whose values are missing. By using a distance function, the similarity of two attributes is DETERMINED.

In KNN imputation, the missing attribute values are imputed by using the attributes value that are most similar to the attribute whose values are missing. By using a distance function, the similarity of two attributes is determined.

The MISSING patterns that are generally OBSERVED are:

1. Missing completely at RANDOM
2. Missing at random
3. Missing that depends on the missing VALUE itself
4. Missing that depends on unobserved input variable

The missing patterns that are generally observed are:

Hadoop and MapReduce is the programming framework developed by APACHE for processing LARGE DATA SET for an application in a distributed computing ENVIRONMENT.

Hadoop and MapReduce is the programming framework developed by Apache for processing large data set for an application in a distributed computing environment.

Some of the common problems faced by data ANALYST are:

1. Common MISSPELLING
2. Duplicate entries
3. Missing VALUES
4. ILLEGAL values
5. Varying value representations
6. Identifying overlapping data

Some of the common problems faced by data analyst are:

The difference between DATA mining and data PROFILING is that:

Data profiling: It targets on the instance analysis of individual attributes. It gives information on VARIOUS attributes like value RANGE, discrete value and their frequency, occurrence of null values, data type, length, etc.

Data mining: It focuses on cluster analysis, detection of unusual records, dependencies, sequence discovery, RELATION holding between several attributes, etc.

The difference between data mining and data profiling is that:

Data profiling: It targets on the instance analysis of individual attributes. It gives information on various attributes like value range, discrete value and their frequency, occurrence of null values, data type, length, etc.

Data mining: It focuses on cluster analysis, detection of unusual records, dependencies, sequence discovery, relation holding between several attributes, etc.

• Tableau
• RapidMiner
• OpenRefine
• KNIME
• GOOGLE SEARCH OPERATORS
• Solver
• NodeXL
• io
• WOLFRAM Alpha’s
• Google Fusion TABLES

Logistic regression is a STATISTICAL METHOD for EXAMINING a DATASET in which there are ONE or more independent variables that defines an outcome.

Logistic regression is a statistical method for examining a dataset in which there are one or more independent variables that defines an outcome.

Some of the best practices for data cleaning includes:

• Sort data by DIFFERENT attributes
• For large datasets cleanse it stepwise and improve the data with each step until you achieve a good data quality
• For large datasets, break them into small data. Working with less data will increase your iteration SPEED
• To handle COMMON cleansing task create a set of utility functions/tools/scripts. It might include, remapping values based on a CSV file or SQL database or, regex search-and-replace, blanking out all values that don’t match a regex
• If you have an issue with data cleanliness, arrange them by estimated frequency and attack the most common problems
• Analyze the summary STATISTICS for each column ( standard deviation, mean, number of missing values,)
• KEEP track of every date cleaning operation, so you can alter changes or remove operations if required.

Some of the best practices for data cleaning includes:

Data CLEANING also referred as data cleansing, deals with identifying and removing errors and INCONSISTENCIES from data in order to enhance the QUALITY of data.

Data cleaning also referred as data cleansing, deals with identifying and removing errors and inconsistencies from data in order to enhance the quality of data.

Various STEPS in an analytics project include:

• PROBLEM definition
• DATA EXPLORATION
• Data PREPARATION
• Modelling
• Validation of data
• Implementation and tracking

Various steps in an analytics project include:

To become a data analyst:

• Robust knowledge on reporting packages (Business Objects), programming LANGUAGE (XML, Javascript, or ETL frameworks), databases (SQL, SQLite, etc.)
• Strong skills with the ability to analyze, organize, collect and disseminate big data with ACCURACY
• Technical knowledge in database design, data models, data MINING and segmentation techniques
• Strong knowledge on STATISTICAL packages for analyzing LARGE datasets (SAS, Excel, SPSS, etc.)

To become a data analyst:

Responsibility of a Data analyst include:

• Provide support to all data analysis and coordinate with customers and staffs
• Resolve business associated issues for CLIENTS and performing audit on data
• Analyze results and interpret data using statistical techniques and provide ongoing reports
• Prioritize business needs and work closely with management and information needs
• Identify new process or areas for IMPROVEMENT opportunities
• Analyze, identify and interpret trends or patterns in complex data SETS
• Acquire data from primary or secondary data SOURCES and maintain databases/data systems
• Filter and “clean” data, and review computer reports
• Determine performance indicators to locate and correct code problems
• Securing database by developing access system by determining user level of access.

Responsibility of a Data analyst include: