Explore topic-wise InterviewSolutions in .

A data cube is a multidimensional data MODEL that stores optimized, SUMMARIZED, or aggregated data for quick and EASY ANALYSIS using OLAP technologies. The precomputed data is stored in a data cube, which makes online analytical processing easier. We all think of a cube as a three-dimensional structure, however in data warehousing, an n-dimensional data cube can be implemented. A data cube stores information in terms of dimensions and facts.

Data Cubes have two categories. They are as follows :

• Multidimensional Data Cube : Data is stored in multidimensional arrays, which allows for a multidimensional view of the data. A multidimensional data cube aids in the storage of vast amounts of information. A multidimensional data cube uses indexing to represent each dimension of the data cube, making it easier to access, retrieve, and STORE data.
• Relational Data Cube : The relational data cube can be thought of as an "expanded version of relational DBMS." Data is stored in relational tables, and each relational table represents a data cube's dimension. The relational data cube uses SQL to produce aggregated data, although it is slower than the multidimensional data cube in terms of performance. The relational data cube, on the other hand, is scalable for data that grows over time.

Snowflake Schema is a MULTIDIMENSIONAL model that is also used in data WAREHOUSES. The FACT TABLES, dimension tables, and sub dimension tables are all contained in the snowflake schema. With fact tables, dimension tables, and sub-dimension tables, this schema forms a snowflake.

Star schema is a sort of MULTIDIMENSIONAL model and is USED in a data warehouse. The fact tables and dimension tables are both contained in the star schema. There are fewer foreign-key JOINS in this design. With fact and dimension tables, this schema forms a star.

Some of the renowned ETL tools currently used in the INDUSTRY are as FOLLOWS :

• Informatica
• Talend
• Pentaho
• Abnitio
• Oracle DATA Integrator
• Xplenty
• Skyvia
• Microsoft – SQL Server Integrated Services (SSIS)

Some of the major data WAREHOUSE SOLUTIONS currently being used in the INDUSTRY are as follows :

• Snowflakes
• Oracle Exadata
• Apache Hadoop
• SAP BW4HANA
• Microfocus Vertica
• Teradata
• AWS Redshift
• GCP BIG QUERY

Metadata is defined as information about data. Metadata is the context that provides data a more complete identity and serves as the foundation for its interactions with other data. It can also be a useful tool for saving TIME, staying organised, and getting the most out of the files you're working with. Structural Metadata describes how an object should be classified in ORDER to fit into a wider system of things. Structural Metadata makes a LINK with other files that allows them to be categorized and used in a variety of WAYS. Administrative Metadata contains information about an object's history, who owned it previously, and what it can be used for. Rights, licences, and permissions are EXAMPLES. This information is useful for persons who are in charge of managing and caring for an asset.

When a piece of information is placed in the correct context, it takes on a whole new meaning. Furthermore, better-organized Metadata will considerably reduce search time.

 Following are the characteristics of a data warehouse:-

• Subject-oriented : Because it distributes information about a theme rather than an organization's actual operations, a data warehouse is always subject-oriented. It is possible to do so with a certain theme. That is to say, the data warehousing procedure is intended to deal with a more defined theme. These themes could include sales, distribution, and marketing, for example. The focus of a data warehouse is never solely on present activities. Instead, it CONCENTRATES on demonstrating and analyzing evidence in order to reach diverse conclusions. It also provides a simple and precise demonstration around a specific theme by removing info that isn't needed to make conclusions.
• Integrated : It is similar to subject orientation in that it is created in a dependable format. INTEGRATION entails the creation of a single entity to scale all related data from several databases. The data has to be STORED in several data warehouses in a shared and widely accessible manner. A data warehouse is created by combining information from a variety of sources, such as a mainframe and a relational database. It must also have dependable naming conventions, formats, and codes. The utilization of a data warehouse allows for more effective data analysis. The consistency of name conventions, column scaling, and encoding structure, among other things, should be validated. The data warehouse integration handles a variety of subject-related warehouses.
• Time-Variant : Data is kept in this system at various time intervals, such as weekly, monthly, or annually. It discovers a number of time limits that are structured between massive datasets and held in the online transaction process (OLTP). Data warehouse time limitations are more flexible than those of operational systems. The data in the data warehouse is predictable over a set period of time and provides information from a historical standpoint. It contains explicit or implicit time elements. Another property of time-variance is that data cannot be edited, altered, or updated once it has been placed in the data warehouse.
• Non-volatile : The data in a data warehouse is permanent, as the name implies. It also means that when new data is put, it is not erased or REMOVED. It incorporates a massive amount of data that is placed into logical business alteration between the designated quantity. It assesses the analysis in the context of warehousing technologies. Data is read-only and REFRESHED at scheduled intervals. This is useful for analyzing historical data and understanding how things work. It is not required to have a transaction process, a recapture mechanism, or a concurrency control mechanism. In a data warehouse environment, operations like delete, update, and insert that are performed in an operational application are lost.

The technical capacity to collect TRANSACTIONS as they change and integrate them into the WAREHOUSE, as well as maintaining batch or planned cycle refreshes, is known as active data warehousing. Automating routine processes and choices is possible with an active data warehouse. The active data warehouse sends decisions to the On-Line Transaction Processing (OLTP) systems AUTOMATICALLY. An active data warehouse is designed to capture and distribute data in real time. They give you a unified VIEW of your customers across all of your BUSINESS lines. Business Intelligence Systems are linked to it.

A system that reflects the condition of the warehouse in REAL time is REFERRED to as real-time data WAREHOUSING. If you perform a query on the real-time data warehouse to learn more about a specific aspect of the company or entity described by the warehouse, the result reflects the status of that entity at the time the query was run. Most data warehouses contain data that is highly latent — that is, data that reflects the BUSINESS at a specific point in time. A real-time data warehouse provides current (or real-time) data with LOW latency.

A fact table with no measures is known as a factless fact table. It's essentially a crossroads of dimensions (it contains nothing but dimensional KEYS). One form of factless table is used to capture an event, while the other is used to describe conditions.

In the first type of factless fact table, there is no measured value for an event, but it develops the relationship among the dimension members from several dimensions. The existence of the relationship is itself the fact. This type of fact table can be utilised to create valuable reports on its own. Various CRITERIA can be used to count the number of occurrences.

The second type of factless fact table is a tool that's used to back up negative ANALYTICAL reports. Consider a store that did not sell a product for a period of time. To create such a report, you'll need a factless fact table that CAPTURES all of the conceivable product COMBINATIONS that were on offer. By comparing the factless table to the sales table for the list of things that did sell, you can figure out what's missing.

Database: A database is a logically organized collection of structured data kept electronically in a computer SYSTEM. A database MANAGEMENT system is usually in charge of a database (DBMS). The data, the DBMS, and the applications that go with them are referred to as a database system, which is commonly abbreviated to just a database.

The following table ENLISTS the difference between data warehouse and database:-

Following are the different types of data mart in data warehousing:

• Dependent Data Mart: A dependent data mart can be developed using data from operational, EXTERNAL, or both sources. It enables the data of the source company to be accessed from a single data warehouse. All data is centralized, which can aid in the development of further data marts.
• Independent Data Mart: There is no need for a central data warehouse with this data mart. This is typically established for smaller GROUPS that exist within a company. It has no connection to ENTERPRISE Data Warehouse or any other data warehouse. Each piece of information is self-contained and can be used independently. The analysis can also be carried out independently. It's critical to maintain a consistent and centralized data repository that numerous users can access.
• Hybrid Data Mart: A hybrid data mart is utilized when a data warehouse contains inputs from multiple sources, as the name implies. When a user requires an ad hoc integration, this feature comes in handy. This solution can be utilized if an organization requires various database environments and quick implementation. It necessitates the least amount of data purification, and the data mart MAY accommodate huge storage structures. When smaller data-centric applications are employed, a data mart is most effective.

Following are the different types of data warehouse:

• Enterprise Data Warehouse:
  • An enterprise database is a database that brings together the various functional areas of an organisation in a cohesive manner. It's a centralised location where all corporate data from various sources and apps can be accessed. They can be utilised for analytics and by everyone in the organisation once they've been saved. The data can be categorised by SUBJECT, and access is granted according to the necessary division. The tasks of extracting, converting, and conforming are taken care of in an Enterprise Datawarehouse.
  • Enterprise Datawarehouse's purpose is to provide a comprehensive overview of any object in the data model. This is performed by FINDING and wrangling the data from different systems. This is then loaded into a model that is consistent and conformed. The data is acquired by Enterprise Datawarehouse, which can provide access to a single site where various tools can be used to execute analytical functions and generate various predictions. New trends or patterns can be identified by RESEARCH teams, which can then be focused on to help the company expand.
• Operational Data Store (ODS):
  • An operational data store is utilised instead of having an operational decision support system application. It facilitates data access directly from the database, as well as TRANSACTION processing. By checking the associated business rules, the data in the Operational Data Store may be cleansed, and any redundancy found can be checked and rectified. It also aids in the integration of disparate data from many sources so that business activities, analysis, and reporting may be carried out quickly and effectively while the process is still ongoing. 
  • The majority of current operations are stored here before being migrated to the data warehouse for a longer period of time. It is particularly useful for simple searches and little AMOUNTS of data. It functions as short-term or temporary memory, storing recent data. The data warehouse keeps data for a long time and also keeps information that is generally permanent.
• Data Mart:
  • Data Mart is referred to as a pattern to get client data in a data warehouse environment. It's a data warehouse-specific structure that's employed by the team's business domain. Every company has its own data mart, which is kept in the data warehouse repository. Dependent, independent, and hybrid data marts are the three types of data marts. Independent data marts collect data from external sources and data warehouses, whereas dependent data marts take data that has already been developed. Data marts can be thought of as logical subsets of a data warehouse.

Following are the disadvantages of using a data warehouse:-

• Loading time of data resources is undervalued: We frequently underestimate the time it will take to gather, sanitize, and post data to the warehouse. Although some resources are in place to minimize the time and effort spent on the process, it may require a significant amount of the overall PRODUCTION time.
• Source system flaws that go unnoticed: After years of non-discovery, hidden flaws linked with the source networks that provide the data warehouse may be discovered. Some fields, for example, may accept NULLS when entering new property information, resulting in workers inputting INCOMPLETE property data, even if it was available and relevant.
• Homogenization of data: Data warehousing ALSO deals with data formats that are comparable across DIVERSE data sources. It's possible that some important data will be lost as a result.

Following are the advantages of using a data warehouse:

• Helps you save time:
  • To stay ahead of your competitors in today's fast-paced world of cutthroat competition, your company's ability to make smart judgments quickly is critical.
  • A Data warehouse gives you instant access to all of your essential data, so you and your staff don't have to worry about missing a deadline. All you have to do now is deploy your data model to start collecting data in a matter of seconds. You can do this with most warehousing solutions without utilising a sophisticated query or machine learning.
  • With data warehousing, your company won't have to rely on a technical professional to troubleshoot data RETRIEVAL issues 24 hours a day, seven days a week. You will save a lot of time this way.
• Enhances the quality of data:
  • The high-quality data ensures that your company's policies are founded on accurate information about your operations.
  • You can turn data from numerous sources into a shared structure using data warehousing. You can assure the consistency and integrity of your company's data this way. This allows you to spot and eliminate duplicate data, inaccurately reported data and disinformation.
  • For your firm, implementing a data quality management PROGRAM may be both costly and time-consuming. You can easily use a data warehouse to reduce the number of these annoyances while saving money and increasing the general productivity of your company.
• Enhances Business Intelligence (BI):
  • Throughout your commercial endeavours, you can use a data warehouse to gather, absorb, and derive data from any source. As a result of the capacity to easily consolidate data from several sources, your BI will improve by leaps and bounds.
• Data standardization and Consistency are achieved:
  • The uniformity of huge data is another key benefit of having central data repositories. In a similar manner, a data storage or data mart MIGHT benefit your company. Because data warehousing stores data from various sources in a consistent manner, such as a transactional system, each source will produce results that are synchronized with other sources. This ensures that data is of higher quality and homogeneous. As a result, you and your team can rest assured that your data is accurate, resulting in more informed corporate decisions.
• Enhances Data Security:
  • A data warehouse improves security by incorporating cutting-edge security features into its design. For any business, consumer data is a vital resource. You can keep all of your data sources integrated and properly protected by adopting a warehousing solution. The risk of a data breach will be GREATLY reduced as a result of this.
• Ability to store historical data:
  • Because a data warehouse can hold enormous amounts of historical data from operational systems, you can readily study DIFFERENT time periods and inclinations that could be game-changing for your business. You can make better corporate judgments about your business plans if you have the correct facts in your hands.

The record of a reality or fact table could be made up of attributes from various dimension tables. The Fact Table, also known as the Reality Table, assists the user in investigating the business aspects that aid him in call taking in order to improve his FIRM. Dimension Tables, on the other hand, make it easier for the reality table or fact table to collect dimensions from which measurements must be taken.

The following table enlists the difference between a fact table and a dimension table:-

Following are the different types of dimension tables in the context of data warehousing:-

• Slowly CHANGING Dimensions (SCD): Slowly changing dimensions are dimension attributes that tend to vary slowly over time RATHER than at a regular period of time. For example, the address and phone number may change, but not on a regular basis. Consider the case of a man who travels to several nations and must change his address according to the place he is visiting. This can be accomplished in ONE of three ways:
  • Type 1: Replaces the value that was PREVIOUSLY entered. This strategy is simple to implement and aids in the reduction of costs by saving space. However, in this circumstance, history is lost.
  • Type 2: Insert a new row containing the new value. This method saves the history and allows it to be accessed at any time. However, it takes up a lot of space, which raises the price.
  • Type 3: Add a new column to the table. It is the ideal strategy because history can be easily preserved.
• Junk Dimension: A trash dimension is a collection of low-cardinality attributes. It contains a number of varied or disparate features that are unrelated to one another. These can be used to implement RCD (rapidly changing dimension) features like flags and weights, among other things.
• Conformed Dimension: Multiple subject areas or data marts share this dimension. It can be utilised in a variety of projects without requiring any changes. This is used to keep things in order. Dimensions that are exactly the same as or a proper subset of any other dimension are known as conformed dimensions.
• Roleplay Dimension: Role-play dimension refers to the dimension table that has many relationships with the fact table. In other words, it occurs when the same dimension key and all of its associated attributes are linked to a large number of foreign keys in the fact table. Within the same database, it might serve several roles.
• Degenerate Dimension: Degenerate dimension attributes are those that are contained in the fact table itself rather than in a separate dimension table. For instance, a ticket number, an invoice number, a transaction number, and so on.

A table in a DATA warehouse's star SCHEMA is referred to as a dimension table. Dimensional data models, which are made up of fact and dimension tables, are used to create data warehouses. Dimension tables contain dimension keys, values, and attributes and are used to describe dimensions. It is usually of a tiny size. The number of rows might range from a few to thousands. It is a DESCRIPTION of the objects in the fact table. The term "dimension table" refers to a collection or group of data pertaining to any quantifiable occurrence. They serve as the foundation for dimensional modelling. It includes a column that serves as a primary key, allowing each dimension ROW or record to be uniquely identified. Through this key, it is linked to the fact tables. When it's constructed, a system-generated key called the surrogate key is used to uniquely identify the rows in the dimension.

Following are the advantages of using a dimension table :

• It features a straightforward design.
• It is SIMPLE to study and comprehend.
• It stores data that has been de-normalized.
• It aids in the preservation of historical data for any dimension.
• It's simple to get info from it.
• It's simple to build and put into action.
• It provides the context for any business operation.

 In a Data Warehouse system, a Fact table is simply a table that holds all of the facts or business information that can be exposed to reporting and analysis when needed. Fields that reflect direct facts, as well as foreign fields that connect the fact table to other dimension tables in the Data Warehouse system, are stored in these tables. Depending on the model type used to construct the Data Warehouse, a Data Warehouse system can have ONE or more fact tables.

Following are the three types of fact tables:-

• Transactional Fact Table: This is a very basic and fundamental view of corporate processes. It can be used to depict the occurrence of an event at any given time. The facts measure are only valid at that specific time and for that specific incident. "One row per line in a transaction," according to the grain associated with the transaction table. It typically comprises data at the detailed level, resulting in a huge number of dimensions linked with it. It captures the smallest or atomic level of dimension measurement. This allows the table to provide users with extensive dimensional grouping, roll-up, and drill-down reporting features. It's packed yet sparse at the same time. It can also be big at the same time, depending on the number of events (transactions) that have occurred.
• Snapshot Fact Table: The snapshot depicts the condition of things at a specific point in time, sometimes known as a "picture of the MOMENT." It usually contains a GREATER number of non-additive and semi-additive information. It aids in the examination of the company's overall performance at regular and predictable times. Unlike the transaction fact table, which adds a new row for each occurrence of an event, this represents the performance of an activity at the end of each day, week, month, or any other time interval. However, to retrieve the detailed data in the transaction fact table, snapshot fact tables or periodic snapshots rely on the transaction fact table. The periodic snapshot tables are typically large and TAKE up a lot of space.
• Accumulating Fact Table: These are used to depict the activity of any process with a well-defined beginning and end. Multiple data stamps are commonly found in accumulating snapshots, which reflect the predictable stages or events that occur over the course of a lifespan. There is sometimes an EXTRA column with the date that indicates when the row was last updated.

OLAP is an acronym for On-Line Analytical Processing. OLAP is a software technology classification that allows analysts, managers, and executives to get insight into information through quick, reliable, interactive access to data that has been converted from raw data to reflect the true dimensionality of the company as perceived by the clients. OLAP allows for multidimensional examination of corporate data while also allowing for complex estimations, trend analysis, and advanced data modelling. It's rapidly improving the foundation for Intelligent Solutions, which includes Business Performance Management, Strategy, Budgeting, Predicting, Financial Documentation, Analysis, Modeling, Knowledge Discovery, and Data Warehouses Reporting. End-clients can use OLAP to perform ad hoc record analysis in several dimensions, giving them the information and understanding they need to make better choices.

Following guidelines must be followed while selecting an OLAP system:-

• Multidimensional Conceptual View: This is one of an OLAP system's most important capabilities. It is feasible to use methods like slice and dice that require a multidimensional view.
• Transparency: Make the technology, the underlying data repository, computing operations, and the disparate NATURE of source data completely accessible to consumers. Users' efficiency and productivity are improved as a result of this transparency.
• Accessibility: OLAP systems must only allow access to the data that is truly needed to do the analysis, giving clients a single, coherent, and consistent picture. The OLAP system must map its own logical schema to the disparate physical data storage, as WELL as to conduct any required transformations. 
• Consistent Reporting Performance: As the number of dimensions or the size of the database grows, users should not experience any substantial reduction in documenting performance. That is, as the number of dimensions grows, OLAP performance should not deteriorate.
• Client/Server Architecture: Make the OLAP tool's server component clever enough that the various clients can be connected with minimal effort and integration code. The server should be able to map and consolidate data from disparate databases.
• Generic Dimensionality: Each dimension in an OLAP method should be seen as equal in terms of structure and OPERATIONAL capabilities. Select dimensions may be granted additional operational capabilities, although such duties should be available to all dimensions.
• Dynamic Sparse Matrix Handling: To optimise sparse matrix handling by adapting the physical schema to the unique analytical model being built and loaded. When confronted with a sparse matrix, the system must be able to dynamically assume the information distribution and change storage and access in order to achieve and maintain a constant level of performance.
• Multiuser Support: OLAP technologies must allow several users to access data at the same time while maintaining data integrity and security.
• Unrestricted cross-dimensional Operations: It gives techniques the ability to determine dimensional order and to perform roll-up and drill-down operations within and across dimensions.
• Intuitive Data Manipulation: Reorientation (pivoting), drill-down and roll-up, and other manipulations can be done INTUITIVELY and precisely on the cells of the scientific model using point-and-click and drag-and-drop methods. It does away with the need for a menu or several VISITS to the user interface.
• Flexible Reporting: It provides efficiency to corporate clients by allowing them to organize columns, rows, and cells in a way that allows for easy data manipulation, analysis, and synthesis.
• Infinite Dimensions and Aggregation Levels: There should be no limit to the number of data dimensions. Within any given consolidation path, each of these common dimensions must allow for an almost infinite number of customer-defined aggregation levels.

Data mining is the process of collecting information in order to find PATTERNS, trends, and usable data that will help a company to make data-driven decisions from large amounts of data. In other WORDS, Data Mining is the method of analysing hidden patterns of data from VARIOUS perspectives for categorization into useful data, which is gathered and assembled in specific areas such as data warehouses, EFFICIENT analysis, data mining algorithm, assisting decision making, and other data requirements, ultimately resulting in cost-cutting and revenue generation. Data mining is the process of automatically examining enormous amounts of data for patterns and trends that go beyond simple analysis. Data mining estimates the probability of future events by utilising advanced mathematical algorithms for data segments.

Following are the differences between data warehousing and data mining:-