Explore topic-wise InterviewSolutions in .

The following SECTIONS describe ways you can adjust Data Integrator performance

•  Source-based performance options
•  Using ARRAY fetch size
•  Caching data
•  Join ordering
•  Minimizing extracted data
•  Target-based performance options
•  Loading method and rows per COMMIT
•  Staging tables to speed up auto-correct loads
•  JOB design performance options
•  Improving throughput
•  Maximizing the number of pushed-down operations
•  Minimizing data type conversion
•  Minimizing local conversion
•  Improving Informix repository performance

The following sections describe ways you can adjust Data Integrator performance

You must define AUDIT points and audit RULES when you want to audit a DATA FLOW.

You must define audit points and audit rules when you want to audit a data flow.

USE of the Case transform is to simplify branch LOGIC in data flows by CONSOLIDATING case or decision-making logic into one transform. The transform allows you to split a data SET into smaller sets based on logical BRANCHES.

Use of the Case transform is to simplify branch logic in data flows by consolidating case or decision-making logic into one transform. The transform allows you to split a data set into smaller sets based on logical branches.

CONSTRUCTS a complete hierarchy from PARENT/child relationships, and then PRODUCES a description of the hierarchy in vertically or HORIZONTALLY flattened format.

•  Parent COLUMN, Child Column
•  Parent Attributes, Child Attributes.

Constructs a complete hierarchy from parent/child relationships, and then produces a description of the hierarchy in vertically or horizontally flattened format.

The Map-Operation transform ALLOWS you to change operation CODES on DATA SETS to produce the desired OUTPUT. Operation codes: INSERT, UPDATE, DELETE, NORMAL or DISCARD.

The Map-Operation transform allows you to change operation codes on data sets to produce the desired output. Operation codes: INSERT, UPDATE, DELETE, NORMAL or DISCARD.

The History PRESERVING transform ALLOWS you to produce a NEW row in your target rather than updating an existing row. You can indicate in which columns the transform identifies changes to be preserved. If the VALUE of certain columns change, this transform creates a new row for each row flagged as UPDATE in the INPUT data set.

The History Preserving transform allows you to produce a new row in your target rather than updating an existing row. You can indicate in which columns the transform identifies changes to be preserved. If the value of certain columns change, this transform creates a new row for each row flagged as UPDATE in the input data set.

•  LOOKUP() : Briefly, It RETURNS SINGLE value based on single CONDITION
•  lookup_ext(): It returns MULTIPLE values based on single/multiple condition(s)
•  lookup_seq(): It returns multiple values based on sequence number

SPECIFIES the transaction SIZE in number of rows. If set to 1000, Data INTEGRATOR SENDS a commit to the underlying database every 1000 rows.

Specifies the transaction size in number of rows. If set to 1000, Data Integrator sends a commit to the underlying database every 1000 rows.

NUMBER of loaders loading with ONE LOADER is known as SINGLE loader Loading. Loading when the number of loaders is greater than one is known as Parallel Loading. The default number of loaders is 1. The maximum number of loaders is 5.

Number of loaders loading with one loader is known as Single loader Loading. Loading when the number of loaders is greater than one is known as Parallel Loading. The default number of loaders is 1. The maximum number of loaders is 5.

•  Row-by-row select - look up the TARGET table using SQL every time it receives an input row. This option is best if the target table is large.
•  Cached comparison table — To LOAD the comparison table into memory. This option is best when the table fits into memory and you are comparing the entire target table
•  Sorted input — To read the comparison table in the order of the primary key column(s) using sequential read.This option improves performance because DATA Integrator reads the comparison table only once.Add a query between the source and the Table_Comparison transform. Then, from the query’s input SCHEMA, drag the primary key columns into the Order By box of the query.

Array fetch SIZE indicates the number of rows retrieved in a single REQUEST to a source database. The default value is 1000. Higher numbers reduce requests, lowering network traffic, and possibly IMPROVE PERFORMANCE. The maximum value is 5000

Array fetch size indicates the number of rows retrieved in a single request to a source database. The default value is 1000. Higher numbers reduce requests, lowering network traffic, and possibly improve performance. The maximum value is 5000

It does not allow DUPLICATED data entering into the TARGET table.It WORKS like Type 1 Insert else Update the rows based on Non-matching and matching data respectively.

It does not allow duplicated data entering into the target table.It works like Type 1 Insert else Update the rows based on Non-matching and matching data respectively.

•  Using the auto-correct load option in the target table.
•  INCLUDING the Table COMPARISON transform in the DATA flow.
•  Designing the data flow to completely replace the target table during each execution.
•  Including a PRELOAD SQL statement to execute before the table loads.

Name match standards illustrate the MULTIPLE ways a name can be represented. They are used in the match process to greatly INCREASE match RESULTS.

Name match standards illustrate the multiple ways a name can be represented. They are used in the match process to greatly increase match results.

The Data CLEANSE transform can GENERATE name match standards and greetings. It can also assign GENDER codes and prenames such as Mr. and MRS.

The Data Cleanse transform can generate name match standards and greetings. It can also assign gender codes and prenames such as Mr. and Mrs.

Use the USA Regulatory transform if USPS certification and/or additional options such as DPV and Geocode are required. GLOBAL Address CLEANSE should be utilized when processing multi-country data.

Use the USA Regulatory transform if USPS certification and/or additional options such as DPV and Geocode are required. Global Address Cleanse should be utilized when processing multi-country data.

•  Data Cleanse: Parse names into GIVEN and family.
•  Address Cleanse: VALIDATE address INFORMATION.
•  Match: FIND DUPLICATES.

•  ENHANCEMENT Benefit
•  Determine gender distributions and TARGET
•  Gender Codes MARKETING CAMPAIGNS
•  PROVIDE fields for improving matching
•  Match Standards results

DIRECTORIES provide information on addresses from POSTAL authorities. Dictionary FILES are used to identify, parse, and standardize data such as NAMES, TITLES, and firm data.

Directories provide information on addresses from postal authorities. Dictionary files are used to identify, parse, and standardize data such as names, titles, and firm data.

The Data CLEANSE transform IDENTIFIES and isolates specific PARTS of mixed data, and standardizes your data based on information stored in the PARSING dictionary, business RULES defined in the rule file, and expressions defined in the pattern file.

The Data Cleanse transform identifies and isolates specific parts of mixed data, and standardizes your data based on information stored in the parsing dictionary, business rules defined in the rule file, and expressions defined in the pattern file.

These are packages that ENHANCE the ABILITY of DATA Cleanse to accurately process various forms of global data by including language-specific reference data and parsing RULES.

These are packages that enhance the ability of Data Cleanse to accurately process various forms of global data by including language-specific reference data and parsing rules.

•  Global_Address_Cleanse
•  Data_Cleanse
•  MATCH
•  ASSOCIATE
•  Country_id
•  USA_Regulatory_Address_Cleanse

• Data_Transfer
•  Date_Generation
•  Effective_Date
•  Hierarchy_Flattening
•  History_Preserving
•  Key_Generation
•  Map_CDC_Operation
•  PIVOT REVERSE Pivot
•  Table_Comparison
•  XML_Pipeline

ADAPTERS are additional Java-based programs that can be installed on the job server to provide connectivity to other SYSTEMS such as Salesforce.com or the JavaMessagingQueue. There is also a SoftwareDevelopment KIT (SDK) to allow customers to CREATE adapters for custom applications.

Adapters are additional Java-based programs that can be installed on the job server to provide connectivity to other systems such as Salesforce.com or the JavaMessagingQueue. There is also a SoftwareDevelopment Kit (SDK) to allow customers to create adapters for custom applications.

The MERGE TRANSFORM.

The Merge transform.

DISCRETE, MULTILINE, and HYBRID.

Discrete, multiline, and hybrid.

All lookup functions RETURN ONE ROW for each row in the source. They differ in how they choose which of several matching ROWS to return.

All lookup functions return one row for each row in the source. They differ in how they choose which of several matching rows to return.

Consider the following:

•  WHETHER or not the flows are INDEPENDENT of each other
•  Whether or not the server can handle the PROCESSING requirements of flows running at the same TIME (in parallel)

Consider the following:

INCORRECT SYNTAX, Job SERVER not RUNNING, port NUMBERS for Designer and Job Server not matching.

Incorrect syntax, Job Server not running, port numbers for Designer and Job Server not matching.

The VALUE that is constant in ONE environment, but MAY CHANGE when a JOB is migrated to another environment.

The Value that is constant in one environment, but may change when a job is migrated to another environment.

•  When the VARIABLE will need to be USED multiple times within a job.
•  When you WANT to reduce the development time required for passing values between job COMPONENTS.
•  When you need to create a dependency between job level GLOBAL variable name and job components.

A Parameter is an EXPRESSION that passes a PIECE of INFORMATION to a WORK flow, data flow or CUSTOM function when it is called in a job. A Variable is a symbolic placeholder for values.

A Parameter is an expression that passes a piece of information to a work flow, data flow or custom function when it is called in a job. A Variable is a symbolic placeholder for values.

A REPOSITORY is a SET of tables that HOLD SYSTEM objects, source and target metadata, and transformation rules. A Datastore is an actual connection to a database that holds data.

A Repository is a set of tables that hold system objects, source and target metadata, and transformation rules. A Datastore is an actual connection to a database that holds data.

The Data Services repository is a SET of tables that HOLDS user-created and PREDEFINED system objects, source and target metadata, and transformation rules. There are 3 types of repositories.

•  A LOCAL repository
•  A central repository
•  A profiler repository

The Data Services repository is a set of tables that holds user-created and predefined system objects, source and target metadata, and transformation rules. There are 3 types of repositories.

FILE FORMAT

File Format

A file format is a set of properties describing the STRUCTURE of a flat file (ASCII). File FORMATS describe the metadata structure. File format objects can describe files in:

•  Delimited format — CHARACTERS such as commas or tabs separate each field.
•  Fixed width format — The column width is specified by the user.
•  SAP ERP and R/3 format.

A file format is a set of properties describing the structure of a flat file (ASCII). File formats describe the metadata structure. File format objects can describe files in:

Data Services ALSO allows you to create a database datastore USING Memory as the Database type. Memory Datastores are designed to enhance processing performance of data flows EXECUTING in real-time JOBS.

Data Services also allows you to create a database datastore using Memory as the Database type. Memory Datastores are designed to enhance processing performance of data flows executing in real-time jobs.

REMOVE REDUNDANT and OBSOLETE OBJECTS from the repository tables.

Remove redundant and obsolete objects from the repository tables.

Three.

•  Database Datastores: provide a simple way to IMPORT metadata directly from RDBMS.
•  Application Datastores: LET users EASILY import metadata from most Enterprise Resource Planning (ERP) SYSTEMS.
•  ADAPTER Datastores: can provide access to an application’s data and metadata or just metadata.

Three.

A DATASTORE is a CONNECTION to a DATABASE.

A datastore is a connection to a database.

An EMBEDDED DATAFLOW is a dataflow that is CALLED from INSIDE ANOTHER dataflow.

An Embedded Dataflow is a dataflow that is called from inside another dataflow.

REAL-time jobs "extract" DATA from the body of the real time MESSAGE received and from any secondary SOURCES used in the JOB.

Real-time jobs "extract" data from the body of the real time message received and from any secondary sources used in the job.

A script is a single-use object that is used to CALL functions and ASSIGN VALUES in a workflow.

A script is a single-use object that is used to call functions and assign values in a workflow.

A TRANSFORM ENABLES you to CONTROL how datasets change in a DATAFLOW.

A transform enables you to control how datasets change in a dataflow.

JOB, WORKFLOW, DATAFLOW.

Job, Workflow, Dataflow.

PROJECT, JOB, WORKFLOW, DATAFLOW.

Project, Job, Workflow, Dataflow.

•  A job is the smallest UNIT of work that you can schedule independently for execution.
•  A work flow defines the decision-making process for executing data FLOWS.
•  Data flows EXTRACT, transform, and load data. Everything having to do with data, including reading sources, transforming data, and loading TARGETS, OCCURS inside a data flow.

•  Stage data in an operational datastore, data warehouse, or data mart.
•  Update STAGED data in batch or real-time modes.
•  Create a single environment for developing, testing, and DEPLOYING the entire data integration PLATFORM.
•  Manage a single metadata repository to capture the relationships between different extraction and access methods and PROVIDE integrated lineage and impact analysis.

DATA Services includes the following standard components:

• Designer
•  Repository
•  JOB Server
• Engines
•  Access Server
•  Adapters
•  Real-time Services
•  Address Server
•  CLEANSING PACKAGES, Dictionaries, and Directories
•  Management CONSOLE

Data Services includes the following standard components:

Business Objects Data Services PROVIDES a graphical interface that allows you to easily create jobs that EXTRACT data from heterogeneous sources, transform that data to meet the business requirements of your organization, and LOAD the data into a single LOCATION.

Business Objects Data Services provides a graphical interface that allows you to easily create jobs that extract data from heterogeneous sources, transform that data to meet the business requirements of your organization, and load the data into a single location.