How to Remove Duplicates in Datastage?

1.	How to Remove Duplicates in Datastage?
Answer» In Datastage, duplicates can be removed using the four ways: Through the Duplicate Removal Stage Using the Hash File Stage. Using a sort stage and setting ALLOW DUPLICATES: false It can be done at any stage by doing a hash portion of the INPUT data and CHECKING OPTIONS under Sort and UNIQUE.

Answer»

In Datastage, duplicates can be removed using the four ways:

Through the Duplicate Removal Stage
Using the Hash File Stage.
Using a sort stage and setting ALLOW DUPLICATES: false
It can be done at any stage by doing a hash portion of the INPUT data and CHECKING OPTIONS under Sort and UNIQUE.

Discussion