The Dedupe tool helps you remove duplicate rows from your datasets based on specific fields or all fields, ensuring that your dataset contains only unique entries.
Configuration Options
Distinct Rows:
This option removes any rows that are exact duplicates based on all fields in the dataset.
Use this when you want to ensure every row is entirely unique.
Dedupe on Fields:
Select this option to remove duplicates based on specific fields.
You can choose one or multiple fields to define the criteria for deduplication. For example, selecting Customer_ID will remove rows with duplicate values in this field, keeping only one instance.
Prioritize Rows By: You can prioritize which row to keep when duplicates are found by selecting another field (e.g., Order_ID) to keep the row with the highest or lowest value.