Data migration flow

The data migration flow diagram illustrates the end-to-end flow of data during data migration and integration, highlighting key components and how data is processed from sources to targets.

Source

Data from a source is collected and processed using queries for database-related sources and external API calls for other types of data.

The following are the supported data sources:

DB table — Structured data stored in a database table.
DB query — Data retrieved from SQL queries.
Text file — Data stored in flat files, such as CSV files.
Service file — Data in various service-oriented formats, such as XML.

DIH core processes

DIH handles the following tasks to migrate and integrate data:

Schedule — Manages the timing and execution of data migration jobs.
Load, parse, and validate — Ensures data is correctly loaded, parsed for usability and consistency, and validated for errors.
Transform — Transforms data to meet target requirements.
Assign keys — Generates or maps keys to ensure correct data relationships.
Validate objects — Checks the correctness and compatibility of data objects before migration.
Track errors — Monitors and logs any errors that occur during the migration process.
Produce reports — Generates data migration process reports.

Target

The target specifies the destination for the processed data, where it will be stored or integrated.

The following are the supported targets:

Database — Defines the target database where the data delivered using DB APIs is stored.
API service — Defines the target services accessed via API calls to integrate data into a service-oriented architecture.
File — Defines the target outbound file, such as Fixed, CSV, XLSX, and EBCDIC.

Data migration flow