You can make changes to a Smart Ingest sync after you create it. This is useful if you need to adjust the sync configuration to better fit your needs. A common scenario for this would be for performance fine-tuning.
Continue reading to learn about the changes you can make to an existing sync and how to make them.
NOTE
This article covers changes you can make to an existing sync. If you need to make changes the sync engine, read Optimizing Smart Ingest Sync Performance.
In this article
Changes you can make
Here are all of the elements that can be changed for an existing sync:
- Status:
- Enable or disable a sync
- Configuration:
- Sync type
- Sync mode (varies based on sync type)
- Sync settings (vary based on sync type)
- Performance settings (vary based on sync type)
- Field mapping changes
- Schedule:
- Sync schedule
- Model:
- Model name
- Edit SQL query (if using SQL editor)
- Change selected data table (if using table selector)
- Change primary key
A sync’s modeling method can’t be changed after you create the sync—meaning you can’t switch an existing sync from the table selector to a SQL query after it is created.
Changes that trigger re-syncs
In some scenarios, making changes to the data model can trigger Smart Ingest to re-sync previous rows.
Changes that can trigger a re-sync include field mapping changes, such as adding new field mappings and changing existing field mappings.
When this happens, it is normal to experience a sync speed similar to a first run. That’s because each row is re-synced to include the new column value in Smart Ingest's change data capture. To learn more, read Optimizing Smart Ingest Sync Performance.
Changing field mapping
If you edit the field mappings in your sync configuration, the sync engine automatically reprocesses the entire model query result set during the next sync run. Depending on the selected sync mode, Smart Ingest syncs the data model rows that change data capture detects as added or changed.
If the sync uses either update or upsert mode (users, custom events, purchase events, catalog, update email address), then Smart Ingest re-syncs all previously synced rows with the updated mappings as "changed" rows.
If the sync uses insert mode (such as for shopping cart items), Smart Ingest doesn’t reprocess previously synced rows. Instead, it only syncs model rows that are detected as added after you save the field mapping changes in your sync configuration.
Changing data type
If you change a model column's data type, the sync engine considers this update as a field mapping update and reprocesses all model rows during the next sync run.
Changing primary key
If you update a data model's primary key by selecting a different column, you need to trigger a full re-sync for all syncs that use that data model. Otherwise, Smart Ingest can't process your model data correctly, which can make your syncs fail.
You don't need to manually trigger a full re-sync if you change the primary key's data type. If you change the primary key's data type in the data model configuration, the sync processes normally. If you make this change in your source or in the SQL editor, the entire data model is automatically re-synced as if you triggered a full re-sync. Additionally, this can create duplicates in your destination data.
Model column changes
Sync configurations don’t automatically detect most changes to your data warehouse columns. Smart Ingest handles the changes differently depending on whether columns are added, renamed, or removed from your data model definition, or if you change a column's data type.
Adding a new column
If you add a new column to your data source, Smart Ingest doesn't automatically add it as a field mapping to your sync configurations. This is because Smart Ingest doesn't know which columns you would like to sync to Iterable, and which Iterable fields they should be mapped to. Follow the field mapping instructions to include new columns in your sync configuration.
To have new columns appear in Smart Ingest field mapping, go to the Model tab and click Preview Results to view and locate the new column. Then, proceed to Configuration and update field mapping to add the new column and its destination.
Renaming an existing column
If you rename a column in your data source, all syncs that had that column mapped in their configuration fail with an unknown field name error. The error message contains the original name of the renamed column.
To resolve this, remove or update the relevant field mappings in your sync configuration to use the new column name.
Removing a column
If you remove a column from your data source, Smart Ingest doesn't automatically remove it from the field mappings in the sync configuration. Make sure to remove the mapping manually by editing the sync.
Editing a Smart Ingest sync
To make changes to an existing sync:
Log in to Iterable as a user with the Manage Integrations project permission and open the project you’re working on.
Go to Integrations > Smart Ingest, and then find the sync you’re looking for.
Click on the overflow menu (three dots) and click on View History.
-
In the pop-up screen, you can go to the Configuration, Schedule, or Model tabs to make changes.
You can also click Edit as JSON to review and make changes to the sync configuration in a text format.
Changes to the sync configuration apply to the next sync run. If you need to make immediate changes, you can manually trigger a sync run by clicking Run on the sync’s View History page.
Testing a Smart Ingest sync
To test your data model and sync configuration changes, you can run a test sync to see how the changes affect your data in Iterable. This is especially useful if you're making significant changes to your sync configuration or data model.
You can test a sync by selecting a row of data from your data model and syncing it as an added, changed, or removed row. This simulates how the sync processes the data and allows you to review the results to ensure the sync processes as expected.
To run a test:
Log in to Iterable as a user with the Manage Integrations project permission and open the project you’re working on.
Go to Integrations > Smart Ingest, and then find the sync you’re looking for.
Click on the overflow menu (three dots) and click on View History.
In the pop-up screen, go to the Configuration tab.
Make your changes to the field mapping and sync settings, and then click Test.
-
In the Test a Row screen, select a row of data to test, and then click Sync as Added Row. This simulates how the sync processes the data.
You can also choose to Sync as Changed Row and Sync as Removed Row to test how the sync processes in those manners.
Review the results to ensure the sync processes as expected, then you can Close the test
Make adjustments as-needed, re-test to confirm you're seeing the results you want.
Save your changes.