Use Smart Ingest to sync data from any supported source to Iterable user profiles. With this sync type, you can sync existing users, update user profiles, and add new users to Iterable.
When you are planning to sync user profiles, there are a few decisions to make for sync setup. For example, you need to decide how you want to match users in Iterable. You must also have a data model ready that meets the requirements for this sync.
Continue reading for more information on the settings available for this sync.
In this article
Data model requirements
To run a user sync, your data model needs to contain a user identifier (either user ID or email address, depending on the project type you're using).
The data model should also contain additional fields that you want to update in the Iterable user profile—with the following exceptions:
- To update shopping cart items, use the Events - Shopping Cart Items sync type.
- To update a user's email address in email-based projects, use the User email address sync type.
Sync mode
Decide whether to update existing users only, or to add new users using upsert (which adds new records in addition to updating existing ones).
What the sync mode does not configure:
Deleted records: Record deletion is controlled with the delete behavior setting, not the sync mode.
Duplicate records: The sync mode you select also doesn’t impact how duplicate records are handled (learn more).
Upsert mode
- Updates existing Iterable users.
- Adds a new user to Iterable for each record that has no matching unique identifier in Iterable.
- Provides standard sync performance.
Update mode
- Only updates existing Iterable users with data from your source.
- Skips records that have no matching primary key (
emailoruserId). - Has noticeably slower sync performance.
IMPORTANT
Selecting update mode requires additional processing time for each sync run. To learn why, read Optimizing Smart Ingest Sync Performance.
If your project is email-based and you have unique userId values in your
Iterable project (which Iterable does not enforce), you can also use different
record matching to control the API used and increase performance speeds. To
learn more, read Email-based project matching users on Iterable User IDs.
Subscription preferences
Decide whether to update users’ subscriptions by using Iterable’s POST /api/users/bulkUpdateSubscriptions
API endpoint.
Selecting Yes adds the following Iterable fields to the top of the Iterable destination fields drop-down:
-
emailListIdsoruserListIds unsubscribedChannelIdsunsubscribedMessageIdssubscribedMessageTypeIdscampaignIdtemplateId
After choosing this option, you still need to map each field to send data to Iterable using field mapping. Remember that some of these system use fields are arrays, and subscription data coming from your data source should be mapped to the correct format for arrays.
If you select No, then you may still see these fields in the destination
field drop-down, however Smart Ingest uses POST /api/users/bulkUpdate
rather than POST /api/users/bulkUpdateSubscriptions.
User matching
With user matching, select the Iterable field that uniquely identifies users and the source field that contains that same value. This is so Smart Ingest can look for existing user records in order to update an existing user profile.
You can use Email or User ID to match rows from your data model to users in Iterable. In general, it's best to select the one that matches the project type you’re syncing with. Regardless of the fields you select in this step, the field names don’t need to match, but they must have matching field values and uniquely identify records so that updates on existing users are successful.
| Project type | User matching field | Description |
|---|---|---|
| Recommended for most syncs in email-based projects. Read more. | ||
| User ID | When in update mode, use for enhanced performance—but only if userId is unique. Iterable doesn’t enforce uniqueness on the Iterable user ID field. Read more. |
|
| UserID | Don’t use. The email field isn’t available for user lookup in userID-based projects. |
|
| User ID | Required as the matching field. Read more. | |
| Hybrid | Email or User ID | Use either field for hybrid projects. Read more. |
Here are some examples of record matching and how it works for each project type and unique identifier.
Email-based projects matching on email
Record matching: EMAIL_ADDRESS (example data source field) and Email
(Iterable field: email).
Smart Ingest looks at the EMAIL_ADDRESS for each row and matches it to the
user that has a matching email in Iterable, then updates that record.
No matches: When a record’s EMAIL_ADDRESS doesn’t match an Iterable
email, then it’s treated as a new record.
- In update mode, these are skipped.
- In upsert mode, they are added to Iterable as a new user.
Duplicates in source data: When more than one record has the same
EMAIL_ADDRESS, then all records are skipped.
Duplicates in Iterable: For this project type, Iterable enforces uniqueness
on email, so duplicates by Email in Iterable can’t exist.
UserID-based projects matching on user ID
Record matching: CUSTOMER_ID (example data source field) and User ID
(Iterable field: userId).
In this example, Smart Ingest looks at the CUSTOMER_ID for each row and
matches it to the user that has a matching userId in Iterable, then updates
that record.
No matches: When a record’s CUSTOMER_ID doesn’t match an Iterable
userId, then it’s treated as a new record.
Duplicates in source data: When more than one record has the same
CUSTOMER_ID, then all records are skipped.
Duplicates in Iterable: For this project type, Iterable enforces uniqueness
on userId, so duplicates by userID in Iterable are not a concern.
Hybrid project matching on email or user ID
Iterable enforces uniqueness on both the email and userId fields for hybrid
projects. You can use either field for record matching. We use email for this
example, but the behavior is the same for userId.
Record matching: EMAIL_ADDRESS (example data source field) and Email
(Iterable field: email).
Smart Ingest looks at the EMAIL_ADDRESS for each row and matches it to the
user that has a matching email in Iterable, then updates that record.
No matches: When a record’s EMAIL_ADDRESS doesn’t match an Iterable
email, then it’s treated as a new record.
- In update mode, these are skipped.
- In upsert mode, they are added to Iterable as a new user.
Duplicates in source data: When more than one record has the same email,
then all records are skipped.
Duplicates in Iterable: For this project type, Iterable enforces uniqueness
on email, so duplicates by Email in Iterable can’t exist.
Email-based project matching users on user ID (recommended for update mode only)
Why this is recommended
In this scenario, selecting User ID for record matching can increase sync speed when you want to fine-tune your sync performance.
There's one exception when you shouldn't match your record matching column to
your project type’s unique identifier: If you have an email-based project, your
sync is using update mode, and your data in Iterable doesn’t have duplicate
userId values.
IMPORTANT
In email-based projects, Iterable doesn’t enforce unique user IDs, so you
should do this only if you're sure that your userId field values in Iterable are unique.
How it works
Matching records on the userId on email-based projects enables a different
data flow that can reduce the time it takes to locate existing records during
sync runs in update mode. When you select Iterable User ID for record matching
in email-based projects, Smart Ingest uses the POST /api/users/bulkUpdate
API endpoint, with the preferUserId parameter. This parameter dictates whether
new users should be created if the request includes a userId that doesn't yet
exist in the Iterable project. This allows Smart Ingest to update existing
users, and not create new ones, without first having to perform a lookup.
Record matching: CUSTOMER_ID (example data source field) and Iterable
User ID (Iterable field: userId).
Smart Ingest looks at the CUSTOMER_ID for each row and matches it to the user
that has a matching Iterable User ID (userId) in Iterable, then updates that
record.
No matches: When a record’s CUSTOMER_ID doesn’t match an Iterable
userId, then it’s treated as a new record.
Duplicates in source data: When more than one record has the same
CUSTOMER_ID, then all records are skipped.
Duplicates in Iterable: For this project type, Iterable doesn’t enforce
uniqueness on userId. Duplicates by userId in Iterable may exist. When two
or more users in Iterable have the same userId, Iterable updates the first
record returned and the rest are skipped. Results are unpredictable, therefore
this setup isn’t recommended unless you enforce uniqueness on the userId field
through other means.
Field mapping
You can sync columns from your data model to Iterable user profile fields. To learn about the different ways to map fields to Iterable, read Field mapping.
NOTE
Need to change email addresses?
For email-based projects, don’t use the Users sync type to change an existing user’s email address to a new email address. Instead, use the User email address sync type.
Delete behavior
Choose a policy for deleting user records in Iterable when they’re removed from your data model.
| Behavior | Description |
|---|---|
| Do nothing | Keep the users in Iterable unchanged |
| Clear | Keep the user in Iterable and clear data fields managed by Smart Ingest |
| Delete | Delete the user from Iterable. Note that this deletes user records when they are removed from the data model. If you’re interested in a sync that deletes user records based on their presence in a data model, instead use the User actions - Delete sync type |
Advanced configuration
Updating objects
When you sync one or more objects that are nested on a user’s profile, you need to determine how to handle the data.
For example, if you have an object field in Iterable that has a value of { "a":1, "b": 2 }, and your data model has a value of { "a": 3, "c": 4 }:
When you choose to Overwrite the entire object, the resulting value in Iterable is
{ "a": 3, "c": 4 }. The old value ofbis removed.When you choose to Merge the old and new objects, the resulting value in Iterable is
{ "a": 3, "b": 2, "c": 4 }. The old value ofais overwritten, and the old value ofbis preserved.
NOTE
This setting is controlled with Iterable's mergeNestedObjects API parameter,
which is false by default. To learn more about how this parameter works, read
our API documentation for POST /api/users/update.
Rate limits and batching
Smart Ingest provides the ability to adjust sync rate limits and batch size. We recommend starting with the default values and making changes to fine-tune your sync performance later.
Associated API: POST /api/users/bulkUpdate
Batch size
User syncs use bulk APIs whenever available, so records are generally processed in batches. By default, a sync runs with 1000 rows (users) per batch. You can adjust this down, or up to 10,000 per batch, as-needed. However, increasing the batch size can lead to performance issues if the batch size is too large. Iterable's batch size limit is generally 4MB.
You may decide to change your batch size when fine-tuning sync performance or responding to errors such as “file size too large”.
Rate limit
This setting controls the rate of requests in a given period coming from Smart Ingest to Iterable, but doesn’t change the API rate limits for your Iterable account. Problems can occur if there are too many requests coming into Iterable at the same time. When this happens, requests are blocked from processing.
By default, this sync sends 5 requests (batches) per second.
Increase if Iterable offers you custom limits.
Decrease the rate if you run multiple Smart Ingest syncs, or have other integrations and are running into rate limit errors during sync runs.