Planning a Smart Ingest Sync

Smart Ingest is a powerful tool that imports data into Iterable from a data warehouse or other connection source. This article will help you plan your Smart Ingest implementation, including understanding your data source, deciding what data to import, and creating a checklist for your Smart Ingest implementation.

# In this article

# Getting started

To begin using Smart Ingest, start with a clear picture of what your goals are and the results you’re looking to achieve. Let’s say there is a particular use case that you’re building a new campaign for.

• What data fields do you need to import for this campaign to work?

• Where does the data go in Iterable? (User profile, event, catalog, etc)

• Do these fields already exist in Iterable?

• How often does this data refresh in your data source, and how fresh does the data need to be for the campaign to run with up-to-date information?

As you work with your stakeholders, it's a great idea to keep track of these discussions. Over time, you can build your understanding of your data source and rely less on your data source administrator to make changes with Smart Ingest.

# Understanding your data source

Work closely with your data source administrator to align on technical needs. You’ll need to know the data table name (or ask for a SQL query), primary key for the table, and all of the field names and their data types.

# Deciding what data to import

Your team should decide what sync type you’re going to use. This defines the destination for the imported data. You can sync with Iterable users, events, lists, catalog, and more.

Additionally, determine the best way to handle records as the sync processes. Important considerations:

• Sync mode: Should this sync add or delete records in Iterable, or just update existing records? Available sync modes vary by sync type.

• Field mapping: Each imported data source field needs a destination field in Iterable.

• New fields: Importing a new-to-Iterable field results in a new field being created. Projects have a 1000 lifetime limit for user profile fields.

• Field data types: Iterable rejects field values when the data type doesn’t match. Make sure you understand any data hygiene issues with the data you’re importing.

• Duplicate records: When Iterable detects a duplicate record in the sync, all records with the matching primary key are skipped. To learn more, read our FAQ: How are duplicate records handled during a sync?

By the time you finish this conversation, you should have a concrete definition of the data that you want to import to Iterable and how it should sync.

# Smart Ingest implementation checklist

Use this checklist to keep track of the actions you need to take to set up Smart Ingest in your project.

# Planning

1. Determine the permissions that your organization can grant to users for Smart Ingest. In order to import data, you need to have the correct permissions in the data source.

   The minimum permissions vary based on the setup: read-only for Basic syncs (default), read-write for Lightning syncs (recommended).

   To learn more about minimum permissions and how to select a sync engine, read Optimizing Smart Ingest Sync Performance.

2. Choose a sync type, and decide the data you’re importing to Iterable (users, events, etc).

3. Working with your data source administrator, determine your data model.

   Necessary details:

   • Data table name, view, or a prepared SQL query (for database connections)
   • File name and path (for file-based connections)
   • Primary key for the data table
   • Names of data source fields to import (these are columns in the data table)
   • Plan the field mapping for each field you’re importing to Iterable
   • Review the available settings for your chosen sync type, and determine what options you’ll need for sync mode, record matching, delete behavior, and more.

# Data source implementation

Data source administrators should complete these steps:

1. Provide your Iterable administrator with:

   • Data source connection details - For specific requirements, read the source connection guide for your particular connector.
   • Data source user credentials - Minimum permissions vary based on setup: read-only for Basic syncs (default), read-write for Lightning— learn more.

2. Allow the IP addresses for Smart Ingest to connect.

Some sources require additional setup to prepare for connection with Smart Ingest.

# Iterable implementation

Any Iterable user with the Manage Integrations permission can create and manage Smart Ingest syncs.

First, connect to your source.

Then, create a new sync. Smart Ingest's guided setup walks you through the following steps:

1. Define your data model.
2. Select a sync type.
3. Enter sync settings as-needed for your chosen sync type.
4. Map the fields that you're syncing to Iterable.
5. Schedule your sync.

# Next steps

Once you've created your first sync and it has run successfully, you can:

• Monitor sync runs.
• Make changes to your syncs as-needed.