Smart Ingest Field Mapping Overview

As part of importing data into Iterable, understanding field mapping is crucial. It ensures that your data fits correctly into Iterable's structure, where you can use it effectively for your marketing campaigns.

To learn more about the importance of field mapping, its basic principles, and how to effectively use it in Smart Ingest, read on.

# In this article

# Why field mapping is important

When you import data into Iterable using Smart Ingest, you need to set up field mappings. These mappings are like bridges that connect the data fields from your source to the matching fields in Iterable.

This step is critical to ensuring that your imported data fits correctly into Iterable's structure. It's like telling Iterable to carefully take data from a certain field in your source and put it into a specific field in Iterable.

By setting up these field mappings, you make sure that the imported data is put in the right place in Iterable, in the data types that Iterable accepts. This careful process makes sure your data says consistent and usable after you import it.

# Field mapping basics

At its simplest, field mapping involves directly importing a value from a column in your data model into a field in Iterable. For instance, the first_name column in your data model could map to the firstName user profile field in Iterable. Values stored in the first_name column would then be imported into the firstName field in Iterable.

Some mapping types offer more advanced features. The template mapping type, for example, lets you use the Liquid template language to modify data before importing it into Iterable. The object mapping type allows you to import data from multiple fields in your data model into a single object field in Iterable.

Some mapping types allow you to sync static values or metadata about the sync job to Iterable. This can be useful for adding context to your data, tracking the progress of your sync, or testing and troubleshooting your sync configuration.

Other mapping types, like object and object array, can create JSON data structures to import into Iterable. These mapping types are useful when you need to import data that doesn't fit neatly into a single field.

When creating field mappings, here are a few things to keep in mind.

# Matching data types

Make sure that the data type of each source field matches the data type of the destination field in Iterable. If the data types don't match, you may encounter errors when importing your data.

For example, you can't map a string type field in your data model to a long type field in Iterable.

To learn more about the different data types supported by Iterable, read Field Data Types.

You may have different data types in your source data model than in Iterable. For example, you may have a timestamp type field in your data model that you want to map to a date type field in Iterable. In this case, you can use the template mapping type to convert the data from a timestamp to a date format accepted by Iterable.

# Field mapping limitations

You can use a source field (a column in your data model) multiple times, but each destination field in Iterable can only be mapped once. This means that you can't map multiple source fields to a single destination field in Iterable. (Instead, you may want to create a JSON object or array).

# Making changes

Remember that you can always edit your field mappings after you've created them. To learn more about making changes to a sync, read Editing and Testing a Smart Ingest Sync.

# Field mapping types

Smart Ingest provides several field mapping types to help you import your data into Iterable. Here's a brief overview of each type:

• Column value - Syncs a field value directly from your data model to Iterable.
• Static value - Syncs a constant value to a field in Iterable.
• Variable - Syncs metadata about the sync job to a field in Iterable.
• Template - Syncs a field value from your data model to Iterable, using Liquid template language. Allows for advanced data operations.
• Array - Syncs a field value that contains simple arrays from your data model to an array in Iterable.
• Object - Syncs a defined set of fields from your data model to an object in Iterable.
• Object array - Syncs an array of a defined set of fields from your data model to a nested array of objects in Iterable.

To learn more about each field mapping type and how to use them, read Smart Ingest Field Mapping Types.

# Syncing null values

For each field mapping, you have an option to sync null (blank) data values, or to skip them. To understand this setting, consider the field that you are syncing to and what it’s used for. For each destination field in Iterable, consider:

• Whether its data value is static or dynamic over time.
• Whether it should always contain a value, or whether a blank value makes sense at times.
• Whether the data in your source is the actual source of truth when that data value is cleared.

By default, Smart Ingest always syncs null values.

To access this setting, click on the settings icon next to the field mapping.

# Always sync

When a field mapping always syncs null values, and a record has a blank value for a field, then the blank value coming from your data model overwrites any existing value in Iterable.

This is the default setting for all field mappings.

Use this setting for data fields that may become empty or need to be cleared for some reason. However, exercise caution, as this could also lead to data loss as field values in Iterable are erased when this happens.

# Don’t sync null values

When a field mapping doesn’t sync null values, and a record has a blank value for a field, then the existing value in Iterable remains on the given record. This prevents data from being erased in Iterable unintentionally.

Field mappings that Don't Sync Null Values reflect a notification dot by the settings icon for that field mapping.

# Mapping to Iterable fields

When you map a field from your data model to a field in Iterable, you're telling Iterable to take the data from your source and put it into a specific field in Iterable.

To map a field from your data model to a field in Iterable:

1. Click on the field selection box in the source or destination field column.

2. Use the field mapping tool to select the mapping type you want to use.

   The mapping tool changes dynamically. For example, if you select a column value mapping, you can choose the column from your data model that you want to map to the field in Iterable.

   For specific instructions for each mapping type, read Smart Ingest Field Mapping Types.

3. Choose the destination field in Iterable that you want to send the data to. The dropdown list does not include fields that have been hidden in Iterable.

4. Update the null value sync setting if needed. This defaults to always sync null values, which clears data in Iterable.

5. Complete the mappings for all fields that you want to sync.

6. Consider testing your sync configuration to ensure that the data is imported correctly.

# Adding new fields to Iterable

In the field mapping step, you can create new Iterable fields to accept data from your source. Here are a few things to keep in mind:

• Hidden fields do not show in Smart Ingest. If a field name is missing from the dropown list, check if it's hidden in Iterable by going to Settings > Data Schema Management.

• Projects are limited to 1000 user fields total, and 8000 event fields (cumulative across all custom event types).

• Only add fields that you're sure you need, and don't create duplicates (like firstName and first_name).

• When creating new fields, Iterable automatically generates a schema for that field based on the first data value sent. You can’t change the schema once it's set. To learn more, read Managing User Profile Fields in Iterable.

# Creating new fields in Smart Ingest while field lock in enabled

When you have a field lock setting enabled in Iterable, Smart Ingest can't create new fields and/or events in the project. This is because the field lock setting prevents new fields and/or events from being created in the project.

To create a new field through Smart Ingest, you can disable the field lock setting in Iterable temporarily.

Alternatively, if you must keep field lock on but need to create a new field, contact your customer success manager and request for the Smart Ingest sync to override field lock.

To learn more about field lock settings, read Protecting a Project's Data Schema.

# Instructions

To add a new field in Iterable during field mapping:

1. Enter the name of a field that you want to create in the destination field search box.

2. When few or no existing fields match, Smart Ingest gives you an option to create a new field.

   

3. Click Create Field to add the field to Iterable.

4. Save your sync configuration.

5. The new field is created in Iterable when the sync runs for the first time.

This adds a new field to Iterable in the same manner as creating new fields using the API. Note that the field won’t actually be created in the Iterable project’s data schema until the sync runs the first time. Iterable assumes its data type based on the first row of data. If it contains a boolean value (true or false), then Iterable assigns the boolean data type, and so on.

Field data types in Iterable cannot be changed after the field is created. If you need to change the data type, you must create a new field name.

# Suggested mappings

Under the field mapping area, there’s an option to Suggest mappings.

When you click this button, Smart Ingest makes recommendations of relevant destination fields for model columns. For example, a model column named first_name would receive a suggested destination field like firstName. This autocompletion works by searching for matching and near-matching column and field names. Before saving your configuration, you should confirm that the autocompleted suggestions are correct.

When you use suggested mappings, watch out for:

• Unwanted mappings

  Smart Ingest may suggest mappings that you don’t want.

  Only add fields to the mapping area if you intend on changing them in Iterable. If you don’t want to change them, remove them from the field mapping area.

• Duplicate field names

  Smart Ingest suggestions may include duplicate fields in autocomplete suggestions, resulting in the same source field being mapped to multiple destination fields.

• Unique Identifiers: email and userId.

  Smart Ingest already uses one of these values to match user records. It might auto-assign the unused field during suggested mappings.

  For example, when you use email to identify records in Iterable for an email-based project, Smart Ingest might suggest mapping your Iterable userId field to a semantically similar source field the field mapping area, which would overwrite the userId field in Iterable.

  NOTE

  For email-based projects: If you intend on changing email addresses in Iterable using a Smart Ingest sync, use the User email address sync type, not Users.

# Testing your sync configuration

After you've set up your field mappings, it's a good idea to test your sync.

This is something you can do only after you've saved your sync configuration, and not during the initial setup flow.

If you want to test the configuration before running the first sync, select the Manual schedule type and change it later.

To learn more about testing your sync configuration, read Editing and Testing a Smart Ingest Sync.

# Want to learn more?

For more information about some of the topics in this article, check out this Iterable Academy course. Iterable Academy is open to everyone — you don't need to be an Iterable customer!

• Smart Ingest: Field Mapping