This article explains different values and data types for fields and events in Iterable and examples of what those types of data would look like as they are passed into Iterable.
NOTES
- All field names are case-sensitive and space-sensitive.
- The character limit for field values is approximately 32k characters, regardless of data type.
- Nested fields count towards the project’s limit for user profile fields (1000).
# In this article
# Overview
A field's data type determines the kind of data that the field can store, and how that data can be used in Iterable. Iterable supports the following data types: string, date, long, double, Boolean, geo_location, object, and nested. Iterable also supports arrays for all data types.
Data types are important because they help you manage your data schema and ensure that your data is accurate and consistent. When you create a field with a specific data type, Iterable expects the field to contain data that matches that type. When Iterable receives data that doesn't match the field's data type, it rejects the data.
WARNING
Once set, a field's data type (string, Boolean, double, etc.) cannot be changed.
# Best practices for data types
Here are some best practices for managing data types in Iterable:
-
Plan your data schema
Before creating fields, plan your data schema. Determine what data types you need to store, how you will use that data, and how you will segment users based on that data. For example, if you need to set a data type that can be segmented by a numeric range, make sure to set the field as a number (long or double).
-
Set the correct data type with the first touch
When creating a field, make sure to set the correct data type by sending an accepted format the first time you make an API request or otherwise upload data.
If you need to change the data type of a field, you must create a new field name with the correct data type and migrate the data from the old field to the new field.
-
Use Data Schema Management
Use Data Schema Management to create new fields. This allows you to explicitly define the data type for a field before sending any data to Iterable, preventing an irreversible data type mistake.
-
Send clean data
When sending data to Iterable, make sure the data is clean and all values are formatted correctly. Data uploads and API calls that attempt to set or update an existing field with a value that cannot be coerced into the assigned data type will fail.
# Assigning a field's data type
To create new fields and set data types, you can use the Data Schema Management screen or Iterable's API.
-
Data Schema Management
When using the Data Schema Management screen, you can make these data types: Boolean, date, double, geo_location, long, object, and string.
-
API
Iterable's API automatically sets a new field's data type to the type of the first value you provide for it. To set a specific data type, you must provide a value that matches an accepted format for the data type you want to set. (If the value you provide doesn't match an accepted format, Iterable usually sets the field's data type to string.)
Uploading data via CSV files automatically sets all new fields to the string data type.
# Creating array and nested data types
Notably, array and nested data types aren’t available in the data type drop-down during field creation in the Data Schema Management screen.
-
Array
To create a field that contains an array, select the data type associated with the values you'll store in the array. Iterable's data is flexible, so you can store arrays of any data type. However, if you need to index arrays of objects and maintain the relationships of each object in the array, use the nested data type instead.
-
Nested
To create a nested data type, which contains an indexed array of objects, contact your customer success manager or Iterable support.
NOTE
Object data type vs nested data type vs nested fields
There are key differences between the object data type, nested data type, and nested fields:
- An object data type can contain a collection of related fields (which may include objects or an array of objects).
- A nested data type can contain multiple objects stored in an indexed array.
- Nested fields are fields within either the object data type or the nested data type.
# Choosing between object and nested data types
Iterable stores JSON objects as one of two data types: object or nested. To choose which data type to use for a given field in your Iterable project, consider how you need to access your data to segment users and personalize content.
For example, consider this array, which describes a user's cats:
{ "email": "user@example.com", "dataFields": { "cats": [ { "name": "Snowball", "color": "white" }, { "name": "Whiskers", "color": "orange" } ] } }
For this data, Iterable can store cats
as an array of objects, or as an array
of nested:
-
For objects, Iterable stores individual fields independently of one another, and they cannot be correlated.
For example, if you tell Iterable to store
cats
as an array of objects, then you can create a segmentation query that selects users who have awhite
cat and a cat namedSnowball
— but you cannot select users who have awhite
cat namedSnowball
. For nested, this limitation goes away, and you can segment for users who have a
white
cat namedSnowball
. However, queries on fields that have a nested type are slower than queries on objects, so you should only select the nested data type when you truly need it.
For a more technical explanation of nested fields, see Nested Field Type (Elasticsearch).
# Data types and segmentation
Iterable's segmentation tool treats fields based on their data type. It's important to consider the values you would like each field to store, and how you plan to segment users and access that data, before you create fields in Iterable.
For example, if a field is set as a string, the segmentation tool will only display string operators for that field. A common mistake is intending to create a data type that can be segmented by a numeric range, or as a date, but instead creating a string data type. This happens when the first value sent to Iterable for a field is a string format. Iterable requires specific formats for the data type to be set correctly.
The data type of an existing field is listed underneath each field in Audience > Segmentation, and can be found in Settings > Data Schema Management.
For instance, you can create age
as a field and set it as a string the
first time the field is sent to Iterable by sending an API request with the
age
field set to 26 years
. Iterable will set the age
field as a string,
and it cannot later be changed to a numeric data type.
When you segment users based on the age
field, Iterable will treat the field
as a string and not as a number. This means that you can't segment users by
age ranges or by users who are older or younger than a certain age. Instead,
you can only segment users by the exact age string value (26 years
) that was sent to Iterable.
In this case, the segmentation tool would treat the field as a string:
If a user wanted to use segmentation options given to a numeric field, like the image below (such as Less Than or Greater Than), it wouldn't be possible since the field has already been set as a string.
User field data types can also be found in Settings > Data Schema Management.
# Data types in Iterable
Below is a description of each data type and an example of how a field of that type would look in a sample payload as it is passed into Iterable using an API endpoint.
# String
Strings can contain both characters and numbers. Iterable treats all data as strings by default. Iterable's API automatically sets a new field's data type to string if the first value you provide for it is a string. If the value you provide doesn't match an accepted format for another data type, Iterable sets the field's data type to string.
When setting a string value with Iterable's API, surround its value in quotes.
The following request body for the POST /api/users/update
endpoint sets the user's first name to Justin
:
{ "email": "user@example.com", "dataFields": { "firstName": "Justin" } }
# Special characters in strings
Iterable receives and stores data in JSON format with UTF-8-encoded Unicode. Some special characters have specific meanings in JSON. These reserved characters need to be escaped when sending data to Iterable. By escaping them, you can include them in the value of a string without disrupting the JSON structure.
The following characters are reserved for JSON use and must be properly escaped to be used inside the value of a string.
-
Double quote - replace with
\"
-
Newline - replace with
\n
-
Backslash - replace with
\\
(To see a complete list of JSON escape characters and use a conversion helper, see this resource: JSON Escape)
# Date
A date field is a specific type of string in Iterable. Fields that are dates have many uses for segmentation in Iterable, like relative dates and date ranges. Iterable has a specific date format that must be used to segment a field by date.
Iterable accepts the following date formats:
-
yyyy-MM-dd (
2000-01-01
) -
yyyy-MM-dd HH:mm:ss (
2000-01-01 00:00:00
) -
yyyy-MM-dd HH:mm:ss ZZ (
2000-01-01 00:00:00 -04:00
) -
yyyy-MM-ddTHH:mm:ss.SSSZZ (
2000-01-01T00:00:00.000-04:00
)
When a creating a date field using Iterable's API, the date field value must be in an accepted format. If the date is not in the correct format, the API call creates a string data type instead of a date.
The following request body for the POST /api/users/update
endpoint sets the date for the anniversary
field:
{ "email": "user@example.com", "dataFields": { "anniversary": "2015-02-03 01:20:55 +00:00" } }
# Long
Longs are numbers that do not have decimal points.
The following request body for the POST /api/users/update
endpoint sets the user profile's age
field:
{ "email": "user@example.com", "dataFields": { "age": 26 } }
# Boolean
True or false. Acceptable Boolean values include true
and false
.
The following request body for the POST /api/users/update
endpoint sets the user profile's hasCat
field:
{ "email": "user@example.com", "dataFields": { "hasCat": false } }
# Double
Doubles are numbers that have decimal points.
The following request body for the POST /api/commerce/updateCart
endpoint sets an item's price
field to a double:
{ "user": { "email": "user@example.com" }, "items": [ { "id": "123", "sku": "123", "name": "XL Pizza", "description": "Piping hot and delicious", "categories": [ "food", "pizza" ], "price": 24.99, "quantity": 1 } ] }
# Geo_location
A geo_location field stores a location's latitude and longitude.
The name of any field that stores a geo_location value must have a
_geo_location
suffix. Example: address_geo_location
.
The geo_location object may only contain two fields: lat
(the latitude)
and lon
(the longitude). Do not put other fields in this object.
For example, the following request body for the POST /api/users/update
endpoint creates a work_geo_location
field on the user's profile:
{ "email": "user@example.com", "dataFields": { "work_geo_location": { "lat": 41.505493, "lon": -81.681290 } } }
NOTES
- Iterable's segmentation tool cannot perform geo_location queries (proximity, etc.). However, catalog items can contain geo_location values, and collections can use geo_location queries to compare them to the values stored on a user's profile.
- When defining a geo_location field on a catalog item, there is no need to
use the
_geo_location
suffix. The suffix only applies to geo_location values stored on user profiles and custom events.
# Object
Objects are collections of related fields. For example, to represent the various attributes of a cat, use an object.
The following request body for the POST /api/users/update
endpoint creates a cat
object on a user's profile:
{ "email": "user@example.com", "dataFields": { "cat": { "name": "Snowball", "color": "White", "age": 4, "favoriteToy": "Fluffy stick" } } }
Iterable's segmentation tool displays objects with a period between the name
of the object and the name of the field being referenced. For example, the
tool would represent the field for the above cat's name as cat.name
.
NOTE
Objects can be edited with API calls (not CSV uploads).
# Nested
NOTES
- Nested fields count toward the project’s limit for user profile fields (1000).
- To create a new nested field, contact your customer success manager or Iterable support.
- Nested fields can be edited with API calls (not CSV uploads, and not in the Settings > Data Schema Management screen).
A nested data type contains an array of objects and allows each field to be indexed in the context of their object group, rather than as independent field values.
To understand the differences between the object data type, nested data type, and nested fields, read Choosing between object and nested data types.
Here is an example of a nested data type that contains multiple items, called
cats
:
{ "email": "user@example.com", "dataFields": { "cats": [ { "name": "Snowball", "color": "White", "age": 4, "favoriteToy": "Fluffy stick" }, { "name": "Whiskers", "color": "Orange", "age": 2, "favoriteToy": "Laser pointer" } ] } }
# Array
Arrays are lists of values that can contain any data type, including objects. Arrays are not a separate data type but a data structure that can store multiple values of any particular data type.
Iterable automatically treats any field with an array value as an array, while the values within the array are treated as the data type of the field. For example, you can store an array of strings within a string data type.
In the Data Schema Management screen, there is no specific option for arrays in the data type drop-down, and fields that contain arrays are treated as the data type of the values they contain (string, date, long, double, Boolean, geo_location, or object).
The following request body for the POST /api/users/update
endpoint creates a favoriteFoods
array on a user's profile, and the
favoriteFoods
field is classified with a string data type:
{ "email": "user@example.com", "dataFields": { "favoriteFoods": [ "Pizza", "Bánh mì", "Ramen" ] } }
NOTE
Arrays can be updated with API calls, but not CSV uploads.
# Want to learn more?
For more information about some of the topics in this article, check out this Iterable Academy course. Iterable Academy is open to everyone — you don't need to be an Iterable customer!