This article explains different values and data types for fields and events in Iterable and examples of what those types of data would look like as they are passed into Iterable.
NOTES
- All field names are case-sensitive and space-sensitive.
- The character limit for field values is approximately 32k characters, regardless of data type.
- Nested fields count towards the project's limit for user profile fields (1000).
# In this article
# Overview
A field's data type determines the kind of data that the field can store, and how that data can be used in Iterable. Iterable supports the following data types:
string
date
long
double
Boolean
geo_location
object
-
nested
(user profile fields only)
Iterable also supports arrays for all data types.
Data types are important because they help you manage your data schema and ensure that your data is accurate and consistent. When you create a field with a specific data type, Iterable expects the field to contain data that matches that type. When Iterable receives data that doesn't match the field's data type, it rejects the data.
WARNING
Once set, a field's data type (string, Boolean, double, etc.) cannot be changed.
# Best practices for data types
Here are some best practices for managing data types in Iterable:
-
Plan your data schema
Before creating fields, plan your data schema. Determine what data types you need to store, how you will use that data, and how you will segment users based on that data. For example, if you need to set a data type that can be segmented by a numeric range, make sure to set the field as a number (long or double).
-
Set the correct data type with the first touch
When creating a field, make sure to set the correct data type by sending an accepted format the first time you make an API request or otherwise upload data.
If you need to change the data type of a field, you must create a new field name with the correct data type and migrate the data from the old field to the new field.
-
Use Data Schema Management
Use Data Schema Management to create new fields. This allows you to explicitly define the data type for a field before sending any data to Iterable, preventing an irreversible data type mistake.
-
Send clean data
When sending data to Iterable, make sure the data is clean and all values are formatted correctly. Data uploads and API calls that attempt to set or update an existing field with a value that cannot be coerced into the assigned data type will fail.
# Assigning a field's data type
To create new fields and set data types, you can use the Data Schema Management screen or Iterable's API.
-
Data Schema Management
When using the Data Schema Management screen, you can make these data types: Boolean, date, double, geo_location, long, object, and string.
-
API
Iterable's API automatically sets a new field's data type to the type of the first value you provide for it. To set a specific data type, you must provide a value that matches an accepted format for the data type you want to set. (If the value you provide doesn't match an accepted format, Iterable usually sets the field's data type to string.)
Uploading data via CSV files automatically sets all new fields to the string data type.
# Creating array and nested data types
Notably, array and nested data types aren't available in the data type drop-down during field creation in the Data Schema Management screen.
-
Array
Iterable supports arrays of values for all data types.
To create a field that contains an array, select the data type associated with the values you'll store in the array. Iterable's data is flexible, so you can store arrays of any data type. However, if you need to index an array of objects and maintain the relationships of each object in the array, use the nested data type instead.
-
Nested
Iterable supports the
nested
data type for user profile fields, but not custom events.To create a nested data type, which contains an indexed array of objects, contact your customer success manager or Iterable support.
NOTE
Object data type vs
nested
data type vs nested fieldsThere are key differences between the object data type,
nested
data type, and nested fields:- An object data type can contain a collection of related fields (which may include objects or an array of objects).
- A nested data type can contain multiple objects stored in an indexed array.
- Nested fields are fields within either the object data type or the nested data type.
# Choosing between object
and nested
data types
Iterable stores JSON objects as one of two data types: object or nested. To choose which data type to use for a given field in your Iterable project, consider how you need to access your data to segment users and personalize content.
For example, consider this array, which describes a user's cats:
{ "email": "user@example.com", "dataFields": { "cats": [ { "name": "Snowball", "color": "white" }, { "name": "Whiskers", "color": "orange" } ] } }
For this data, Iterable can store cats
as an array of objects, or as an array
of nested:
-
For objects, Iterable stores individual fields independently of one another, and they cannot be correlated.
For example, if you tell Iterable to store
cats
as an array of objects, then you can create a segmentation query that selects users who have awhite
cat and a cat namedSnowball
— but you cannot select users who have awhite
cat namedSnowball
. For nested, this limitation goes away, and you can segment for users who have a
white
cat namedSnowball
. However, queries on fields that have a nested type are slower than queries on objects, so you should only select the nested data type when you truly need it.
For a more technical explanation of nested fields, see Nested Field Type (Elasticsearch).
# Data types and segmentation
Iterable's segmentation tool treats fields based on their data type. It's important to consider the values you would like each field to store, and how you plan to segment users and access that data, before you create fields in Iterable.
For example, if a field is set as a string, the segmentation tool will only display string operators for that field. A common mistake is intending to create a data type that can be segmented by a numeric range, or as a date, but instead creating a string data type. This happens when the first value sent to Iterable for a field is a string format. Iterable requires specific formats for the data type to be set correctly.
The data type of an existing field is listed underneath each field in Audience > Segmentation, and can be found in Settings > Data Schema Management.
For instance, you can create age
as a field and set it as a string the
first time the field is sent to Iterable by sending an API request with the
age
field set to 26 years
. Iterable will set the age
field as a string,
and it cannot later be changed to a numeric data type.
When you segment users based on the age
field, Iterable will treat the field
as a string and not as a number. This means that you can't segment users by
age ranges or by users who are older or younger than a certain age. Instead,
you can only segment users by the exact age string value (26 years
) that was sent to Iterable.
In this case, the segmentation tool would treat the field as a string:
If a user wanted to use segmentation options given to a numeric field, like the image below (such as Less Than or Greater Than), it wouldn't be possible since the field has already been set as a string.
User field data types can also be found in Settings > Data Schema Management.
# Data types in Iterable
Below is a description of each data type and an example of how a field of that type would look in a sample payload as it is passed into Iterable using an API endpoint.
# String
Strings can contain both characters and numbers. Iterable treats all data as strings by default. Iterable's API automatically sets a new field's data type to string if the first value you provide for it is a string. If the value you provide doesn't match an accepted format for another data type, Iterable sets the field's data type to string.
When setting a string value with Iterable's API, surround its value in quotes.
The following request body for the POST /api/users/update
endpoint sets the user's first name to Justin
:
{ "email": "user@example.com", "dataFields": { "firstName": "Justin" } }
# Special characters in strings
Iterable receives and stores data in JSON format with UTF-8-encoded Unicode. Some special characters have specific meanings in JSON. These reserved characters need to be escaped when sending data to Iterable. By escaping them, you can include them in the value of a string without disrupting the JSON structure.
The following characters are reserved for JSON use and must be properly escaped to be used inside the value of a string.
-
Double quote - replace with
\"
-
Newline - replace with
\n
-
Backslash - replace with
\\
(To see a complete list of JSON escape characters and use a conversion helper, see this resource: JSON Escape)
# Date
A date field is a specific type of string in Iterable. Fields that are dates
have many uses for segmentation in Iterable, like relative dates
and date ranges. Iterable requires a specific date format in order to segment a
field as a date
type.
When creating a date field using Iterable's API, the date field value must be
in an accepted format. If the value is not in the correct format, the API call
creates a string
data type instead of date
.
Iterable accepts the following ISO 8601 standard date formats, with some additional formatting requirements:
-
yyyy-MM-dd: complete date (
2000-01-01
) -
yyyy-MM-dd HH:mm:ss: complete date and time (
2000-01-01 00:00:00
)- A space is required between the date and time values.
-
yyyy-MM-dd HH:mm:ss ZZ: complete date and time with time zone
(
2000-01-01 00:00:00 -04:00
)- The time zone offset (
ZZ
) is optional, but if present, it must be a four-digit offset as described in the next section. - A space is required between the date and time values, and between the time and time zone values.
- The time zone offset (
-
yyyy-MM-ddTHH:mm:ss.SSSZZ: complete date and time with milliseconds and
time zone (
2000-01-01T00:00:00.000-04:00
or2000-01-01T00:00:00.000Z
)- A literal
T
is required between the date and time values. - The milliseconds value (
.SSS
) is required. - The time offset (
ZZ
) value is required and must beZ
(for UTC) or a four-digit hourly offset with a+
or-
sign (such as+08:00
, or-04:00
).
- A literal
The following request body for the POST /api/users/update
endpoint sets the date for the anniversary
field:
{ "email": "user@example.com", "dataFields": { "anniversary": "2015-02-03 01:20:55 +06:00" } }
# Specifying the time zone of a date value
When you set a date
field in Iterable, you can specify the time zone in its
value (ZZ).
When a time zone is absent, Iterable assumes the date is in UTC (00:00
or
Z
).
To specify a time zone other than UTC, include the time zone offset in the date
format. The time zone offset is the difference between UTC and the local time
zone. When using the date format with time zone, use the format of ±hh:mm
(e.
g., -00:00
for UTC, -05:00
for EST, +02:00
for CEST, etc.).
# Common problems with date formats
Iterable only accepts the date formats listed above. If the value of a field
contains a format that is not accepted for the date
field type, the following
happens:
- If the field already exists in
date
format, Iterable rejects the request and returns an error. - If the request creates a new field, Iterable sets the field's data type to
string
instead ofdate
.
Examples of formats that Iterable does not consider valid for the date
field
type:
-
2024-11-25T13:37:19.039
(missing offset) -
2024-11-25T13:37:19.039 00:00
(offset is missing a preceding+
or-
) -
2024-11-25T13:37:19Z
(missing milliseconds) -
2024/11/25T13:37:19.039Z
(using slashes instead of hyphens) -
2024-11-25 13:37:19.039Z
(missingT
separator between date and time)
# Long
Longs are numbers that do not have decimal points.
The following request body for the POST /api/users/update
endpoint sets the user profile's age
field:
{ "email": "user@example.com", "dataFields": { "age": 26 } }
# Boolean
True or false. Acceptable Boolean values include true
and false
.
The following request body for the POST /api/users/update
endpoint sets the user profile's hasCat
field:
{ "email": "user@example.com", "dataFields": { "hasCat": false } }
# Double
Doubles are numbers that have decimal points.
The following request body for the POST /api/commerce/updateCart
endpoint sets an item's price
field to a double:
{ "user": { "email": "user@example.com" }, "items": [ { "id": "123", "sku": "123", "name": "XL Pizza", "description": "Piping hot and delicious", "categories": [ "food", "pizza" ], "price": 24.99, "quantity": 1 } ] }
# Geo_location
A geo_location field stores a location's latitude and longitude.
The name of any field that stores a geo_location value must have a
_geo_location
suffix. Example: address_geo_location
.
The geo_location object may only contain two fields: lat
(the latitude)
and lon
(the longitude). Do not put other fields in this object.
For example, the following request body for the POST /api/users/update
endpoint creates a work_geo_location
field on the user's profile:
{ "email": "user@example.com", "dataFields": { "work_geo_location": { "lat": 41.505493, "lon": -81.681290 } } }
NOTES
- Iterable's segmentation tool cannot perform geo_location queries (proximity, etc.). However, catalog items can contain geo_location values, and collections can use geo_location queries to compare them to the values stored on a user's profile.
- When defining a geo_location field on a catalog item, there is no need to
use the
_geo_location
suffix. The suffix only applies to geo_location values stored on user profiles and custom events.
# Object
Objects are collections of related fields. For example, to represent the various attributes of a cat, use an object.
The following request body for the POST /api/users/update
endpoint creates a cat
object on a user's profile:
{ "email": "user@example.com", "dataFields": { "cat": { "name": "Snowball", "color": "White", "age": 4, "favoriteToy": "Fluffy stick" } } }
Iterable's segmentation tool displays objects with a period between the name
of the object and the name of the field being referenced. For example, the
tool would represent the field for the above cat's name as cat.name
.
NOTE
Objects can be edited with API calls (not CSV uploads).
# Nested
A nested data type contains an array of objects that is indexed in their context, rather than as independent field values.
To understand the differences between the object data type, nested data type, and nested fields, read Choosing between object and nested data types.
Important notes about nested
data types for Iterable:
- Nested data types are only available for user profile fields, not custom events.
- Nested fields are not available in the Data Schema Management screen.
- To create a new nested field, contact your customer success manager or Iterable support.
- Nested fields can be edited with API calls (not CSV uploads, and not in the Settings > Data Schema Management screen).
Here is an example of a nested data type that contains multiple items, called
cats
:
{ "email": "user@example.com", "dataFields": { "cats": [ { "name": "Snowball", "color": "White", "age": 4, "favoriteToy": "Fluffy stick" }, { "name": "Whiskers", "color": "Orange", "age": 2, "favoriteToy": "Laser pointer" } ] } }
# Array
Arrays are lists of values that can contain any data type, including objects. Arrays are not a separate data type but a data structure that can store multiple values of any particular data type.
Iterable automatically treats any field with an array value as an array, while the values within the array are treated as the data type of the field. For example, you can store an array of strings within a string data type.
In the Data Schema Management screen, there is no specific option for arrays in the data type drop-down, and fields that contain arrays are treated as the data type of the values they contain (string, date, long, double, Boolean, geo_location, or object).
The following request body for the POST /api/users/update
endpoint creates a favoriteFoods
array on a user's profile, and the
favoriteFoods
field is classified with a string data type:
{ "email": "user@example.com", "dataFields": { "favoriteFoods": [ "Pizza", "Bánh mì", "Ramen" ] } }
NOTE
Arrays can be updated with API calls, but not CSV uploads.
# Want to learn more?
For more information about some of the topics in this article, check out this Iterable Academy course. Iterable Academy is open to everyone — you don't need to be an Iterable customer!