Creating a Data Feed Web Service

You can create a data feed in Iterable that fetches live data from an external web service. Your marketing team can then use the data to personalize campaigns with up-to-date, relevant information.

This article covers the requirements and runtime behavior for data feeds in Iterable so you can design, develop, and tune a reliable data feed web service that powers real-time communication and drives timely engagement.

# In this article

# Requirements for a data feed web service

To work with Iterable, a data feed must:

• Respond to an HTTP GET request with expected HTTP response codes. See HTTP response code handling.
• Return a supported response format (JSON, XML, RSS, or Atom). See Data feed response bodies.
• Respond within 10 seconds. See Response times, retries, and timeouts.

Beyond basic functional requirements, here are some additional considerations to keep in mind:

• Whether you need to allow Iterable's IP addresses to access the service (for example, if it's behind a firewall).

• How to manage traffic and scale your data feed web service for performance and reliability. Iterable offers rate limits and caching.

• How to monitor data feed reliability to ensure your team is aware of any errors as they occur and can take action to remediate issues.

• Whether the feed should be static or dynamic. Static feeds return the same response for everyone and typically scale well. Dynamic feeds personalize responses per recipient by using query string parameters or custom HTTP headers. This choice also affects how Iterable caches the response.

• Whether to include an Authorization header and token in the request. Iterable supports static tokens only (not dynamic OAuth flows).

# Traffic management and scaling for data feeds

It's important for a data feed to be reliable and responsive when campaigns send. Without proper traffic management and scaling, a data feed can become unreliable and slow down campaigns, causing delays in sending and potentially impacting the customer experience.

Consequences of not managing traffic and scaling for data feed requests include:

• If a response is unavailable, Iterable skips messages to users. Iterable logs a send skip event with the reason DataFeedError on the user's profile.
• If the response is cached incorrectly, Iterable serves outdated data to users instead of fresh data.

Iterable provides rate limiting (beta feature) and caching to help manage traffic and scale a data feed web service.

# Data feed rate limiting (beta feature)

Iterable can apply a rate limit to a data feed to throttle requests to the service and prevent it from being overwhelmed. This is a good way to prevent overloading the service and ensure that it is reliable when sending high-volume campaigns.

Rate limiting is a setting that can be enabled on a per-data feed basis in Iterable. To learn more, see Data feed rate limiting.

# Data feed caching

Iterable can cache the response from a data feed web service for 1 hour, which can reduce the number of requests to the service and improve performance. This reduces load on the service and helps ensure it is responsive when sending campaigns.

However, not all scenarios are suitable for caching. Work with your marketing team to determine if caching is a good fit for your data feed and desired outcomes, and ensure your service is able to handle inbound requests without caching if needed.

Caching is a setting that can be enabled on a per-template basis in Iterable. Make sure your marketing team knows when to use it and when not to. To learn how to enable caching in a template, see Cache data feed response (up to 1 hr).

# Caching architecture

Cache behavior varies based on the type of data feed (static or dynamic):

• Static data feeds (no Handlebars in the URL or headers) use a global cache shared across all sending nodes. For blast campaigns, Iterable pre-warms this cache before sending begins. This provides scalable caching—the same cached data is used for all users across all sending nodes.

• Dynamic data feeds (Handlebars in the URL or headers) use a local cache on each sending node. When sending to a large number of users, Iterable scales campaign sends by processing batches of users in parallel on different nodes. Even if many users share the same data feed URL, Iterable requests the data feed once per unique URL per node. Caching still provides scalability, but it's not as efficient as static data feeds and ultimately requires more requests to the service.

# Design considerations for caching

When designing your data feed web service, consider how caching affects your service's traffic patterns and decide whether caching is a good fit for your data feed and desired outcomes.

Here are some common scenarios and their considerations for caching:

• Static data feeds with infrequent updates: If the data feed content changes infrequently (daily or weekly), caching is beneficial and reduces load significantly. The global cache used for static feeds means that your service receives minimal requests while campaigns are sending. This enables efficient scaling of campaign sends without risking stale content.

• Static data feeds with frequent updates: If the data feed updates multiple times per day and freshness is critical (like inventory levels that change quickly), caching may cause campaigns to deliver stale content. Ensure your service can handle the full request volume without caching, or design around acceptable staleness windows. Iterable's data feed caching persists for one hour, and that period is not configurable. If the data feed is updated multiple times per hour, the cached data risks becoming stale and caching may not be a good fit for your data feed's purpose.

• Dynamic data feeds with unique-per-user URLs: When the URL contains a merge tag unique to each user (such as a user ID), caching provides no benefit—each user generates a unique cache that is unlikely to be reused in the next hour before the cache expires. Design your service to handle up to five requests per user per campaign (to account for retries), and let your marketing team know that caching should be disabled for templates that use the data feed.

• Dynamic data feeds with shared request patterns: When dynamic URLs and/or headers resolve to a limited set of values shared across users (such as locale, product category, or membership tier), caching can help reduce load. But, because caching is per-node, your service still receives multiple requests during high-volume sends—one from each sending node processing each unique request for the first time.

# Response times, retries, and timeouts

Iterable expects a data feed's web service to return a complete response within 10 seconds.

If the data feed's web service times out, or a response is not received within 10 seconds, Iterable re-queries the same web service 3 minutes later.

After 5 unsuccessful attempts, Iterable skips sending the message to the current user and logs a send skip event with the reason RetriesExhaustedError.

Overall, Iterable may make up to 5 requests per user per campaign (to account for retries). Your service should be able to handle this volume of requests, and you might want to use rate limiting to prevent overloading your service.

# HTTP response code handling

When a data feed queries the URL for its associated web service, Iterable handles HTTP response codes as follows:

# Monitoring data feed reliability

Iterable retries data feed requests and determines when to skip sending on a per-user basis. Iterable doesn't pause campaigns from sending based on aggregate error rates for the data feed's endpoint.

To ensure your web service is reliable when sending campaigns, implement monitoring and alerting for your data feed service (for example, high error rate, latency spikes, and timeout increases) so your team can quickly pause sends and remediate issues.

# Creating a dynamic data feed

When you set up a data feed to query a web service, you can include merge tags or other Handlebars expressions in the URL's query string, or as custom HTTP headers. Doing this creates a dynamic data feed.

Dynamic data feeds can fetch information relevant to particular users from their user profiles or a campaign's triggering event. They can also send data to your server to record and track information about the requests made to your service.

You can input data sourced from your user profiles, events, or campaign context data into the data feed URL or custom headers.

At send time, Iterable replaces the merge tags and Handlebars expressions with values and sends them to your service as part of the request, either in the URL query string or as custom headers. Your service can then use the data to record transactional data and/or personalize the response for each recipient.

For example, a dynamic data feed might query the following web service URL:

https://www.example.com/sample_data_feed.json?email={{#urlEncode}}{{email}}{{/urlEncode}}

Make sure your marketing team knows the full data feed URL and request headers, including the merge tags and Handlebars expressions. These properties are set when the data feed is configured in Iterable and don't vary by template.

# Data feed response bodies

Data feed web services must:

• Use UTF-8 character encoding.

• Provide response sizes of 4MB or less (the max allowable size for Iterable's preview with data functionality).

  Smaller data feed responses are easier to preview and test in Iterable. Larger responses or complex queries require more time to generate and may cause timeouts when Iterable tries to retrieve the response.

  NOTE

  If a template uses more than one data feed, the total response size for all data feeds must be less than 4MB.

For the data feed's response body, Iterable supports the following formats:

• JSON
• XML
• RSS
• Atom

# JSON data feed response format

JSON data feeds should return a top-level object (not an array) that includes a key-value pair. The object can contain a single object or an array of objects.

Example JSON response:

{
  "items": [
    {
      "sku": "TOY-1001",
      "name": "Classic Wooden Yo-Yo",
      "description": "A beginner-friendly wooden yo-yo with a smooth spin.",
      "price": 9.99,
      "inStock": true,
      "imageUrl": "https://example.com/images/toys/classic-wooden-yoyo.jpg",
      "productUrl": "https://example.com/products/classic-wooden-yoyo"
    },
    {
      "sku": "TOY-1002",
      "name": "Racing Toy Car",
      "description": "Die-cast toy car with pull-back motor action.",
      "price": 18.99,
      "inStock": true,
      "imageUrl": "https://example.com/images/toys/racing-toy-car.jpg",
      "productUrl": "https://example.com/products/racing-toy-car"
    },
    {
      "sku": "TOY-1003",
      "name": "Jumbo Building Blocks Set",
      "description": "50-piece colorful block set for creative play.",
      "price": 44.99,
      "inStock": false,
      "imageUrl": "https://example.com/images/toys/jumbo-building-blocks-set.jpg",
      "productUrl": "https://example.com/products/jumbo-building-blocks-set"
    }
  ]
}

You can reference the data in your template like this:

[[#each items]]
  [[name]] ([[sku]]) - [[price]]
  [[description]]
  [[#if inStock]]In stock[[else]]Out of stock[[/if]]
  [[productUrl]]
[[/each]]

# Invalid JSON responses

If you have a list of items, wrap it in an object with a key name (like items): { "items": [...] }.

Importantly, JSON arrays are not valid data feed responses.

The following response is invalid because it's an array (wrapped in a square bracket[) rather than an object with a top-level key name:

[
  {
    "sku": "TOY-1001",
    "name": "Classic Wooden Yo-Yo",
    "description": "A beginner-friendly wooden yo-yo with a smooth spin.",
    "price": 9.99,
    "inStock": true,
    "imageUrl": "https://example.com/images/toys/classic-wooden-yoyo.jpg",
    "productUrl": "https://example.com/products/classic-wooden-yoyo"
  },
  {
    "sku": "TOY-1002",
    "name": "Racing Toy Car",
    "description": "Die-cast toy car with pull-back motor action.",
    "price": 18.99,
    "inStock": true,
    "imageUrl": "https://example.com/images/toys/racing-toy-car.jpg",
    "productUrl": "https://example.com/products/racing-toy-car"
  },
  {
    "sku": "TOY-1003",
    "name": "Jumbo Building Blocks Set",
    "description": "50-piece colorful block set for creative play.",
    "price": 44.99,
    "inStock": false,
    "imageUrl": "https://example.com/images/toys/jumbo-building-blocks-set.jpg",
    "productUrl": "https://example.com/products/jumbo-building-blocks-set"
  }
]

# XML data feed response format

Iterable supports standard XML for data feed responses.

Example XML response:

<blog>
  <title>Iterable Blog</title>
  <link>https://www.iterable.com/blog/</link>
  <description>The latest product updates, marketing tips, and customer stories from Iterable.</description>
  <post>
    <title>How To Scale Personalization With Modular Content</title>
    <link>https://iterable.com/blog/modular-content-personalization/</link>
    <pubDate>2025-12-15T10:00:00Z</pubDate>
    <description>Learn how to scale personalization by assembling messages from reusable content blocks.</description>
    <author>Katie Gray</author>
  </post>
  <post>
    <title>Your BFCM Playbook: How To Ditch Discounts and Drive Long-Term Loyalty</title>
    <link>https://iterable.com/blog/bfcm-strategy-ditch-discounts/</link>
    <pubDate>2025-08-27T10:00:00Z</pubDate>
    <description>Swap that dog-eared BFCM playbook for smarter, data-driven customer marketing strategies.</description>
    <author>Manasi Patel</author>
  </post>
  <post>
    <title>Rapid Fire with the CEO: 10 Questions for Iterable's Sam Allen</title>
    <link>https://iterable.com/blog/rapid-fire-with-the-ceo-10-questions-for-iterables-sam-allen/</link>
    <pubDate>2025-08-11T10:00:00Z</pubDate>
    <description>Get to know Iterable's new CEO Sam Allen through ten rapid-fire questions about his vision, values, and first week on the job.</description>
    <author>Lauren Kopulsky</author>
  </post>
</blog>

Iterable converts the XML response to a JSON object (such as in the JSON format shown above) for processing.

If an XML element contains multiple child elements with the same tag name (such as multiple <post> tags), they are automatically converted into a JSON array under that key name.

The XML response above is converted to the following JSON object:

{
  "blog": {
    "title": "Iterable Blog",
    "link": "https://www.iterable.com/blog/",
    "description": "The latest product updates, marketing tips, and customer stories from Iterable.",
    "post": [
      {
        "title": "How To Scale Personalization With Modular Content",
        "link": "https://iterable.com/blog/modular-content-personalization/",
        "pubDate": "2025-12-15T10:00:00Z",
        "description": "Learn how to scale personalization by assembling messages from reusable content blocks.",
        "author": "Katie Gray"
      },
      {
        "title": "Your BFCM Playbook: How To Ditch Discounts and Drive Long-Term Loyalty",
        "link": "https://iterable.com/blog/bfcm-strategy-ditch-discounts/",
        "pubDate": "2025-08-27T10:00:00Z",
        "description": "Swap that dog-eared BFCM playbook for smarter, data-driven customer marketing strategies.",
        "author": "Manasi Patel"
      },
      {
        "title": "Rapid Fire with the CEO: 10 Questions for Iterable's Sam Allen",
        "link": "https://iterable.com/blog/rapid-fire-with-the-ceo-10-questions-for-iterables-sam-allen/",
        "pubDate": "2025-08-11T10:00:00Z",
        "description": "Get to know Iterable's new CEO Sam Allen through ten rapid-fire questions about his vision, values, and first week on the job.",
        "author": "Lauren Kopulsky"
      }
    ]
  }
}

In a template, you can reference the data like this:

[[#each blog.post]]
  [[title]] by [[author]]
  [[description]]
  Read more: [[link]]
[[/each]]

# RSS data feed response format

Iterable supports standard RSS 2.0 format.

Example RSS 2.0 response:

<rss version="2.0" xmlns:media="http://search.yahoo.com/mrss/">
  <channel>
    <title>Iterable Blog</title>
    <link>https://www.iterable.com/blog/</link>
    <description>The latest product updates, marketing tips, and customer stories from Iterable.</description>
    <language>en-US</language>
    <lastBuildDate>Mon, 15 Dec 2025 10:00:00 GMT</lastBuildDate>
    <item>
      <title>How To Scale Personalization With Modular Content</title>
      <link>https://iterable.com/blog/modular-content-personalization/</link>
      <guid isPermaLink="true">https://iterable.com/blog/modular-content-personalization/</guid>
      <pubDate>Mon, 15 Dec 2025 10:00:00 GMT</pubDate>
      <description>Learn how to scale personalization by assembling messages from reusable content blocks.</description>
      <author>Katie Gray</author>
      <media:content url="https://iterable.com/wp-content/uploads/2025/12/How-To-Scale-Personalization-With-Modular-Content.png" medium="image" height="1200" width="2400"/>
    </item>
    <item>
      <title>Your BFCM Playbook: How To Ditch Discounts and Drive Long-Term Loyalty</title>
      <link>https://iterable.com/blog/bfcm-strategy-ditch-discounts/</link>
      <guid isPermaLink="true">https://iterable.com/blog/bfcm-strategy-ditch-discounts/</guid>
      <pubDate>Wed, 27 Aug 2025 10:00:00 GMT</pubDate>
      <description>Swap that dog-eared BFCM playbook for smarter, data-driven customer marketing strategies.</description>
      <author>Manasi Patel</author>
      <media:content url="https://iterable.com/wp-content/uploads/2025/08/Black-Friday-Cyber-Monday-Playbook.png" medium="image" height="1200" width="2400"/>
    </item>
    <item>
      <title>Rapid Fire with the CEO: 10 Questions for Iterable's Sam Allen</title>
      <link>https://iterable.com/blog/rapid-fire-with-the-ceo-10-questions-for-iterables-sam-allen/</link>
      <guid isPermaLink="true">https://iterable.com/blog/rapid-fire-with-the-ceo-10-questions-for-iterables-sam-allen/</guid>
      <pubDate>Mon, 11 Aug 2025 10:00:00 GMT</pubDate>
      <description>Get to know Iterable's new CEO Sam Allen through ten rapid-fire questions about his vision, values, and first week on the job.</description>
      <author>Lauren Kopulsky</author>
      <media:content url="https://iterable.com/wp-content/uploads/2025/08/Dark-6.png" medium="image" height="1200" width="2400"/>
    </item>
  </channel>
</rss>

After retrieving the response, Iterable automatically extracts the content from <rss><channel> and converts it into a JSON object. The data available in your template is the contents of the channel tag. You don't need to navigate through rss.channel in your Handlebars expression. Instead, start directly with the channel tag's children (like title, description, item).

The item field is usually an array of item objects that are nested under the channel tag.

In a template, you can reference the data like this:

[[#each item]]
  [[title]]
  [[description]]
  Read more: [[link]]
[[/each]]

# Atom data feed response format

Iterable supports standard Atom XML format.

Example Atom response:

<feed xmlns="http://www.w3.org/2005/Atom">
  <title>Iterable Blog</title>
  <link href="https://www.iterable.com/blog/"/>
  <updated>2025-12-15T10:00:00Z</updated>
  <author>
    <name>Iterable</name>
  </author>
  <id>urn:uuid:60a76c80-d399-11d9-b93C-0003939e0af6</id>
  <entry>
    <title>How To Scale Personalization With Modular Content</title>
    <link href="https://iterable.com/blog/modular-content-personalization/"/>
    <id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id>
    <updated>2025-12-15T10:00:00Z</updated>
    <summary>Learn how to scale personalization by assembling messages from reusable content blocks.</summary>
  </entry>
  <entry>
    <title>Your BFCM Playbook: How To Ditch Discounts and Drive Long-Term Loyalty</title>
    <link href="https://iterable.com/blog/bfcm-strategy-ditch-discounts/"/>
    <id>urn:uuid:2336d796-dfc9-5fcc-bbbb-91eb455f0b7b</id>
    <updated>2025-08-27T10:00:00Z</updated>
    <summary>Swap that dog-eared BFCM playbook for smarter, data-driven customer marketing strategies.</summary>
  </entry>
  <entry>
    <title>Rapid Fire with the CEO: 10 Questions for Iterable's Sam Allen</title>
    <link href="https://iterable.com/blog/rapid-fire-with-the-ceo-10-questions-for-iterables-sam-allen/"/>
    <id>urn:uuid:3447e8a7-e0da-6edd-cccc-a2fc566g1c8c</id>
    <updated>2025-08-11T10:00:00Z</updated>
    <summary>Get to know Iterable's new CEO Sam Allen through ten rapid-fire questions about his vision, values, and first week on the job.</summary>
  </entry>
</feed>

After retrieving the response, Iterable extracts the content starting from <feed> and converts it into a JSON object similar to the examples in other formats above. The data available in your template is the contents of the feed tag. You don't need to navigate through feed in your Handlebars expression. Instead, start directly with the feed tag's children (like title, entry, etc.).

The entry field is usually an array of entry objects that are nested under the feed tag.

In a template, you can reference the data like this:

[[#each entry]]
  [[title]]
  [[summary]]
  Read more: [[link._href]]
[[/each]]

# Next steps

When the data feed is ready to test in Iterable, developers should work with their marketing team to:

• Provide the necessary information to configure the data feed in Iterable, including all of the properties listed in Data feed properties.
• Advise on which data feed settings should be used in templates. For a description of each setting available, see Template settings for data feeds.
• Test the data feed to make sure it's working as expected before using it for production marketing campaigns.
• Continually maintain the data feed to ensure it's up to date and working as expected. The marketing team should report any issues to the development team, and vice-versa.

# Want to learn more?

For more information about some of the topics in this article, check out these resources. Iterable Academy is open to everyone — you don't need to be an Iterable customer!

Iterable Academy

• Product Bite: Data Feeds

Support docs

• Data Feeds Overview
• Managing Data Feeds
• Using Data Feeds in Templates
• Using Data Feeds in Snippets
• Handlebars Overview
• Templates Overview