Microsoft Fabric + Smart Ingest

Smart Ingest lets you pull data stored in your Microsoft Fabric data warehouse and import to Iterable. This allows you to sync data into Iterable for use in your marketing campaigns, segmentation, and analytics. With Smart Ingest, you can easily connect to your Microsoft Fabric instance and start syncing data in just a few steps. Smart Ingest handles the heavy lifting by automatically tracking changes to your data and syncing only the incremental changes, ensuring your data in Iterable is always fresh!

# In this article

# Connection requirements

To connect Microsoft Fabric to Iterable, you need to set up a few things in Microsoft Entra, Microsoft Fabric, and Iterable. You also need to decide on the sync engine you want to use.

Your Microsoft administrator can follow the steps below to set up a service principal for Microsoft Fabric, and may optionally need to allowlist Smart Ingest IPs or provide additional permissions for the Lightning sync engine.

Then, the Iterable user with the Manage Integrations project permission can connect Microsoft Fabric to Iterable with the following details:

# Microsoft Fabric connection details

• Host – The hostname or IP address of your Microsoft Fabric instance. If using the hostname, make sure to only insert the hostname and not the full URL (remove the leading http:// and the final /).

• Port – The port number of your Microsoft Fabric instance. The default is 1433, but yours may be different.

• Database – The name of the database in your Microsoft Fabric instance. Smart Ingest queries the data from this database.

• Request timeout duration (optional) - The time in seconds that Smart Ingest waits for a response from Microsoft Fabric before timing out. The default is 30 seconds.

• Trust server certificate (optional) - Whether to trust the Microsoft Fabric server certificate. The default is false. Use this setting for self-signed certificates not signed by a trusted certificate authority. If you set this to true, Smart Ingest will trust the server certificate without validating it.

# Microsoft service principal credentials

• Tenant ID (or Directory ID)
• Client ID (or Application ID)
• Client secret

The Microsoft service principal needs permissions to access Microsoft Fabric, which depend on your sync engine choice:

• Basic sync engine: Read all data using SQL (ReadData)
• Lightning sync engine: Read all data using SQL (ReadData) and additional permissions granted using a SQL script provided in the Smart Ingest sync setup.

# Connecting Microsoft Fabric to Smart Ingest

In Microsoft Entra, you need to create a service principal. Then, you'll need to grant it access to your Fabric resource. You may also need to allow Smart Ingest IPs if your organization has a firewall in place.

After that, you can connect Microsoft Fabric to Iterable.

# Step 1: Create a service principal in Microsoft Entra

First, create a service principal. These are the credentials Smart Ingest will use to access your Microsoft Fabric resources.

1. Sign in to the Microsoft Entra admin center.
2. Go to Identity > Applications > App registrations and click New registration.
3. Enter a name for your application, and select Accounts in this organizational directory only as the supported account type.
4. Click Register.
5. Note the registration's Application (client) ID and Directory (tenant) ID.
6. Click Add a certificate or secret and then New client secret.
7. Name your client secret and select an expiration date.
8. Note your client secret Value.

For more detailed instructions on service principal creation, see Microsoft's documentation: Register a Microsoft Entra app and create a service principal.

# Step 2: Configure Microsoft Fabric

Microsoft Fabric may not be accessible via service principal by default. To connect, take the following steps:

1. Sign in to Microsoft Fabric.
2. Go to Synapse Data Engineering.
3. Go to Settings > Governance and insights > Admin portal.
4. On the sidebar, select Tenant settings.
5. Scroll down to find Developer settings, and select Service principals can use Fabric APIs so that the toggle is set to Enabled.
6. Next go to OneLake settings and select Users can access data stored in OneLake with apps external to Fabric so that the toggle is also Enabled.

# Step 3: Attach your service principal to your Fabric warehouse

1. Sign in to Microsoft Fabric.
2. Open the workspace that contains your warehouse.
3. Hover over your warehouse, click on the overflow menu (three dots), and select Manage permissions.
4. On the next page, click on Add user.
5. Enter the name of your service principal, and select Read all data using SQL (ReadData).
6. Click Grant.

This provides read permission to the service principal.

# Step 3.1: Granting additional permissions for Lightning sync engine (optional)

If you're using the Lightning sync engine, you'll also need to run the SQL script provided by Hightouch to grant additional permissions. This script is available in the Iterable Smart Ingest UI when you select the Lightning sync engine.

# Step 4: Allow Smart Ingest IPs in Microsoft Entra

Entra can protect your data by preventing unknown external IP addresses from connecting to your Fabric database. Regardless of how you're authenticating, your Entra administrator may need to add Smart Ingest IP addresses as part of your trusted network. This is a one-time setup.

To allowlist Smart Ingest IPs in Entra, ask your Entra administrator to allowlist the Smart Ingest IPs for your region by updating the conditional access policies in the Microsoft Entra admin center.

To learn more about conditional access network policies, see the Microsoft Entra documentation on Conditional Access: Network Assignment.

# Step 5: Connect Microsoft Fabric to Smart Ingest

To connect Smart Ingest to Microsoft Fabric:

1. Log in to Iterable as a user with the Manage Integrations project permission and open the project you’re working on.

2. Go to Integrations > Smart Ingest.

3. Click Connect a New Source.

4. Select Microsoft Fabric and click Continue.

5. Enter your Microsoft Fabric instance details: Host, Port, and Database. Optionally, you can also enter Request timeout duration and Trust server certification.

6. Select the sync engine you want to use: Basic or Lightning.

   • If you select Basic, you can switch to Lightning later.
   • If you select Lightning:
     • Additional permissions are required. When you select the Lightning sync in this step, Smart Ingest reveals a SQL script provided by Hightouch that will grant the minimum permissions needed to use the Lightning sync engine. Provide your Microsoft Fabric administrator this SQL query, or use the Basic sync if permissions cannot be granted.
     • To change the sync engine back to Basic, create a new connection to Microsoft Fabric.

7. Enter your Microsoft Fabric credentials:

   • Tenant ID (or Directory ID)
   • Client ID (or Application ID)
   • Client secret

8. Click Continue. Smart Ingest automatically tests the connection.

9. When the connection test is successful, click Continue. (If there are problems connecting, click Back and review the connection details for accuracy.)

10. Add a name for your data source to display in Iterable.

11. Click Finish.

# Next steps

You've now connected your data warehouse to Smart Ingest. The next thing you can do is create a sync.

# Troubleshooting

Most issues with the Microsoft Fabric source can be resolved by verifying that your Microsoft Fabric configuration is correct. It must allow external connections via service principal.