Articles in this section

See more

SSH Tunneling for Smart Ingest

Tunneling lets Smart Ingest securely open a connection to a data warehouse in your private network or Virtual Private Cloud (VPC) without exposing it to the internet. SSH tunnels are secure, authenticated, encrypted, and dedicated to your workspace. To learn more about SSH tunneling, read this introductory article (SSH.com).

Smart Ingest supports two types of SSH tunnels: standard and reverse. This article explains the differences between the two and provides instructions for setting up each type.

The following Smart Ingest sources support SSH tunneling:

In this article

Tunnel types

Both standard and reverse tunnels open a secure port connection between Smart Ingest and a supported source. However, they differ in implementation, and you may prefer one based on your network specifications.

Standard SSH tunnel

Standard SSH tunnel requires you to run sshd on a bastion host accessible from the public internet. Smart Ingest opens an SSH connection to your bastion host, then opens a port forwarding connection to the source within your private network. To learn how to set this up, read Create a standard SSH tunnel.

Reverse SSH tunnel

Reverse SSH tunnel allows you to forward a port by connecting as a client to an SSH server managed by Smart Ingest. This eliminates the need for a bastion host in your infrastructure, but requires you to maintain the connection. If your connection goes down for any reason, your systems should automatically re-open the connection. You can use programs like autossh, Docker container restart policies, or process supervisors like supervisord to help maintain the connection. To learn how to set this connection up, read Create a reverse SSH tunnel.

Create a standard SSH tunnel

Requirements

  • Allow connections from the bastion host to your warehouse.
  • Set up a user on the bastion host named hightouch.

Instructions

To connect Smart Ingest to a data source via a standard tunnel:

  1. Log in to Iterable as a user with the Manage Integrations project permission and open the project you’re working on.

  2. Go to Integrations > Smart Ingest.

  3. Click Connect a New Source.

  4. Select your data source name and click Connect.

  5. Find Step 1 and choose Connect via SSH Tunnel.

  6. Click New Tunnel.

  7. Select Standard Tunnel then click Continue.

  8. Enter a Display Name for your tunnel.

    Standard tunnel

  9. Enter the SSH Host and SSH Port. These are the IP address and port for your public-facing bastion server host. The port is most likely 22, standard for sshd.

  10. Fill out the Service Host and Service Port. These are the IP address and port for your data warehouse.

  11. Click Create Tunnel.

  12. Copy or download the generated SSH public key. Add this to the \~/.ssh/authorized_keys file for the hightouch user on your bastion server. You can use ssh-copy-id to help with this.

  13. The tunnel Status turns green when the connection is established. Your tunnel is now ready for use.

Create a reverse SSH tunnel

Requirements

  • You need a server within your VPC to act as the SSH client.

  • The SSH client server must be able to connect to both the public internet and your data source.

Instructions

To connect Smart Ingest to a data source via a reverse tunnel:

  1. Log in to Iterable as a user with the Manage Integrations project permission and open the project you’re working on.

  2. Go to Integrations > Smart Ingest.

  3. Click Connect a New Source.

  4. Select a data source name and click Connect.

  5. Find Step 1 and choose Connect via SSH Tunnel.

  6. Click New Tunnel.

  7. Select Reverse Tunnel then click Continue.

  8. Enter a Name for your tunnel, then click Create Reverse Tunnel.

    Reverse tunnel

  9. Click Download Private Key to save the generated private key to your local computer as a .pem file.

    IMPORTANT

    Iterable doesn't store your private key. As a result, we can't recover a lost key. Please make a local copy of this key and store it securely.

  10. Copy the provided ssh command and save it.

    Your command may be similar to this example:

    ssh -i path/to/key.pem \
    -R 0.0.0.0:56000:$SERVICE_HOST:$SERVICE_PORT \
    tunnel.hightouch.io -p 49100 \
    -o ExitOnForwardFailure=yes

  11. Upload the private key to your SSH client server, store it securely, and ensure its permissions are set to 0400 (allows owner to read).

  12. Edit the SSH command:

    • Update path/to/key.pem to match the path for the location of the private key you saved.

    • Set or replace the $SERVICE_HOST and $SERVICE_PORT variables with your internal warehouse service host and port.

  13. From your SSH client server, run the modified ssh command.

TIP

You most likely want to wrap this ssh command with a process manager to restart in case of failure. Consider autossh.

  1. The tunnel Status turns green when the connection is established. Your tunnel is now ready for use.

Reverse SSH host key info

For security reasons, you may need proof that your SSH tunnel is connecting to the correct Smart Ingest host.

To confirm that your tunnel is connected to the right host, check that the Smart Ingest hostname is correctly set to one of these:

  • tunnel.aws-us-east-1.hightouch.com
  • tunnel.aws-eu-west-1.hightouch.com
  • tunnel.aws-ap-south-1.hightouch.com
  • tunnel.gcp-us-east4.production.hightouch.com

The actual hostname varies depending on your Iterable region.

You can also check that the Host Key is correctly included in ~/.ssh/known_hosts as follows:

ssh-rsa
AAAAB3NzaC1yc2EAAAADAQABAAABgQCXjpH+c24rYGDBoUdHGN3Yazp4SrDU8Z0Oqb0WRaKJz0VpSPS+Kqmc0MioY3yK1aPDh32VSEFUykEHIobV7dsH9uKlR3OllTkxN9pIk3xfJBn1/x4TS2VX1I+XM7KE6xJH/QmBVaVyNzsYzZ+lR/hUdrVC3czV+ReJa5j+YS23atS3x9NznxHSBBOLcq0RpFj65/WZt0kJwbIqhrGYIq5qXR/8d7gzyCj6vg2iNNUszTFQvcFeQ8zeXC63ke4C9oCFHbneX2odvsDY5DQM6WF7H6j91qRhGEq+niQg2zCns1kJq1Gfa7bY3OxIHmQ3HP0R8ExJMsBBqdKoL5uNY8UK717yNuFzBQUV8fgJi1HZC6JR6uDVpLadbXUV3WD0LA0YQ1h24KKGtPRDrnyZxNTdn0R19asz31npyFHKrP95OIz2RoGuqsn9fr3FzO1wfXjdzYp5l6AAy04K6yPNQfTC/+LLk8U1PWbSsMBqlQnHkp/eMaSdTEVMVxqhAFKCz8M=

NOTE

Smart Ingest is co-developed by Iterable and Hightouch. Hightouch is a data processor for this feature. Smart Ingest data operations and schemas may contain the Hightouch name, but the feature is fully supported by Iterable.

Standard tunnel troubleshooting

If you're having trouble establishing a connection with a standard tunnel:

  • Check that you've allowlisted Smart Ingest IP addresses on your bastion host.

  • Check that the hightouch user exists on your bastion host and that the Smart Ingest public key is in their ~/.ssh/authorized_keys file.

  • Check permissions on the hightouch user's SSH files:

    • ~/.ssh directory should be 0700

      • Fix: chmod 0700 /home/hightouch/.ssh

    • ~/.ssh/authorized_keys file should be 0644

      • Fix: chmod 0644 /home/hightouch/.ssh/authorized_keys

    • ~/.ssh directory and ~/.ssh/authorized_keys file should be owned by the hightouch user

      • Fix: chown -R /home/hightouch/.ssh

  • Check that the bastion host can network to your warehouse.

    • nc -z \$warehouse_host \$warehouse_port

Reverse tunnel troubleshooting

If you're having trouble establishing a connection with a reverse tunnel:

  • Check that your SSH client server is running and can access the public internet.

  • Check that you've uploaded the private key and that it's only readable by the user initiating the SSH connection (chmod 0400).

  • Add the -v (verbose) flag to your SSH command to see more detailed errors.

Related articles