SFTP Connections

There are many wonderful applications for SFTP (Secure File Transfer Protocol) as a means to create an ongoing import of data into CaliberMind. For example:

  • Custom File Feeds from your internal Data Warehouse
  • File imports from your company's software product
  • Migrating and consolidating legacy systems
  • Ingesting budget and financial data for ROI analysis

Because of this we, at CaliberMind, wanted to make this as easy as possible. All CaliberMind customers get access to a free SFTP server that they can use to send data to us on a set schedule. If you need a good quick way to connect your computer to an SFTP, we recommend FileZilla. Also, if you have your own SFTP site you'd prefer to use, we support that too!

We can support 3 different data load styles:

  • Rebuild - each new file will completely replace the previous file sent
  • Upsert - each new file will add (or update) rows to your table in CaliberMind based on a primary key and datestamp.
  • Append - each file contains new rows to get added to the table (doesn't check unique ids or dedupe data). For example you'd select this style if you were sending event logs, logins, etc.

Creating a CaliberMind-provided SFTP

  1. Navigate to Settings > Integrations > New Connection
  2. Click "SFTP (CM)" > Add Connector

  1. Create folders: each folder will be created as a list (table) in CaliberMind. So a folder called "marketing/budget" would show up as a table called sftp_marketing_budget. In the example below, the user is planning to send 5 different files. Each may have a different file load style (see above), and will begin syncing within 1 hr landing on your SFTP:
    1. Rebuild - each new file will completely replace the previous file sent
    2. Upsert - each new file will add (or update) rows to your table in CaliberMind based on a primary key and datestamp.
    3. Append - each file contains new rows to get added to the table (doesn't check unique ids or dedupe data). For example you'd select this style if you were sending event logs, logins, etc.

  1. After creating the SFTP, you'll see displayed your secure login information to provide to your IT team. Please treat this access info as highly-sensitive! Also copy and paste your credentials into a safe place -- for security reasons, we won't show them again after the SFTP is created.

Sending data into CaliberMind from your internally-hosted SFTP

Before creating a connection to your internal SFTP, request your IT team whitelists the following IP addresses. Here is the list of CaliberMind IP addresses.

  1. Navigate to Settings > Integrations > New Connection
  2. Click "SFTP File Store" > Add Connector
  1. Name your connection and add in connection information (your IT/Engineering team can provide this).
  • Host
  • Port
  • Username
  • Password

  1. Select which folders on your SFTP to sync over. Each folder will be created as a list (table) in CaliberMind. So a folder called "marketing/budget" would show up as a table called sftp_marketing_budget.
  2. For each folder, select a load style. Each folder maps to 1 table and may have a different file load style (see above), and will begin syncing within 1 hr landing on your SFTP.

That's it! Moving forward, you will start to see LISTS for each folder making use of your previously siloed data!


How did we do?

Powered by HelpDocs

Powered by HelpDocs