Precog Universal API Connector for Analytics and Machine Learning

Precog’s Universal API Connector provides a fast, simple, No Code approach for configuring analytic ready access to any REST API. Analysts and Data Scientists can create entirely new analytic ready connectors in minutes with Zero code ever.

To demonstrate how to configure a new API Connector, we’ll walk through each of the configuration steps.

Variables

User-defined Variables can be used anywhere in the connector configuration. We recommend defining a Variable for any value you’ll need to use more than once.

Throttling

Precog will automatically throttle requests so as not to exceed the rate limit prescribed by the API. If the API you’re configuring enforces a rate limit, enter that information here.

Authentication

Precog supports authentication via Basic Authentication, OAuth 2.0, and Request Token Authentication. In this example, we consider the OAuth 2.0 authentication flow.

Enter the Authorization Endpoint and Token Endpoint , as specified by the API documentation. Then enter your Client Id and Client Secret , which are generally provided when you create a new application in the API’s developer portal.

Then enter any additional Scopes and Authorization Parameters, as required per the API documentation. Select the Client Authentication Method that will be used during the authentication flow.

Authentication

Request headers can be defined globally and per dataset. In this section, define the request headers that will be applied globally (i.e. to each API request).

Datasets – Parent Datasets

Next, we configure the datasets. In this example, each dataset connects to an Asana API endpoint.

Here we create the Projects dataset, which provides data about all the Asana Projects available.

Now that we have access to all the Asana Projects, we’d like to gather information about each of the Projects individually. To achieve this, we can use the Projects dataset as a Parent Dataset for a subsequent API request.

Here we see that the Project dataset is configured with the Projects dataset as its Parent Dataset. This configuration specifies that each Project Id value (which Asana calls a gid) will be passed individually into the endpoint:

https://app.asana.com/api/1.0/projects/{{precog_dac4c027-b3bf-4550-81a4-8764256563ec}}

in place of the variable {{precog_dac4c027-b3bf-4550-81a4-8764256563ec}}. This will produce one dataset that contains the detailed information about all the Asana projects.

Datasets – Pagination

API endpoints often return results spread across multiple pages. To support these APIs, Precog supports various kinds of pagination. In particular, Precog supports: Token Pagination , Offset Pagination , and Header Pagination, each of which include various configurations of their own.

Here we see the Offset Pagination configuration for Precog’s MailChimp connector. Precog will automatically make one request per page until there are no more pages available.

Support for New Connectors

We love supporting customers as they configure new API Connectors. If you’re working on configuring your own API Connector and you’d like some help, we’d love to jump on a call with you.

Additionally, we’re always adding new pre-configured API Connectors as well as adding new features, so let us know if there’s a Connector or feature you’d like to see added!

Summary

Once a new connector is configured Precog will automatically create tables from the source data. These are fully SQL ready tables you can use in any BI or ML tool, load into a Data Warehouse or simply download as a CSV for offline use.

The “magic” of Precog is you don’t need to do any manual mapping or parsing of the API data, simply pick from the list of tables presented in the Precog UI and start using it with your favorite tool. And one of the best features is these tables are all fully customizable. You can add or remove new columns easily thru the UI to get the exact data you need in minutes.

Precog can manage all your tables and load them on a schedule from every 30 seconds to once per day and everything in between, or you can choose to load them anytime by simply choosing “Load All” from the menu. When you load the data Precog will automatically fetch the most recently updated data from the API.

Now anyone can access any API and get analytic ready tables in minutes that work with your favorite BI tool, Machine Learning tool or any Data Warehouse or Database.

Ready to Start?

FROM OUR CUSTOMERS

GiddyUp

Precog delivers on the dream of simple data architecture that is roaring across the world. Precog solves all these problems, keeping your warehouse up to date with all the data you need and making the ELT dream a reality.

Venkatarama Cherukupalli
Walnut St. Labs

Precog lets us prototype analytics projects quickly — building marketing dashboards based on data from a variety of sources — without needing a data engineer or developer — we create new data sources in a few hours to sources like Brightlocal, a popular local SEO SaaS solution, and h... Read More

Chris Dima - CEO
Alteryx

We welcome Precog to the Alteryx technology partner ecosystem as a partner extending the capabilities of our platform, further simplifying analytics for our customers.

Hakan Soderbom - Director of Technology Alliances
Data.World

Enterprises struggle to understand and trust the data sources powering their business analyses,” said Jon Loyens, co-founder and chief product officer at data.world. “Adding ways to integrate sources to our catalog introduces more flexibility to our users, increasing their efficiency a... Read More

Jon Loyens - Co-Founder and CPO
SouthEnd

We recognized a need in our customer base to perform advanced analytics on SAP data sets — we performed an extensive evaluation of Precog and chose it as a strategic solution for our go to market needs based on its performance and given their strong strategic relationship with SAP.

Alfredo Poncio - CEO
SouthEnd
Everflow

Precog changed the game for us — instead of grueling data integration work, Precog offers a ‘connect and go’ experience — this allows us to reallocate resources to our product and our customers.

Sam Darawish - CEO
SendaRide

Precog is the vital tool in our ability to pull data from a variety of business sources quickly and cleanly. Our internal MongoDB backend, as well as other cloud services like Hubspot, were a constant challenge to the business teams desire for reporting data prior to using Precog. With the... Read More

Josh Wilsie - VP