How To Access Any Data Source from Alteryx
By Chris Dima
May 24, 2021

Transcript below:

Jeff Carr

Welcome to the Alteryx Inspire 2021 conference. I’m Jeff Carr with Precog and I’m here today to talk to you about the Precog solution that works with Alteryx Designer. So we’ll be going through some slides today and the first slide will introduce the breakout session. Connecting people to data. We sort of jokingly refer to it as API Palooza, we’re connecting any API to Alteryx in minutes.

What is Precog for Alteryx?

So what is Precog for Alteryx? Precog is an Alteryx technology partner. We provide an easy to use no code tool for connecting APIs and other data sources directly to Designer. It is an on-premise or cloud solution to meet whatever needs you have. You’re not just limited to cloud, and it allows you to load data directly to Alteryx via a free macro that is provided in the Alteryx gallery. So how do customers use Precog for Alteryx today?

Use it to access new data sources, especially APIs quickly and easily. So, we spend a lot of time with Alteryx users and we spent a lot of time in the Alteryx community and we see lots of requests for people that want to get to new SAS APIs, maybe HR applications, or other business applications, as well as public APIs, like or other sources like that. And they want to be able to easily add that data into their workflows. We provide a no-code tool that allows you to do that very quickly. All of these connectors that you create in Precog to these different data sources, including APIs are all high performance, they’re all streaming, they’re all fast, and they allow you to automatically keep the data in those workflows up to date.

So Precog is often referred to by our customers as the connector factory. So it allows you a very scalable, manageable way to create consistent integrations with any data source, including APIs, SQL databases, obviously relational databases, IOT streams, Kafka streams, and more. So, let’s talk for a minute about why people use Precog for Alteryx.

All sources, one solution

It’s a single tool that handles the entire connector process. So it’s a live connection to the data that handles authentication, handles pagination, including complex pagination metaphors that you see in many APIs, it handles throttling and it does all of this through a simple GUI, just checking boxes. Next and even more importantly, is it handles all the parsing and the structure of the data. So many of the people listening to this session have probably worked with API data, recognizing that it typically hands you JSON, or sometimes XML, but any kind of semi-structured data. And you need to parse that out and get that into a nice clean form that you can use in your Alteryx workflow.

Precog does that automatically. So as soon as we receive that JSON response from the API, we automatically parse that into what we call sensible tables. And then as I mentioned, it allows you to stream the data directly into Alteryx via the macro. So what does that look like in practice? Well, you can create any new API integration or connector if you will in a few hours. Most of the work is actually just reading the API documentation from the vendor. Precog will automatically create SQL ready tables, it will correctly tabulate those into sensible tables. We’ll never try to create a table with a thousand columns or something that’s not going to work in Designer or any other SQL tool.

It gets all of your custom fields from apps like Salesforce, HubSpot, and any other API or SAS app that allows you to customize it. This is very unique. A lot of tools don’t allow you to easily get custom fields that are working towards some sort of default schema. If you’re trying to use it, maybe an ODBC driver or something. So we allow you to automatically get all of your custom fields and then Precog will also discover new fields as they’re added. So when you add new fields to your application, your SAS application or API, we’ll discover those and incorporate those into the schema and into the tables that we load into Designer.

And it’s much faster and more accurate than driver-based approaches. As I mentioned, when you buy some sort of commercial ODBC driver, it’s typically written to the generic schema of the application, and if you’ve customized it at all, like people commonly do with things like Salesforce, it won’t pick those up, whereas Precog will always pick those up automatically.

Precog provides a consistent experience for accessing ALL data

So what are some more reasons why people today use Precog for Alteryx? It provides a consistent experience and performance across all data structures. As I mentioned, it is a no-code tool. It’s a purely visual tool, but when you create these connections in Precog, these API connections, they will always be consistent, high-performance streaming live connections. We’re really focused on empowering end users to get the data when they need it and not have to go through all the complicated steps sometimes required to be able to access new data sources in Alteryx.

It’s very cost-effective and delivers immediate value for Alteryx users. Today, Precog literally supports over six thousand data sources with more being added all the time, not only by us, but by our customers, since you can make your own connectors. And then as I mentioned before, it provides consistent performance, manageable scale or platform based approach to creating these new connections, very end-user friendly. And then when the data source changes, we allow you to update the connector very simply in a few clicks.

Let’s just take a minute and talk about kind of, again, what’s under the hood in terms of a Precog API connector. So broadly there’s sort of two things that we have helped automate or make much more simple. The first one is just a physical way of connecting to that data source and being able to read the information from it. So that’s things like authentication, the request where you get post/put, a pagination, including what they call multi step requests in pagination, and then throttling as well. And we also have the ability to pass variables between end points, which is something that you see in more advanced APIs, which again is very difficult to do in existing tooling.

And then the second part is we handle all of the data restructuring automatically. So when we’re presented with the JSON data from the API, or in some cases, maybe XML data, we’re able to easily parse that data into sensible tables, including any level of nesting or any sort of complexity in there. We can handle highly heterogeneous data, very large object sizes, all of those things. And we can transform those quickly and efficiently into the tables that you need to be able to flow into your Alteryx Designer workflow.

So let’s take a moment and see what that looks like graphically. So this is sort of a simple example, lots of data sources in the upper left, and then Precog loads those into Alteryx in the lower right. So what are some of the characteristics? Any data source, regardless of size or complexity, we have built in MTLS support. So that’s something that more and more of your regulated customers are probably asking for and so we support that out of the box in Precog. It’s always a no-code user experience. It allows you to load data from any API in minutes. It’s very fast, with loaded gigabit speed. And then as I mentioned before, public cloud, private cloud or on-prem option, so your customers can implement this how it makes the most sense for them. They’re not going to need to be relying on a cloud only solution that we might provide. They can implement it, how they need to implement it.

Precog for Alteryx: easier, faster, simpler

So in summary, Precog is all about helping you get self-service data access for your Designer implementation and make that easier, faster, simpler, and more consistent. So we break it down into four areas, consistency, performance, extensibility, and integration, and those are the four legs of our stool or chair that we provided Precog and make it very simple for you all to get more data into Alteryx very quickly.

So that’s the summary of Precog for Alteryx. As I mentioned, we’re a current Alteryx technology partner, we work very closely with them. We’re thrilled to be here working with you at Inspire 2021. And we encourage you to reach out to us with any questions. As I mentioned, my name is Jeff Carr. It’s just [email protected]. You’re welcome to reach out to me directly with any questions. And certainly we’ll have an opportunity at the end of this session to do Q and A with people virtually attending the session. Thank you for your time today.

Becky Conning

Thanks Jeff. Alteryx gives us unprecedented capabilities to cleanse, blend and predict with data. But data like this open FDA, web API data often comes in the form of complicated JSON and needs to be retrieved from systems like cloud storage, web APIs, no SQL databases. As we can see from this workflow, there’s quite a bit of complexity just getting to the data in the first place, dealing with pagination and nesting in the data that gets returned. Wouldn’t it be great if we had more time to exploit the full potential of Alteryx, and could spend less time working with the plumbing of how to get the data in the first place? Precog gives you this superpower. Let’s grab the same URL and jump in with Precog.

Getting started with Precog

So you can see here are a number of preconfigured sources. These are super easy to use and many of them were provided by members of our community. It’s very straightforward to configure a new web source. And once it’s configured, you can export it and share it with others. We’ve just created a new web source and we’re just going to paste in that URL from before. But instead of having to use a macro to iteratively work over each page of the data, we can simply select the type of pagination that is required by the API. Now that we’ve got access to the raw data, Precog can infer the relational schema from the multidimensional mess of data.

We can bring these tables just as they are into our Alteryx workflows, and we can join them together or do other operations as we please. But Precog provides another capability that allows us to select a number of tables and automatically join them together. This means we don’t have to worry about join conditions, we automatically get the data in the format that we want, and we get dramatic performance improvements due to Precog’s streaming evaluation systems. Precog can load these tables into a database or warehouse and Alteryx can connect to that or we can load the data in a streaming manner directly into our Alteryx workflows. To do that, generate an access link, drop a Precog macro onto your workflow and paste the link.

Precog is now retrieving the data chunk by chunk from the open FDA web API, transforming the data into the sensible table that we want and streaming this into your Alteryx workflow without holding onto any of the data at rest.

Precog supercharges your Alteryx Workflows

Now that we have the data, we can select the fields that we want, and we have the same result as before, but with far less work, far less maintenance and way preferable performance. We’ve simplified and supercharged our workflow and we’re ready to blend, predict and cleanse. Precog is taking care bringing the data from this web API into the workflow so you can focus on what’s next and it gets even easier when you’re using the preconfigured sources, whether it’s Zoho Inventory, HubSpot or Shopify, as you’ll see over the next few minutes, we can bring these data sets into our Alteryx workflow rapidly and without any complex decisions to be made. All we have to do is select the primary key.

These pre-configured connectors are not black boxes. You can change the requests that have been made, including the hosting. You can change the type of authentication that’s being performed. You can even take data from one endpoint and provide it to another, perhaps to get more detailed information about a listing of records. And Precog of course can provide Delta loading to give you just information that’s changed since the last time you loaded.

We’re so excited to see how you unleash the full power of Alteryx on thousands of data sources, as well as supercharging and simplifying your existing workflows. And next month, when a hot new SAS application hits the market, you can be the first to bring data and value from that application into your Alteryx workflows with Precog.

All sources in minutes

Over the last few minutes, we’ve provided authentication information for Zoho Inventory, Shopify and HubSpot to Precog and selected the data sets that we would like to load. We’re now selecting the resulting tables that we want to bring into our Alteryx workflow. Repeating the process that we saw earlier, where we copy the access link, drag a Precog macro to our workflow and paste in that access link. That data is now loaded chunk by chunk being transformed in a streaming manner, loaded into our workflow from those three APIs into one single Alteryx workflow. And we set this whole thing up in just a matter of minutes. So you can see here, we’ve got demo data from these separate systems, Shopify orders and HubSpot form submissions. That’s the power of Precog and Alteryx together.


Ready to Start?



We chose to use Precog because they were the only company willing to handle our complex data connections. Precog was extremely helpful getting us set up and running smoothly. Since then it has been one of those tools that just works solidly and reliably which is one less thing our team nee... Read More

Derek Binkley - Engineering Manager

Precog is an important partner for Cured and a critical member of our data stack. The Precog platform has delivered data connectors to necessary data sources other vendors could not or would not, and in a very short timeframe. The product is intuitive, efficient, cost-effective, and doesn&... Read More

Ashmer Aslam - CEO Cured
Walnut St. Labs

Precog lets us prototype analytics projects quickly — building marketing dashboards based on data from a variety of sources — without needing a data engineer or developer — we create new data sources in a few hours to sources like Brightlocal, a popular local SEO SaaS solution, and h... Read More

Chris Dima - CEO

We welcome Precog to the Alteryx technology partner ecosystem as a partner extending the capabilities of our platform, further simplifying analytics for our customers.

Hakan Soderbom - Director of Technology Alliances

We recognized a need in our customer base to perform advanced analytics on SAP data sets — we performed an extensive evaluation of Precog and chose it as a strategic solution for our go to market needs based on its performance and given their strong strategic relationship with SAP.

Alfredo Poncio - CEO

Precog is the vital tool in our ability to pull data from a variety of business sources quickly and cleanly. Our internal MongoDB backend, as well as other cloud services like Hubspot, were a constant challenge to the business teams desire for reporting data prior to using Precog. With the... Read More

Josh Wilsie - VP