Alteryx Macro for Massive Complex Data Sets
By Nic Flores
January 1, 1970

Introduction

Are you trying to work with huge complex data sets in Alteryx Designer? Would you like to? Would you like to increase processing performance of Designer by 100x for many data sets? Would you like to be able to connect to live data sources like API’s or MongoDB?? Then the Precog Macro for Alteryx Designer can help.

Precog is an Alteryx Partner solution which

  • Simplifies and speeds up workflows that deal with large, complex and nested data
  • Prevents data silos
  • Automatically keeps workflows up to date with the latest data
  • Empowers Alteryx Designer users to easily fulfill their own data requests from APIs, S3, Azure Blob Storage, Data lakes, Operational databases, MongoDB and more
  • Connects with your workflows via a simple tool available on the Alteryx Gallery

Precog allows Alteryx Designer users to build workflows that work with large, complex and nested data (such as JSON) in minutes. Modern data sources like Web APIs, IoT, Marketing, NoSQL and SaaS provide large amounts of JSON data. Precog allows data requests from such sources to be fulfilled in a self service manner in minutes. Simply connect to a data source, explore the data and pick what you?re interested in. Precog automatically transforms what you?ve picked into the table you want and makes it available in Alteryx Designer at hyper speed. The latest data direct from your data source is just a workflow run away. This makes it easy to keep your workflows up to date.?

Precog is simple to learn and use and there are no limits on data size or complexity. Precog works well with JSON objects, line delimited JSON, JSON arrays and CSV as well as making it easy to work with directories of millions of such files at once. Users can easily curate the exact tables they need and incorporate these into existing Alteryx Designer workflows to clean, blend and augment with other data. For example: you can rapidly clean and join data from API endpoints, MongoDB collections and relational databases.

Precog can crunch massively complex data sets in record time. Workflows which took hours to run with Alteryx Designer alone are ready in seconds with Precog.?

Start with Precog Explorer for free then level up to Precog Desktop or the scalable Precog Pro which connect to Alteryx Designer directly, providing access to live data from the sources listed above. Precog Pro and Precog Desktop connect to Alteryx Designer via the Precog macro which is available in the Alteryx Gallery and at https://staging-Precogmtm.kinsta.cloud/alteryx.

Precog Pro is scalable allowing your team to work with terabytes of data at once.

Once installed the following is an example of how to use Precog with Alteryx Designer:

Adding your data source

Browsing your data and picking what you?re interested in

Clicking ?Results?

In the ?Reusable access link? section clicking ?Generate link? then clicking ?Copy link?

Clicking ?In/Out? from the tool palette and dragging ?Precog? onto your workflow

Pasting the access link you copied into the ?Precog access link? field

Running your workflow

To produce the same results as above without Precog the workflow becomes complex and takes many hours to run. With Precog Desktop it runs in less than thirty seconds. The data used in this example is available here https://Precog-marketing.s3.amazonaws.com/daily-stats.json

Below is an example of using the same dataset with the ?Tabulate JSON? macro from the Alteryx Gallery. The workflow runs for 5 minutes before running out of memory on a machine with 32GB of RAM. Yet another example of how adding Precog adds value and speed to your workflows.

Summary

Using Precog with Alteryx Designer opens up new data sources and makes it fast and easy to work with large complex data sources like API?s, JSON data stored in S3 or data from popular NoSQL databases such as MongoDB or DynamoDB. Precog provides results much faster and makes adding data and adjusting to changes simple and fast. Alteryx Designer is the most popular data prep tool available today and Precog makes it even better.

Request a trial license of Precog today at [email protected]

NEWS & BLOG