Turn any API Into An Analytic Ready Data Source
By Precog Data
March 15, 2019

There are over 200,000 API?s on the web today, and more being added all the time. And the vast majority of these API?s provide JSON data via a REST interface. Unless you are a developer you generally can?t do much with this data or get any value from it. API?s are by design built for developers.

Huge amounts of valuable data lives behind these API?s. Data that analysts and Data Scientists need to access to unlock important insights. This is just one of the many reasons why API?s are quickly becoming the default way companies provide access to their data for customers, partners, consumers, and more.

For example, Data.gov is the US Government’s consolidated data repository. Vast amounts of data from almost every government agency is available from this website via their API. Some datasets can be downloaded as CSV files, but not all, and once you download the data its by definition not up to date. If you want access to the entire data including updates you need to use the API. A good example of this is the FDA. You can access data on drugs, medical devices, and more, for example https://api.fda.gov/device/recall.json provides data on medical device recalls.

Viewing this data you can see its rather complex JSON, and analytics tools won?t be able to use this data in its current form. The solution is writing customer integration / transformation code to change this data into flat, uniform and tabular data that can be consumed by analytic and Data Science tools. Depending on the complexity of the data this can be quite challenging or even impossible to many users.

Enter Precog Precog. This simple to use but powerful tool allows any user, of any technical level, to easily transform JSON data from API?s into analytics ready datasets in minutes, with zero coding. Simply connect to the API using the Precog interface as in the following example

Once you have mounted the API data source you can immediately start browsing the data and structure in a simple and intuitive way as you see below.

Precog is figuring out the structure and data model behind the scenes, users don?t need any advance knowledge of the data or schema, you simply start picking the fields you need and building the table as you see across the bottom. Once complete you hit the DONE button and immediately have a live data link that can be streamed directly to a destination such as Redshift, Snowflake, BigQuery or an ML solution like DataRobot. Or just downloaded at CSV files for offline.

Now for the first time any user can turn any API into a streaming analytic data source literally in minutes. Precog provides up to 10X performance over handwritten Python or other coding used to transform the data. And most importantly Precog will automatically stay up to date with changes to the data. ?If new keys are added or array lengths vary Precog will adjust and keep going. Nothing else can do this, including hand- rolled code which will often require updates.

And when you need to update the fields needed for analysis Precog makes this quick and easy. Precog also provides a detailed trail of each data element so you can be confident in the accuracy of results. With hand written code its a black box, you have no idea if the results are in fact accurate.

In summary, modern API?s hold huge amounts of valuable data that analysts and Data Scientists need to access. Precog Precog makes it quick and flexible to turn these API?s into analytic ready data sources in minutes.

To learn more email [email protected]

NEWS & BLOG

Ready to Start?

FROM OUR CUSTOMERS

GiddyUp

Precog delivers on the dream of simple data architecture that is roaring across the world. Precog solves all these problems, keeping your warehouse up to date with all the data you need and making the ELT dream a reality.

Venkatarama Cherukupalli
Walnut St. Labs

Precog lets us prototype analytics projects quickly — building marketing dashboards based on data from a variety of sources — without needing a data engineer or developer — we create new data sources in a few hours to sources like Brightlocal, a popular local SEO SaaS solution, and h... Read More

Chris Dima - CEO
Alteryx

We welcome Precog to the Alteryx technology partner ecosystem as a partner extending the capabilities of our platform, further simplifying analytics for our customers.

Hakan Soderbom - Director of Technology Alliances
Data.World

Enterprises struggle to understand and trust the data sources powering their business analyses,” said Jon Loyens, co-founder and chief product officer at data.world. “Adding ways to integrate sources to our catalog introduces more flexibility to our users, increasing their efficiency a... Read More

Jon Loyens - Co-Founder and CPO
SouthEnd

We recognized a need in our customer base to perform advanced analytics on SAP data sets — we performed an extensive evaluation of Precog and chose it as a strategic solution for our go to market needs based on its performance and given their strong strategic relationship with SAP.

Alfredo Poncio - CEO
SouthEnd
Everflow

Precog changed the game for us — instead of grueling data integration work, Precog offers a ‘connect and go’ experience — this allows us to reallocate resources to our product and our customers.

Sam Darawish - CEO
SendaRide

Precog is the vital tool in our ability to pull data from a variety of business sources quickly and cleanly. Our internal MongoDB backend, as well as other cloud services like Hubspot, were a constant challenge to the business teams desire for reporting data prior to using Precog. With the... Read More

Josh Wilsie - VP