JSON to Insights: More Details About Precog
By Jeff Carr
December 9, 2019

More details about Precog

Self-service and pre-preparation

Traditionally data is pre-prepared into a normalised schema in a warehouse for analysts. Precog speeds up both the compute and labour of this process as well as ensuring accuracy.

There is an alternative approach. Providing analysts with direct access to Precog enables them to fulfil their own data requests by picking the exact data they need for their analysis. This enables analysts to find the needle in the haystack, improves performance and eliminates the time and cost involved in preparing, updating and storing un-needed data.

Both approaches have benefits and you should choose the one that is right for your team. Precog will provide improved performance and ease of use, whether you choose to prepare the data in advance for your analysts or whether you give your analysts the self-service power to fulfil their own data requests.

Self-service and pre-preparation

Each report contains a list of drugs and a list of reactions. We picked data about reactions and drugs in the same table. As such, Precog provides us with a table where the number of rows is the number of reactions multiplied by the number of drugs.

Instead, we could pick one table for reactions and another for drugs. This alternative approach provides a normalised schema over the data which can then be joined together as needed. This is useful, however it pushes the responsibility for joins onto downstream software such as Power BI or Snowflake.

Precog is powered by proprietary technologies that make such joins orders of magnitude faster. Picking a table containing the information needed for your analysis can substantially improve query times and reduce compute and storage costs compared to the traditional approach.

Streaming, virtualisation and scale

Tables in Precog are streaming and virtualised. We can think of this as though the Precog table were a grocery list rather than the groceries themselves. We can use the same list in different stores the same way we can use the same Precog table with different datasets. We can also export tables and import them into different copies of Precog.

Precog does not load the whole dataset into memory, nor does it store the data at rest. Precog performs streaming selects, filters and tabulation at the byte level and streams the data to the destination. This approach reduces memory and storage requirements. Combined with Precog’s built in clustering, this makes scaling straightforward and efficient as well as enabling more efficient cloud solutions.

Previous page: Analysing the tabulated data with SQL

NEWS & BLOG

The Failed Promise of Cloud ETL

The Failed Promise of Cloud ETL

With the constant explosion of new data sources and applications, both public and private, the old way doesn’t scale, period. What does scale is Precog’s concept of “just-in-time data sources.”

read more

Ready to Start?

FROM OUR CUSTOMERS

GiddyUp

Precog delivers on the dream of simple data architecture that is roaring across the world. Precog solves all these problems, keeping your warehouse up to date with all the data you need and making the ELT dream a reality.

Venkatarama Cherukupalli
Walnut St. Labs

Precog lets us prototype analytics projects quickly — building marketing dashboards based on data from a variety of sources — without needing a data engineer or developer — we create new data sources in a few hours to sources like Brightlocal, a popular local SEO SaaS solution, and h... Read More

Chris Dima - CEO
Alteryx

We welcome Precog to the Alteryx technology partner ecosystem as a partner extending the capabilities of our platform, further simplifying analytics for our customers.

Hakan Soderbom - Director of Technology Alliances
Data.World

Enterprises struggle to understand and trust the data sources powering their business analyses,” said Jon Loyens, co-founder and chief product officer at data.world. “Adding ways to integrate sources to our catalog introduces more flexibility to our users, increasing their efficiency a... Read More

Jon Loyens - Co-Founder and CPO
SouthEnd

We recognized a need in our customer base to perform advanced analytics on SAP data sets — we performed an extensive evaluation of Precog and chose it as a strategic solution for our go to market needs based on its performance and given their strong strategic relationship with SAP.

Alfredo Poncio - CEO
SouthEnd
Everflow

Precog changed the game for us — instead of grueling data integration work, Precog offers a ‘connect and go’ experience — this allows us to reallocate resources to our product and our customers.

Sam Darawish - CEO
SendaRide

Precog is the vital tool in our ability to pull data from a variety of business sources quickly and cleanly. Our internal MongoDB backend, as well as other cloud services like Hubspot, were a constant challenge to the business teams desire for reporting data prior to using Precog. With the... Read More

Josh Wilsie - VP