Fast, Flexible & Agile Analytics for MongoDB
How Veho moved beyond
the limitations of the BI connector
Startup business Veho relies upon a crowd-sourced labor pool of drivers using their own vehicles to cover the “last mile” for package delivery. On the surface, an approach that is similar to Uber and Lyft in the ridesharing industry will enable Veho to upend the package delivery industry by improving the customer experience at a much-reduced cost.
Data includes facilities, facility types, locations, street addresses, access information, time zones, indoor navigation, area coverage by zip code, facility capacity for drivers and vehicles, delivery jobs, job creation and expiration dates and times, delivery priorities and failures, repeated attempts, progress and status, messages, routes, directions, drop off locations, estimated payout, package ID, package delivery window, recipient signatures, and so much more.
Started in 2016 and the subject of a Harvard Business School graduate project in 2017, Veho now operates in Denver, Chicago, and Dallas with future expansion plans. Under the hood, the critical element for introducing change in the industry is the application and the data. Using GPS and mobile phone technology, Veho shares the data that enables non-professional drivers to deliver the high levels of service required to meet their customer experience expectations.
Veho data is very dynamic and covers various data types from multiple sources.
- Drivers may have 15-20 deliveries. The route order and related directions will change as deliveries are
added or subtracted.
- Packages are loaded in delivery vehicles based upon stop order. Packages are labeled/identified and tracked based on the sequence.
- Notes can be added to customer preferences for delivery. Notes can also be added to explain challenges a driver may have in completing a delivery.
- Map coordinates (latitude and longitude) are provided for pinpoint location accuracy. In addition, drivers upload photos of delivered packages for validation.
Due to the dynamic nature of the data and because of the need to have very efficient exchange of the data, Veho chose to use MongoDB Atlas (Cloud managed in AWS) as the database for their application.
Data and Analytics Challenges
A critical element of Veho’s business is being able to provide up to the minute reporting on the status of deliveries. This means they need to be able to access their application data easily and quickly to generate reports. Since their application uses MongoDB this is a challenge. MongoDB is a document database that uses JSON documents in order to store records, just as tables and rows store records in a relational database. In the document data model, each record and its associated data is considered a “document”. In other words, everything related to a database object is packaged together. This semi-structured format, that packages data more naturally and logically using keys and values moves away from the restrictions of traditional SQL database schemas. Business intelligence tools in the marketplace today rely on a traditional, SQL-structured format in order to use data for analysis and reporting. As fantastic as JSON is for efficient, painless data exchange, it can be equally frustrating to transform into a useful format for popular business intelligence and analytics tools. Initially, Veho tried using the MongoDB BI Connector. They were connecting it to Tableau desktop for business analysis and to generate critical customer reports. The experience proved to be slow and inefficient as Veho analysts waited for their MongoDB JSON data that has multiple levels of nesting to be “unwrapped” so they could review the data in Tableau. And as their business grew and data volumes became larger the BI connector was not meeting their needs. It became clear this approach would not meet the requirements for efficient use of the data as the company founders had envisioned.
Finding the Right Solution
WAs a recent graduate of Techstars Boulder, Veho was put in touch with Precog through mutual connections. Making their home in the heart of downtown Boulder Colorado, Precog has productized data ingest and transformation for analytics, visualization, data science and machine learning. The product, Precog, is scalable but most importantly, it enables an analyst to create the tables they want without having to code or go through extensive data extraction and transformation procedures.
As Precog CEO Jeff Carr explains,
“Precog was built from the ground up to handle any data, regardless of size, complexity or structure, and make it fast and easy to build the exact tables you need for any analytics. Unlike the BI Connector or any driver based approach, Precog doesn’t try to treat MongoDB JSON like a relational DB.”
Through a simple connection built into the browserbased user interface, Veho was able to connect to their Atlas MongoDB data source and immediately begin to view their data in a logical and organized manner. Keys and values that were of interest could be selected by a simple point and click of a mouse, creating columns for a curated, tabularized dataset that the analyst would use for their work and reporting. Once the tables were defined the entire process is automated to move the data into their Metabase BI solution. Updating tables and adding new tables takes minutes and can be done by anyone, not just a MongoDB DBA, and the performance is much faster than the MongoDB BI Connector.
In the words of Fred Cook, Cofounder and CTO at Veho,
“Precog lets us very quickly build complicated dashboards and BI queries without being constrained by our mongodb schema. The Precog team provided expert support in recommending how to fit Precog into our BI system and helped with every step of the setup.”
Precog for MongoDB to the Rescue!
A smartphone application-based, package delivery business working with a document database makes sense. It is said that the last 5 miles of delivering packages that have been shipped across the country is 70% of the process. In a manner similar to what Uber and Lyft have done with the ride sharing industry, Veho has a vision to radically change package delivery. Toward that end, efficient interexchange and use of data that is generated from thousands of sources and that is rapidly changing is a key characteristic of the Veho business model. Precog for MongoDB is a tool that empowers end users to get the data they want in the form they need in minutes. All this means better customer satisfaction and the ability for Veho to keep growing.
Veho was able to replace the MongoDB BI Connector with Precog for MongoDB and increase the speed and flexibility of their analytics instantly. As part of the Veho information technology environment, Precog automatically sources MongoDB data and pushes curated tables to a Postgres database in a public cloud infrastructure. Analysts then use Metabase for business intelligence to power the reports their customers need.