The challenge of translating data from different sources, in different forms, into a single comprehensive language was exciting because it’s a common issue in IoT. We put our data modeling skills to work, and created a solution that helps support STEM education in renewable energy.
Business problem: Divergent data sources
Bonneville Environmental Foundation (BEF) is a nonprofit whose mission is to develop and support the use of renewable energy resources. One of their key projects delivers comprehensive education on renewable energy science and engineering to K–12 students.
The project relies on an Internet of Things (IoT) infrastructure that powers an educational web portal—a site that pulls live energy data from solar panels on over 100 buildings into an interface that students and teachers can use for learning.
But BEF had a conundrum: multiple data providers meant dealing with four different APIs, all of which formatted the information in different ways. Attempting to get the disparate data into a usable interface was a mess—so BEF called on Metal Toad for an innovative solution.
Technology solution: Unifying and displaying diverse data
We built an ETL (extract, transform, load) system that would resolve the discrepancies. After pulling data from the sensors and automatically testing it using Simpletest, the biggest challenge was transforming it. The four different data monitoring services (Power One, DECK, Tigo, and Locus) used a variety of units for the same measurements—kelvin versus centigrade, kilowatt hours versus joules. Our system automatically transforms all the heterogeneous information into a single common format, using data modeling to create a unified vision.
An interesting extra challenge came from the rate limits set by the data monitoring companies—each source had a strict limit on the rate at which data could be extracted. So we created a solution that pulls data in a carefully controlled, automated speed to keep the info flowing uninterrupted.
Once we’d extracted and transformed the data, we used Highcharts to load it into interactive vector-based charts. Students and teachers can easily manipulate and analyze the data as part of their STEM curriculum.
Impact: A powerful data source for STEM education
BEF now has a powerful, easy-to-use web interface with unified data from four different APIs being used at over 100 buildings. Educators and students in over 660 schools rely on the data and visualizations to power their STEM curricula—one of the most comprehensive renewable energy education programs in the nation.