August 28, 2024
We have loads of dashboards at Spotify. Our Insight groups and analysts from throughout the corporate are always whipping up new dashboards for stakeholders and themselves, serving to reply these large data-driven questions daily. These dashboards sort out every thing from regularly used key metrics to exploratory insights and operational stories. It’s clear to us: dashboards assist us transfer quicker and keep data-informed.
In 2023, Spotifiers — largely knowledge scientists and the like — created greater than 4,900 dashboards in Tableau or Looker Studio. These dashboards have been utilized by greater than 6,000 Spotifiers in 2023. There isn’t any centralized crew at Spotify solely answerable for creating dashboards, however somewhat a vigorous and free market of dashboards — with the Analytics Platform crew offering the scaffolding for this market — consisting of dashboard producers and dashboard customers.
At the center of our knowledge visualization efforts, we lean on two principal instruments to amp up our dashboard sport: Tableau and Google’s Looker Studio. With these in our toolbox, we’ve constructed extra programs and frameworks to maximise our potential to distribute, share, and uncover insights like by no means earlier than.
Let’s speak Tableau
Tableau’s superior customization choices allow us to create detailed, visually beautiful dashboards that aren’t simply informative however can even present distinctive person experiences. We can craft a extremely particular expertise for each use case — whether or not which means distinctive drill-down capabilities or a customized chart sort. These dashboards are sometimes regarded as full-fledged inner merchandise, not simply knowledge artifacts, every with a definite person base with particular wants.
Next up: Looker Studio
Looker Studio, our fast and nimble sidekick, gives a seamless integration with Google merchandise that we usually use, like BigQuery. With Looker Studio, our customers can go from SQL to chart to dashboard in a heartbeat. It’s notably fashionable amongst our engineering and product groups for its simple, intuitive person expertise.
Both Tableau and Looker Studio are important to our technique, every providing distinctive benefits. Looker Studio excels in fast, easy-to-use visualizations, whereas Tableau supplies complicated and detailed dashboards for extra nuanced wants. This dual-tool strategy permits each Spotify worker to decide on the answer that most closely fits their particular necessities, and we offer directions for getting began with each instruments by means of Golden Path workouts.
To guarantee our most generally shared dashboards are appropriately and precisely interpreted — after which showcased throughout the group — we develop them according to particular requirements and practices.
It begins with dashboard manufacturing. Building a dashboard, whether or not that’s in Tableau or in Looker Studio, can contain loads of transferring elements. It requires particular expertise in knowledge visualization and storytelling — expertise that require experience and time that many people could not have. On high of that, Spotifiers trying to construct a dashboard are possible in numerous elements of the org and never all the time in communication or imposing the identical requirements. To alleviate high quality challenges with distributed dashboard design — at scale — we developed a Dashboard Quality Framework.
There are two elements to the Dashboard Quality Framework:
- Vital Signs: A set of computerized checks to make sure the dashboard is “alive and well.” These are enabled by the knowledge we get from API and logs and embody data on proprietor standing, dashboard description, utilizing LDAP teams for sharing, underlying knowledge endpoint lifecycle, and whether or not updates and extract refreshes have been made lately.
- Spicy Dashboard Design Checklist: A guidelines of information visualization and dashboard design greatest practices that would flip a vanilla report right into a “spicy” dashboard. The guidelines consists of objects associated to visible design, usability, insights, and belief. This is a handbook step that requires the human eye to find out. To allow this guidelines at scale, we’ve turned this right into a self-evaluating guidelines that flags the dashboard when efficiently accomplished.
Based on the outcomes of those measurements, a dashboard then receives certainly one of these high quality labels:
- Low — fails Vital Signs (no matter Spicy Design Checklist consequence)
- High — passes Vital Signs
- Golden — passes Vital Signs and Spicy Design Checklist
Ten % of eligible dashboards at Spotify have achieved Golden standing — and we attempt to maintain greater than 80% of all dashboards at High or above, informing dashboard house owners when their dashboards don’t meet the Quality Framework standards.
Below you will see the precise particulars of this framework — stealing is inspired!
To assist our Tableau customers, we’ve created a service that helps create and handle Tableau extracts. We’ve developed a SQL scheduling instrument at Spotify that helps us schedule and run batch knowledge workflows in Google BigQuery. We’ve enhanced this service to provide and publish Tableau .hyper recordsdata that can be utilized in dashboards on Tableau Cloud. These workflows run BigQuery SQL when upstream dependencies can be found and are sometimes quicker, can load extra knowledge, and are extra handy to edit because the SQL is in .yaml recordsdata as an alternative of embedded inside Tableau.
And to actually drive dwelling the concept dashboards are a product, we equip all dashboard house owners with statistics and insights about their very own dashboards — exhibiting variety of energetic customers, weekly retention, customers by org or job household, and a capability to simply electronic mail customers (assume model updates or deprecation bulletins). This additionally permits folks to contemplate if they need to market the dashboard to a brand new viewers, or if they need to deprecate them when utilization may be very low. It’s all about empowering dashboards house owners to make nice data-informed choices about their knowledge and dashboard merchandise.
Insightful and exquisite dashboards aren’t any good locked away — they must be accessible to stakeholders!
To present easy accessibility for dashboard customers, we developed the Dashboard Portal. Dashboard Portal is an inner website consisting of a searchable catalog of all printed dashboards at Spotify, each from Tableau and Looker Studio. It consists of methods to prepare and curate teams of dashboards and provides further context to dashboards for viewers.
The main options of Dashboard Portal embody:
- Search: Users can search by title, description, and area names to search out present dashboards. No extra “Do we have a dashboard about X?”!
- Curation: Dashboard authors can create curated collections of dashboards that turn out to be a go-to hub for his or her groups. Think “The Ads Dashboard Hub,” containing all of the dashboards the crew consider are essential for his or her stakeholders to pay attention to.
- Trust and Quality: The Dashboard Quality Framework labels are displayed as a badge on high of the dashboard thumbnails and on all dashboard pages, giving viewers a fast understanding of a dashboard’s high quality, and extra importantly, whether or not they can confidently belief it. We’re capable of highlight and draw consideration to our Golden dashboards — our cream of the crop! In doing this, we’ve seen that customers usually tend to view dashboards with a Golden label.
- Ubiquity: While Tableau and Looker Studio every have their very own discovery mechanisms, we’ve discovered that having a Dashboard Portal that surfaces content material from a number of instruments permits the person to not have to consider the place the dashboard lives, however somewhat simply discover it.
- Enhanced Context: Dashboards are embedded straight into the Portal and boosted with further context: dashboard proprietor, final knowledge refresh date, utilization stats, and underlying SQL queries.
Our sturdy strategy to dashboard creation and administration showcases our dedication to knowledge democratization and knowledgeable decision-making throughout the corporate. Our subtle instruments and frameworks be sure that each dashboard isn’t solely visually interesting but in addition a well-oiled machine delivering actionable insights.
Special because of Jacob Olsufka, Arielle Silverman, Pavlina Mitsou, Daniel Wolmerud, Alex Pavlov, Axel Örnefalk, Robert Hjortsmarker, and Abhishek Upadhyay.
Tags: Data, engineering tradition