Lightdash: Dashboards As Code & SQL Runner Chart Issues

Dec 4, 2025 by Alex Johnson 56 views

Lightdash: Addressing the Gap in Dashboards as Code Support for SQL Runner Charts

As data professionals increasingly adopt the practice of "Dashboards as Code" (DaC) for version control, collaboration, and automation, the nuances of various BI tools become crucial. This article delves into a specific challenge encountered within Lightdash, a popular open-source BI platform, concerning its DaC implementation and its interaction with SQL Runner charts. We'll explore the issue, its implications, and potential solutions, offering insights for both Lightdash users and the development team.

Understanding the Challenge: SQL Runner Charts and DaC in Lightdash

When implementing Dashboards as Code (DaC) within Lightdash, a discrepancy arises with SQL Runner charts. The core problem manifests when using lightdash download on a dashboard containing a SQL Runner chart. While the chart's reference appears in the dashboard's YAML configuration, a corresponding YAML file for the chart itself is absent. Subsequently, attempting to upload this downloaded configuration using lightdash upload results in failure due to the missing chart reference. This behavior creates a significant gap in the DaC workflow, hindering users from fully managing their dashboards through code.

This issue highlights a critical need for Lightdash to either bypass SQL Runner charts during the download/upload process or, more ideally, to fully support the download and upload of these charts. Addressing this limitation is essential for maintaining a consistent and reliable DaC experience within Lightdash.

Breaking Down the Technical Issue

To grasp the problem fully, let's dissect the technical aspects. The lightdash download command is designed to extract the configuration of dashboards and their components into YAML files. These files serve as a representation of the dashboard's structure and settings, allowing users to store them in version control systems like Git.

SQL Runner charts, however, present a unique challenge. These charts are created directly from SQL queries within Lightdash's SQL Runner, offering flexibility in data exploration and visualization. Unlike charts built using Lightdash's native chart editor, SQL Runner charts may not have a direct, one-to-one mapping to a YAML configuration. This difference is where the disconnect occurs: the dashboard YAML references the SQL Runner chart, but no corresponding YAML file exists for the chart's definition.

The Impact on Users and DaC Workflows

The absence of support for SQL Runner charts in Lightdash's DaC workflow significantly impacts users who rely on these charts for their data analysis and reporting. The inability to download and upload dashboards containing SQL Runner charts disrupts the seamless transition between development, testing, and production environments, a cornerstone of modern DaC practices.

Consider a scenario where a data analyst has meticulously crafted a dashboard with SQL Runner charts to monitor key performance indicators (KPIs). They intend to promote this dashboard from a development environment to a production environment using DaC. However, the lack of support for SQL Runner charts forces them to manually recreate these charts in the production environment, a time-consuming and error-prone process. This manual intervention defeats the purpose of DaC, which aims to automate and streamline such deployments.

Potential Solutions and Future Directions

Lightdash has two primary paths to address this issue:

Skipping SQL Runner Charts: A pragmatic, albeit less ideal, solution is to have lightdash download ignore SQL Runner charts altogether. This would prevent the upload failures caused by missing chart references. However, it would also mean that SQL Runner charts would not be managed as code, requiring manual intervention for deployment and updates.
Full Support for SQL Runner Charts: The optimal solution is to extend Lightdash's DaC capabilities to fully support SQL Runner charts. This would involve creating a mechanism to represent SQL Runner chart definitions in YAML, allowing them to be downloaded, uploaded, and version-controlled like any other dashboard component. This approach would provide a comprehensive DaC experience for Lightdash users.

Ideally, Lightdash should fully support the download and upload of SQL runner charts. This would involve creating a YAML representation for these charts, capturing their SQL queries, visualization settings, and any other relevant configurations. This comprehensive approach would ensure that all aspects of a dashboard, including SQL Runner charts, can be managed as code.

Steps to Reproduce the Issue: A Practical Guide

To illustrate the bug, let's outline the steps to reproduce it within Lightdash:

Create a Dashboard: Begin by creating a new dashboard in Lightdash.
Add a SQL Runner Chart: Within the dashboard, add a chart created using the SQL Runner. This involves writing a SQL query and selecting a visualization type.
Download the Dashboard: Use the lightdash download command to export the dashboard's configuration to YAML files.
Inspect the Output: Examine the downloaded YAML files. You'll notice that the dashboard YAML includes a reference to the SQL Runner chart, but there's no corresponding YAML file for the chart itself.
Upload the Dashboard: Attempt to upload the downloaded YAML files using the lightdash upload command.
Observe the Error: The upload process will fail due to the missing chart reference.

This step-by-step guide clearly demonstrates the issue and its impact on the DaC workflow.

Implications for Lightdash Users

The inability to fully manage dashboards with SQL Runner charts as code presents several challenges for Lightdash users:

Manual Intervention: Users are forced to manually recreate SQL Runner charts when deploying dashboards across different environments, increasing the risk of errors and inconsistencies.
Version Control Limitations: Changes to SQL Runner charts cannot be tracked and managed using version control systems, hindering collaboration and auditability.
Automation Barriers: Automating dashboard deployments becomes difficult, as SQL Runner charts require manual handling.

These limitations can significantly impact the efficiency and reliability of data workflows, especially in organizations that heavily rely on DaC practices.

A Call for Community Engagement and Collaboration

The Lightdash community plays a vital role in shaping the platform's future. Addressing the SQL Runner chart issue requires collaboration between users and the development team.

Users can contribute by:

Sharing Use Cases: Providing detailed examples of how they use SQL Runner charts and how the current limitation impacts their workflows.
Suggesting Solutions: Proposing potential approaches for representing SQL Runner charts in YAML and integrating them into the DaC workflow.
Testing and Feedback: Participating in testing new features and providing feedback to the Lightdash team.

The Lightdash team can foster this collaboration by:

Prioritizing the Issue: Acknowledging the importance of SQL Runner chart support in DaC and prioritizing its implementation.
Open Communication: Engaging with the community to discuss potential solutions and gather feedback.
Transparency: Providing updates on the progress of the issue and the planned approach.

By working together, the Lightdash community can overcome this challenge and enhance the platform's DaC capabilities.

The Path Forward: Enhancing Lightdash's DaC Capabilities

The discussion surrounding SQL Runner charts and DaC in Lightdash highlights the ongoing evolution of BI platforms and their integration with modern software development practices. As DaC becomes increasingly prevalent, BI tools must adapt to provide comprehensive support for managing all aspects of dashboards as code.

For Lightdash, addressing the SQL Runner chart issue is a crucial step towards achieving this goal. By either skipping these charts or, ideally, fully supporting their download and upload, Lightdash can significantly enhance its DaC capabilities and provide a more seamless experience for its users.

The ideal solution involves representing SQL Runner charts in YAML, capturing their SQL queries, visualization settings, and other relevant configurations. This comprehensive approach would ensure that all dashboard components can be managed as code, enabling version control, automation, and collaboration.

Conclusion: Embracing the Future of Dashboards as Code

The challenge with SQL Runner charts in Lightdash's DaC implementation presents an opportunity for growth and improvement. By addressing this issue, Lightdash can solidify its position as a leading open-source BI platform that embraces modern development practices.

As the data landscape continues to evolve, Dashboards as Code will become an increasingly critical capability for BI tools. Lightdash's commitment to addressing this issue demonstrates its dedication to providing a robust and user-friendly platform for data exploration and visualization.

By fostering community engagement, prioritizing user feedback, and implementing comprehensive solutions, Lightdash can empower data professionals to build, deploy, and manage their dashboards with greater efficiency and control. The future of DaC in Lightdash is bright, and the journey towards full support for SQL Runner charts is a significant step in the right direction.

For more information on Dashboards as Code and its best practices, you can visit reputable resources like the one available at dbt Labs.