As someone who leads a Data Engineering and Data Science team and has for 15+ years, this is exactly the problem. Too many folks with access to data who do not understand the data, its relationships, and what the outputs their individual efforts create mean.
Decentralized/embedded Engineers/Scientists; self-service dashboards; low-code BI/data tooling; and, now, LLM-driven text to SQL/viz lipstick on a pig have been floated as some of the solves to the problems seen in the analytics space over the 25 years I've worked in the space. Unfortunately, to date, nothing has actually solved the root issue: lack of data understanding and, its end result, trust in the deliverables.
But, to your specific point, SQL isn't the solve here, either. Too many folks know enough SQL to pull data and use it as they see fit, but too few folks understand the data, its structure/schemas, and valid use of those data. THAT requires time, energy, knowledge, and experience in the space. NO TOOLING, other than experience, solves for this--today (note: will LLMs get to a place where they can? Maybe; but, let's be honest--probably not).
Dashboards are great at giving quick hit information of KPIs and the ability to drill down into them; but, the most important thing to solve are always:
1. Data Management practices
2. Understanding of data, its relationships, and proper use of those data/metrics in deriving insight to drive the business forward.
I am excited to see what the future holds, but my grey beard doesn't allow me to ever, Ever, EVER trust any next-gen tooling being it hasn't held true to date.
I'm a SRE and have used my fair share of BI tooling. I have a rule that if the dashboard has a sufficient number of consumers/users then it should actually be in an internal application. BI tools are for me to play and prototype with data, they don't produce products in themselves. Much of that has to do with what you mentioned here, the other half is that I will absolutely never devote another part of my life to debugging some dashboard query I made six months ago.
I am data science manager at my own startup. Everywhere I go, I tell people proudly, I am ardently and stridently against data democratization.
I honestly will prefer people make decisions with their guts than with ill-understood data. Instead what I get is people who don't understand the data, its context, its meanings (and what it doesn't mean) trying to lead people down the wrong road while using the data as crutch. It is so frustrating to me.
Intuition of folks with good understanding of the business, especially when their salaries depend on it is often a much much better compass than some rubbish someone is claiming data is saying.
Another term I hate with a passion "data driven". No! data drives nothing. "Data informed" is where its at. You take the data, our best understanding, mix it our understanding of things the data doesn't cover, and use to that inform the best decisions we can make.
Decentralized/embedded Engineers/Scientists; self-service dashboards; low-code BI/data tooling; and, now, LLM-driven text to SQL/viz lipstick on a pig have been floated as some of the solves to the problems seen in the analytics space over the 25 years I've worked in the space. Unfortunately, to date, nothing has actually solved the root issue: lack of data understanding and, its end result, trust in the deliverables.
But, to your specific point, SQL isn't the solve here, either. Too many folks know enough SQL to pull data and use it as they see fit, but too few folks understand the data, its structure/schemas, and valid use of those data. THAT requires time, energy, knowledge, and experience in the space. NO TOOLING, other than experience, solves for this--today (note: will LLMs get to a place where they can? Maybe; but, let's be honest--probably not).
Dashboards are great at giving quick hit information of KPIs and the ability to drill down into them; but, the most important thing to solve are always:
1. Data Management practices
2. Understanding of data, its relationships, and proper use of those data/metrics in deriving insight to drive the business forward.
I am excited to see what the future holds, but my grey beard doesn't allow me to ever, Ever, EVER trust any next-gen tooling being it hasn't held true to date.