AI Governance & Observability Platform
Trust, Lineage & Observability for Data.
Enterprise AI-assisted governance platform demonstrating metadata management, trust scoring, observability, lineage analysis, AI-assisted incident summarisation, and dashboard health monitoring.
Problem
As organisations scale their data estate, no one can answer simple questions: Where did this metric come from? Can I trust this table? What broke this dashboard? Data governance and observability are usually manual, reactive and opaque.
Solution
This platform makes the data estate observable and trustworthy. It ingests metadata, builds a knowledge graph of lineage with NetworkX, scores each asset for trust, monitors dashboard health, and uses AI to summarise incidents into plain-language explanations for stakeholders.
Architecture
A Streamlit application orchestrates the experience. Metadata is pulled from Snowflake via SQL, lineage is modelled as a directed graph in NetworkX, a trust-scoring engine computes reliability signals, and an LLM layer generates incident summaries. Power BI surfaces executive-level health metrics.
Key Features
Challenges
Turning raw lineage into a graph that stayed readable at scale, and designing a trust score that combined freshness, completeness and usage signals into a single defensible number.
Lessons Learned
Governance only works when it is observable and explainable. Pairing graph analytics with AI summarisation made complex data problems understandable to non-technical stakeholders — the real unlock.
Future Improvements
- Real-time lineage capture via query log parsing
- Anomaly detection on data quality metrics
- Slack / Teams alerting with AI root-cause hints
