VectiQ Data Lake - Lakehouse, BI & Vietnamese NL2SQL
A lakehouse in development: dbt models, Dagster schedules, Vietnamese NL2SQL and dataset profiling, being integrated into Bronze, Silver and Gold layers
Overview
Most companies we work with do not lack data; they keep it in several places that never meet - operational systems in PostgreSQL and MySQL, project budgets in Excel, AI results in another bucket - and the monthly report is assembled by hand.
VectiQ Data Lake is the lakehouse we are building to bring these together: raw records land as Parquet in S3-compatible object storage (Bronze), are cleaned and typed (Silver), and become business marts for finance, operations, sales and executive KPIs (Gold).
The components exist and are being integrated; the Bronze-to-Gold chain does not yet run end to end. dbt models clean, type and deduplicate records, hash personal fields and keep snapshots of customer history. Dagster schedules hourly ingestion and a nightly Gold build.
An NL2SQL service takes a question in Vietnamese, has an LLM write the SQL, and lets through only a single SELECT on Gold tables with a row limit before DuckDB runs it. A profiling module summarises any uploaded dataset in Vietnamese and flags outliers by 3-sigma and IQR. BI dashboards, a data catalog with lineage and a feature store for our AI products are later steps.