DS9.
Data and reporting

Leaving proprietary BI without losing the data

Mapping the flows inside Qlik and building an independent data layer on ClickHouse that can be queried in plain words.

Client
retail group with central purchasing
Status
Pilot · 2026
Outcome
  • The original BI data flows are mapped and documented, including business definitions.
  • Raw import and incremental partition publishing are deployed.
  • Data is available outside the original tool, so migration is no longer tied to one vendor.
Stack
ClickHousePythonPowerShellQlik CLI and QVDMS SQL ServerFirebirdsystemd timer
ORDIS Firebird 2.5 COS MS SQL Server PROFIT MS SQL Server Read-only layer named queries JSON over HTTP audit and timeouts writing to the sources is impossible Reporting statements and exports Language model questions in plain words
Figure. The same principle as the reporting layer: the sources stay untouched, the new layer only reads them.

The problem

Proprietary BI is comfortable until you decide to leave. That is when you find that the business logic lives not in the database but in the tool's transformation scripts, that metric definitions exist only in the analysts' heads, and that exporting data is not the same as exporting meaning.

The customer needed two things at once: to stop depending on licences, and to make the data available to a language model so that sales, stock, margin and promotions could be queried in plain words.

What was built

The first step was not technical. It was mapping the flows: what is taken from where, where it is transformed, and what each metric actually means. Without that part, a data migration is a transfer of numbers stripped of meaning.

Then came an independent data layer on ClickHouse. Loading happens in two stages: a raw import from the sources, then publication into final partitions. Publication is atomic, so a consumer never sees a half-loaded partition. Every run carries a fingerprint of its inputs and a checkpoint, so after a failure it resumes where it stopped rather than starting over.

On top of the finished layer sits an interface for a language model built on the same principle as the reporting layer: the model gets a bounded set of questions, not database access.

This project is at pilot stage, not full production. Raw import and incremental publishing are running; parity with the original data sets and the language model layer are the next milestones. We state it this way deliberately — a BI migration sold as finished does not thereby become finished.

Result

The customer now has their data on an open platform, documented definitions, and a migration path that does not depend on renewing a licence.