dbt
dbt-core & dbt Cloud — profiles.yml
dbt is a different shape of workload than a BI tool: it issues writes (CREATE TABLE / MERGE) alongside reads (tests, source freshness checks, documentation queries). Pointing dbt at the Gateway is straightforward; the interesting decision is which parts to cache and which to let pass through. Invalidation rules do the heavy lifting — DML from a dbt build automatically busts cached reads on the affected tables.
Should dbt go through the Gateway?
Three layers of dbt traffic, three different answers:
-
Reads dbt issues itself. Tests, source freshness
checks,
dbt show, documentation queries — yes, route these through the Gateway. They repeat across runs and benefit from caching with short TTLs. - Writes dbt issues. CREATE TABLE, MERGE, INSERT — yes, route them through the Gateway too, so invalidation rules can fire on DML and mark cached reads on those tables stale automatically.
- Downstream BI / serving queries. The whole point — aggregates and reports built on top of dbt models. Route them through the Gateway so cache rules pay off and invalidation rules tied to dbt DML keep them fresh.
Update profiles.yml
Edit ~/.dbt/profiles.yml (or the dbt Cloud connection
config) and replace the warehouse host with your gateway address.
my_project:
target: prod
outputs:
prod:
type: snowflake
account: your-account
user: dbt_runner
role: TRANSFORMER
warehouse: TRANSFORM_WH
database: ANALYTICS
schema: dbt_prod
my_project:
target: prod
outputs:
prod:
type: snowflake
account: your-slug.gateway.airbrx.ai
user: dbt_runner
role: TRANSFORMER
warehouse: TRANSFORM_WH
database: ANALYTICS
schema: dbt_prod
For Databricks, change host: to your gateway address and
leave http_path: alone — the Gateway forwards that part
unchanged.
Authentication
Use whatever credentials your dbt service runner used before — typically a service-account user with key-pair auth (Snowflake) or a Personal Access Token (Databricks). The Gateway forwards them; no Airbrx-issued credential is involved unless you choose to use one.
Pair with invalidation rules
This is where dbt + Airbrx earns its keep. When a dbt run lands new data in a table, you want every cached read on that table to refresh on the next request — not wait out a one-hour TTL. An invalidation rule says exactly that: "on INSERT / MERGE / UPDATE / DELETE on table X, invalidate any cache rule keyed by table X."
Rules of thumb:
- Per-table invalidation rules for any table dbt writes to — the rule fires when the dbt build's MERGE lands.
- Cache rules with shorter TTLs on the read side, since you have automatic invalidation as the safety net. TTL becomes a bound-of-staleness, not the primary freshness mechanism.
- Out-of-band updates — if data lands in a table from something other than dbt (Fivetran, an ingestion job), fire a manual marker via the markers API to keep cached reads fresh.
Verify the cache is doing work
dbt's own logs don't surface response headers, so use the App's traffic page filtered by the dbt service-account user. You'll see the typical dbt fingerprint — many short tests, repeated source freshness queries, the occasional MERGE — and which of them are hitting cache.
Notes worth knowing
-
Adapter compatibility. The Gateway speaks Snowflake
and Databricks wire protocols. dbt's
dbt-snowflakeanddbt-databricksadapters work with no swap. - dbt Cloud. Same idea — change the host on the connection in the project settings.
- Compile / parse. Those phases don't touch the warehouse, so the Gateway isn't involved.
- Long-running models. For a model that takes minutes, there's nothing for the Gateway to do — that's warehouse work, by design. The cache pays off on everything downstream of the model.
Where to go next
- Rules as the differentiator — how cache and invalidation rules pair to handle dbt's read+write mix.
- Markers API — fire invalidation markers from non-dbt ingestion paths.
- Other connection recipes.
Ready to point dbt at a gateway address?
Airbrx is in private preview. Create an account and we'll set you up.
Create an account