Features
dbt source freshness validates that raw tables are loaded within an SLA by comparing the latest timestamp to thresholds you define.
dbt source freshness tests help teams guarantee that raw data arrives on time by flagging sources whose latest loaded timestamp exceeds a configured SLA.
dbt source freshness is a built-in test that queries a source table’s loaded_at_field, calculates its age versus now(), and marks the source as pass, warn, or error based on your SLA.
Stale data produces misleading dashboards, lost trust, and costly reruns. Source freshness offers an early-warning system that catches upstream ETL issues before they ripple through models.
During dbt source freshness or dbt build, dbt selects max(loaded_at_field) from each defined table, converts it to UTC, then compares the result against warn_after and error_after intervals.
sources.yml configurationsources:
- name: stripe
freshness:
warn_after: {count: 30, period: minute}
error_after: {count: 60, period: minute}
tables:
- name: charges
loaded_at_field: created
Specify loaded_at_field for each table and include freshness thresholds at the source or table level. Keep warn_after looser than error_after to avoid false alarms.
Store timestamps in UTC, align SLAs with business requirements, run freshness checks on every CI deploy, and export the JSON artifact to observability tools like Grafana or Slack alerts.
Results appear in the CLI, target/freshness.json, and the dbt docs HTML site. CI pipelines can parse the JSON to fail builds on error status.
Yes. Execute dbt source freshness in Galaxy’s terminal or as a scheduled workflow, then open the generated artifact within Galaxy. The AI copilot summarizes failing sources for quicker triage.
Fresh data is the foundation of trustworthy analytics. dbt source freshness enforces ingestion SLAs, ensuring that dashboards and machine-learning features rely on timely data. By catching upstream delays early, teams prevent cascading model failures and avoid costly backfills.
Run it at least as frequently as your strictest `error_after` interval—hourly for hourly SLAs, daily for daily loads.
Yes. Add the flag `--select freshness:true` or include freshness in your `build` job so tests execute alongside model builds.
Checks are lightweight single‐row queries, so impact is minimal. Large schemas with many sources may add seconds, not minutes.
Galaxy’s file browser and AI copilot let you open `target/freshness.json`, filter failing sources, and share summaries with teammates.
.avif){kind=logo}
