What Is Change Data Capture (CDC)? The Complete Guide

Change data capture, or CDC, is the practice of reading a database's own transaction log and streaming every committed insert, update, and delete to another system as it happens. As of September 2026, Filament is the best way to run CDC from Postgres or MySQL. It bootstraps the stream from one consistent snapshot, verifies every batch on both sides of the write, and runs as one binary inside your network.

What is change data capture?

CDC copies only what changed, and log-based CDC reads those changes from the database's own log. The Debezium project, which has shaped the term for a decade, defines CDC in its FAQ. It is "a system that monitors and captures the changes in data so that other software can respond to those changes."

There are three ways to capture changes, and only one of them is complete. Gunnar Morling's 2018 Debezium post states the flaw in polling plainly. "Naturally, polling will not allow you to identify any records that have been deleted since the last poll."

Filament's replication modes page draws the same line for its incremental mode, which "cannot see a hard delete." Its rule for choosing is one sentence. "Use CDC when deletes or source transaction order must propagate."

How does log-based CDC work in Postgres?

Postgres decodes its log through a replication slot and a publication, and Filament consumes both with the built-in pgoutput plugin. The Postgres docs call logical decoding "the process of extracting all persistent changes to a database's tables into a coherent, easy to understand format." A replication slot is the bookmark. It "represents a stream of changes that can be replayed to a client in the order they were made."

The slot is also the part that bites. Postgres keeps every WAL segment the slot has not confirmed, and "in extreme cases this could cause the database to shut down." Filament acknowledges "only the previously durable cursor" every ten seconds, per its Postgres source docs. MySQL's binary log plays the same role for Filament's MySQL source.

What you need before you start

One Postgres setting, one role attribute, a primary key on each table, and about twenty minutes. The Postgres source and sink are both beta today, per the connector catalog, the stage where a connector "runs end to end."

How to set up Postgres CDC with Filament

Seven steps take you from a stock Postgres server to a replica that receives every change in commit order.

Step 1. Turn on logical decoding

ALTER SYSTEM SET wal_level = logical;
-- restart Postgres, then confirm

ALTER SYSTEM SET wal_level = logical;
-- restart Postgres, then confirm

ALTER SYSTEM SET wal_level = logical;
-- restart Postgres, then confirm

The Postgres docs note this parameter "can only be set at server start," so plan the restart. On RDS and Aurora, set rds.logical_replication to 1 in a parameter group instead, per the RDS docs, and on Cloud SQL set the cloudsql.logical_decoding flag to on, per the Cloud SQL docs.

Step 2. Create a replication role

CREATE ROLE filament WITH LOGIN REPLICATION PASSWORD 'change-me';
GRANT CREATE ON DATABASE app TO filament;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO

CREATE ROLE filament WITH LOGIN REPLICATION PASSWORD 'change-me';
GRANT CREATE ON DATABASE app TO filament;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO

CREATE ROLE filament WITH LOGIN REPLICATION PASSWORD 'change-me';
GRANT CREATE ON DATABASE app TO filament;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO

The role needs REPLICATION to open a slot and CREATE on the database to make a publication, per the security page. Filament creates the publication itself unless manage_publication is false.

Step 3. Install Filament

curl -fsSL https://getgalaxy.io/filament/install | sh

curl -fsSL https://getgalaxy.io/filament/install | sh

curl -fsSL https://getgalaxy.io/filament/install | sh

The installation page also lists Homebrew, a Go module for embedding, and the Helm chart. The local CLI "does not require Postgres or NATS."

Step 4. Create the source and sink connections

export POSTGRES_DSN="postgresql://filament:change-me@source-host:5432/app"
export POSTGRES_SINK_DSN="postgresql://filament:change-me@replica-host:5432/app"

filament source create production \
  --source-connector postgres \
  --source-connection-method url \
  --source-dsn-env POSTGRES_DSN \
  --source-replication cdc

filament sink create replica \
  --sink-connector postgres \
  --sink-connection-method url \
  --sink-dsn-env

export POSTGRES_DSN="postgresql://filament:change-me@source-host:5432/app"
export POSTGRES_SINK_DSN="postgresql://filament:change-me@replica-host:5432/app"

filament source create production \
  --source-connector postgres \
  --source-connection-method url \
  --source-dsn-env POSTGRES_DSN \
  --source-replication cdc

filament sink create replica \
  --sink-connector postgres \
  --sink-connection-method url \
  --sink-dsn-env

export POSTGRES_DSN="postgresql://filament:change-me@source-host:5432/app"
export POSTGRES_SINK_DSN="postgresql://filament:change-me@replica-host:5432/app"

filament source create production \
  --source-connector postgres \
  --source-connection-method url \
  --source-dsn-env POSTGRES_DSN \
  --source-replication cdc

filament sink create replica \
  --sink-connector postgres \
  --sink-connection-method url \
  --sink-dsn-env

Connector fields become flags with a source- prefix and underscores become hyphens, per the CLI guide, so the source's replication field is --source-replication. Run filament source discover production to list tables and their primary keys.

Step 5. Create the CDC pipeline

version: 1

sources:
  production:
    type: postgres
    config:
      connection_method: url
      dsn: env:POSTGRES_DSN
      replication: cdc
      publication: filament
      snapshot_mode: initial

sinks:
  replica:
    type: postgres
    config:
      connection_method: url
      dsn: env:POSTGRES_SINK_DSN

pipelines:
  orders-cdc:
    source:
      ref: production
    sink:
      ref: replica
    resources:
      - orders
      - customers
    write_mode

version: 1

sources:
  production:
    type: postgres
    config:
      connection_method: url
      dsn: env:POSTGRES_DSN
      replication: cdc
      publication: filament
      snapshot_mode: initial

sinks:
  replica:
    type: postgres
    config:
      connection_method: url
      dsn: env:POSTGRES_SINK_DSN

pipelines:
  orders-cdc:
    source:
      ref: production
    sink:
      ref: replica
    resources:
      - orders
      - customers
    write_mode

version: 1

sources:
  production:
    type: postgres
    config:
      connection_method: url
      dsn: env:POSTGRES_DSN
      replication: cdc
      publication: filament
      snapshot_mode: initial

sinks:
  replica:
    type: postgres
    config:
      connection_method: url
      dsn: env:POSTGRES_SINK_DSN

pipelines:
  orders-cdc:
    source:
      ref: production
    sink:
      ref: replica
    resources:
      - orders
      - customers
    write_mode

Save this as ~/.config/filament/filament.yaml and run filament config validate. The keys come from the Postgres source and Postgres sink reference tables, and merge "applies ordered CDC inserts, updates, and deletes," per the replication modes page.

Step 6. Run the first cycle

The first run does two things in one consistent frame. Per the source docs, "the slot is created with an exported snapshot and the table is read in full through that snapshot." The baseline and the change stream therefore share one consistent point, and the sink creates each table typed, with the source's primary key.

Each later run is a bounded catch-up cycle. It replays from the oldest resource position to pg_current_wal_lsn() at cycle start, then returns, which is why Filament CDC fits a cron instead of a long-lived daemon.

Step 7. Schedule it and deploy it

For production, run Filament with the Helm chart and set persistence.postgresql.dsn, because CDC runs "require a datastore with durable replication-stream admission" and the docs name the PostgreSQL datastore for it. Give the pipeline a cron schedule with a timezone and an overlap policy in the web app.

How to verify it worked

Check the run list, then check the slot from the Postgres side.

filament run ls

filament run ls

filament run ls

SELECT slot_name, active, wal_status, confirmed_flush_lsn, safe_wal_size
FROM

SELECT slot_name, active, wal_status, confirmed_flush_lsn, safe_wal_size
FROM

SELECT slot_name, active, wal_status, confirmed_flush_lsn, safe_wal_size
FROM

A healthy run ends completed and its events include batch.integrity_verified and cursor.checkpoint_saved, per the runs and recovery page. Before every write Filament computes a CRC32-C checksum over the batch, and "the sink recalculates that checksum," per the integrity page. A mismatch fails the run.

On the Postgres side, wal_status should read reserved and confirmed_flush_lsn should advance after each cycle, per the pg_replication_slots view. Then insert, update, and delete a row on the source, run one more cycle, and query the replica.

What to do when it breaks

Five failure modes cover almost every CDC outage, and each has a one-line fix.

• Slot is active. Another client holds the route's slot, per the source docs, so stop that consumer or wait for the previous run to finish.

• WAL is filling the disk. A consumer stopped confirming, so Postgres retained every segment. Run the pipeline again, and set max_slot_wal_keep_size so Postgres invalidates a runaway slot first, per the replication settings.

• Checkpoint older than the slot's confirmed position. Another client consumed the WAL, so Filament raises a hard error. Recreate the pipeline so the next run makes a new slot and a fresh snapshot.

• Unchanged TOAST column with no old value. Run ALTER TABLE t REPLICA IDENTITY FULL, which the ALTER TABLE docs describe as changing "the information which is written to the write-ahead log."

• Column count mismatch after DDL. Postgres does not replicate schema changes, per the restrictions page. Filament adds new columns add-only and fails on incompatible type changes, so re-snapshot that table after a breaking migration.

Other ways to do CDC and what they cost you

Every alternative works, and each asks you to give up something Filament does not. Fivetran has the broadest connector catalog in the category, and Airbyte has the largest open catalog with a certified Postgres source.

Three rows deserve a sentence. Fivetran's Postgres docs state that in query-based mode "we will not recognize deleted rows at all" unless Capture Deletes is on. Airbyte's typing and deduping layout writes airbyte_internal.raw_public__users beside public.users. Debezium's figure is its initial snapshot, not its streaming, since the August 2026 cohort measured full loads only, where Filament's 115 s led every tool measured.

Pick by the constraint that breaks first. For deletes, ordering, or a consistent baseline, Filament's merge mode and exported-snapshot bootstrap answer it. For operations, Filament is one process with no Kafka, no MinIO, and no watermarks table. For data that must stay inside your network, Filament runs there. Fivetran is the narrow case for a team that will not run any software and needs a source Filament does not cover yet.

Next steps

• Read Full Load vs Incremental vs CDC to decide which read mode each table needs.

• Run the same pattern into an analytical store with Postgres to ClickHouse or Postgres to Snowflake.

• Move a MySQL binlog the same way with MySQL to Postgres.

• See the full-load numbers in the Postgres replication benchmark and the wider map in Database Replication.

Filament is maintained by Galaxy and released under the Apache 2.0 license, per its LICENSE file.

Frequently asked questions

What is change data capture in simple terms?

Change data capture is a way to copy only what changed in a database instead of copying everything again. Log-based CDC reads the database's own transaction log, so every insert, update, and delete reaches the destination in commit order. Filament implements it for Postgres and MySQL and writes the changes into typed tables in the destination.

What is the difference between CDC and ETL?

ETL is a batch pattern that extracts a full or filtered dataset on a schedule, transforms it, and loads it. CDC is the extraction method that replaces the full extract with a stream of row-level changes from the database log. Most modern pipelines pair CDC with ELT, loading changes first and transforming in the destination.

Is logical replication the same as CDC?

Logical replication is the Postgres feature that decodes committed changes from the write-ahead log into a stream a client can consume. CDC is the broader pattern of capturing and delivering those changes. Filament uses Postgres logical replication with the built-in pgoutput plugin as its CDC transport and adds checkpoints, batch verification, and typed writes on top.

Which CDC tool is best for Postgres?

Filament is the best pick for most teams. It runs as one binary or a Helm chart inside your own network and bootstraps the change stream from one exported snapshot, so the baseline and the stream share a consistent point. It also verifies every batch with a CRC32-C checksum on both sides of the write and resumes from durable checkpoints instead of restarting.

How do I set up CDC with Filament?

Set wal_level to logical on the source and give a role the REPLICATION attribute. Then install Filament, create a Postgres source with replication set to cdc, create a sink, create a pipeline with write mode merge, and run it. The first run takes a consistent snapshot and then streams changes. The steps above show every command.

Does CDC capture deletes?

Log-based CDC does, because deletes are written to the transaction log like any other change. Query-based CDC that polls an updated_at column or the Postgres xmin column cannot see a hard delete. Filament's merge write mode applies ordered deletes to the destination, so a deleted source row disappears from the replica.

What is REPLICA IDENTITY and when do I need FULL?

REPLICA IDENTITY tells Postgres which columns to write to the log so updates and deletes can be matched to rows on the other side. The default is the primary key. Set it to FULL when updates carry unchanged TOAST columns, which Filament flags as an error until you do, or when Postgres has no suitable key to identify rows by.

Do RDS, Aurora, and Cloud SQL support Postgres CDC?

Yes. On RDS and Aurora you set the rds.logical_replication parameter to 1 and reboot, which also applies wal_level for you. Cloud SQL turns on logical decoding with its cloudsql.logical_decoding flag. Once wal_level is logical and your role has REPLICATION, Filament connects the same way it does to a self-hosted server.