MySQL to Apache Iceberg: Lakehouse Ingestion Guide

Getting MySQL into Apache Iceberg comes down to three questions. Must deletes arrive, which catalog does the lake already run, and who operates the pipeline? As of November 2026 twelve tools answer them across four methods. You can run a log-based engine such as Filament or OLake, stream through Flink CDC, buy a managed platform such as Fivetran or Estuary, or script a batch library such as dlt.

Apache Iceberg is an open table format that tracks Parquet files through metadata snapshots, so a folder in object storage behaves like a database table.

Every write to an Iceberg table adds a snapshot, which is why commit cadence matters as much as the capture method on this route.

What decides how MySQL lands in Iceberg?

Five criteria decide this route, and throughput is the one that needs a public run before anyone may claim it.

• Binlog capture decides whether deletes arrive, so the tool must read row events from the binary log, which MySQL 8.4 enables by default in ROW format.

• Catalog support matters because the lake’s engines already point at one catalog, so the writer must speak the REST catalog API, Glue, Nessie, Polaris, Hive, or a SQL catalog.

• Updates and deletes land as equality deletes on v2 tables, rewritten files, or v3 deletion vectors, and the Iceberg spec says that in v4 “Writing new equality deletes is no longer allowed”.

• Commit cadence sets the maintenance bill, because each write creates a snapshot and the maintenance docs warn that small files cause “an unnecessary amount of metadata and less efficient queries from file open costs”.

• Deployment weight and pricing unit close the list, from a single process to a JVM sidecar, a Flink cluster, or a hosted service billed by rows, gigabytes, or credits.

Tools that run MySQL to Iceberg

Twelve tools write MySQL into an Iceberg table you catalog yourself, and they split on whether capture is log-based and whether a cluster sits in between.

OLake is the tool built for exactly this route, and its README lists the widest catalog set on the page plus binlog capture for MySQL and MariaDB. The trade is a Java 17 writer beside the Go process, because the Go Iceberg library “doesn’t support Equality delete writes”, and Temporal in the Kubernetes install.

Debezium offers two Iceberg paths. Debezium Server runs the community-maintained memiiso sink without Kafka in upsert or append mode. The Apache Iceberg Kafka Connect sink adds exactly-once commits but needs Kafka, and its Debezium transform is marked experimental.

Flink CDC snapshots MySQL in parallel without the global read lock and writes Iceberg at-least-once through a pipeline connector that requires a primary key, a Flink cluster, and a Hadoop, Hive, or Glue catalog.

The managed tier splits on what you pay for. Fivetran’s Managed Data Lake Service writes Parquet with Iceberg metadata behind a Polaris-based catalog and bills monthly active rows above a free 500,000. Estuary’s Iceberg materialization merges through Spark on EMR Serverless and bills per gigabyte plus per connector. Airbyte’s S3 Data Lake destination is certified but still alpha, with Cloud from 20 dollars a month.

The code-first tools are cheapest to start and narrowest. dlt writes Iceberg through pyiceberg but its sql_database source has no MySQL binlog capture. Sling gates CDC behind a CLI Pro Max token, and ingestr streams the binlog under a Functional Source License that converts to Apache 2.0 later.

Six names are missing on purpose. PeerDB, AWS DMS, and Stitch have no Iceberg destination. Hevo writes only Snowflake-managed Iceberg, Databricks Lakeflow lands in Unity Catalog, and Striim writes Iceberg to Google Cloud Storage only.

Where Filament fits on MySQL to Iceberg

Filament meets the capture, catalog, delete, and deployment criteria outright and documents its own limits on commits. The MySQL source and the Iceberg sink are both listed out of preview in the connector overview, the flag the docs clear “after live validation confirms the supported behavior”. Filament is built by Galaxy, which also publishes this site.

On capture, the source page says a table with no CDC checkpoint is first read “in full through a consistent snapshot”. After that, “the change stream then continues from the binlog position captured just before that snapshot opened”. Read locks are held in a side session for an instant, and the stream tracks a GTID set when gtid_mode is on or the binlog file and position when it is not.

On catalogs, the sink writes “through a REST or iceberg-go-registered catalog”, with named providers for REST, Polaris, and Lakekeeper and a generic type for Glue, SQL, and Hive. Data files go to s3, gs, or azure storage through the catalog’s FileIO.

On deletes, the sink page lists full replace, append, and upsert, incremental upsert and delete, and CDC append and merge, where merge keeps the latest operation per key. Rows stay buffered as Arrow batches and become visible only when the table’s own commit succeeds. Before each write a CRC32-C checksum covers the batch’s buffers, counts, and operations, and the sink recomputes it at its final boundary.

On deployment, Filament is Apache 2.0, runs as one process for evaluation or on Kubernetes through the Helm chart, and embeds in Go. There is no meter.

The gaps are written down too. Atomicity is per table and not per run, so the sink page warns that “an earlier table can commit before a later table fails”. Schema evolution is add-only, CDC runs are bounded catch-up cycles rather than a continuous stream, the local SQLite context cannot run CDC, and the sink page documents no partition spec option.

No route gets a speed claim without a public run, and this route has one. The August 31, 2026 cohort loaded the 298 million row NYC taxi set from MySQL into an Iceberg REST catalog with Filament and OLake on the same host.

Filament finished the MySQL load in about half OLake’s wall time with a fraction of the CPU, and OLake finished the Postgres load 13 percent sooner. Both are full loads, and the methodology says results “do not predict incremental or CDC performance”.

Setting up MySQL to Iceberg with Filament

One replication user, one YAML file, and one command move a database. CDC adds two more commands. The MySQL blocks below are copied from the MySQL to Snowflake how-to, and the sink fields follow the Iceberg sink page.

Check the server first.

SHOW VARIABLES WHERE Variable_name IN
  ('log_bin', 'binlog_format', 'gtid_mode', 'binlog_expire_logs_seconds');

CREATE USER 'filament'@'%' IDENTIFIED BY 'change-me';
GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'filament'@'%';
GRANT SELECT, LOCK TABLES ON shop.* TO 'filament'@'%'

SHOW VARIABLES WHERE Variable_name IN
  ('log_bin', 'binlog_format', 'gtid_mode', 'binlog_expire_logs_seconds');

CREATE USER 'filament'@'%' IDENTIFIED BY 'change-me';
GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'filament'@'%';
GRANT SELECT, LOCK TABLES ON shop.* TO 'filament'@'%'

SHOW VARIABLES WHERE Variable_name IN
  ('log_bin', 'binlog_format', 'gtid_mode', 'binlog_expire_logs_seconds');

CREATE USER 'filament'@'%' IDENTIFIED BY 'change-me';
GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'filament'@'%';
GRANT SELECT, LOCK TABLES ON shop.* TO 'filament'@'%'

On Amazon RDS and Aurora the retention knob is mysql.rds_set_configuration('binlog retention hours', 24), and Airbyte’s docs note the RDS default is zero hours, so set it before the first run.

Declare the route in ~/.config/filament/filament.yaml, which the local context applies on change per the CLI guide. This is the full-load shape the benchmark used.

version: 1

sources:
  shop-mysql:
    type: mysql
    config:
      connection_method: url
      dsn: env:SHOP_MYSQL_DSN

sinks:
  lake:
    type: iceberg
    config:
      catalog:
        provider: rest
        uri: https://catalog.example.com/api/catalog
        warehouse: s3://shop-lake/warehouse
        auth:
          type: client_credentials
          client_id: env:ICEBERG_CLIENT_ID
          client_secret: env:ICEBERG_CLIENT_SECRET
        properties:
          s3.region: us-east-2
      namespace: shop

pipelines:
  shop-to-lake:
    source:
      ref: shop-mysql
    sink:
      ref: lake
    resources:
      - customers
      - orders
    sync_mode: full
    write_mode

version: 1

sources:
  shop-mysql:
    type: mysql
    config:
      connection_method: url
      dsn: env:SHOP_MYSQL_DSN

sinks:
  lake:
    type: iceberg
    config:
      catalog:
        provider: rest
        uri: https://catalog.example.com/api/catalog
        warehouse: s3://shop-lake/warehouse
        auth:
          type: client_credentials
          client_id: env:ICEBERG_CLIENT_ID
          client_secret: env:ICEBERG_CLIENT_SECRET
        properties:
          s3.region: us-east-2
      namespace: shop

pipelines:
  shop-to-lake:
    source:
      ref: shop-mysql
    sink:
      ref: lake
    resources:
      - customers
      - orders
    sync_mode: full
    write_mode

version: 1

sources:
  shop-mysql:
    type: mysql
    config:
      connection_method: url
      dsn: env:SHOP_MYSQL_DSN

sinks:
  lake:
    type: iceberg
    config:
      catalog:
        provider: rest
        uri: https://catalog.example.com/api/catalog
        warehouse: s3://shop-lake/warehouse
        auth:
          type: client_credentials
          client_id: env:ICEBERG_CLIENT_ID
          client_secret: env:ICEBERG_CLIENT_SECRET
        properties:
          s3.region: us-east-2
      namespace: shop

pipelines:
  shop-to-lake:
    source:
      ref: shop-mysql
    sink:
      ref: lake
    resources:
      - customers
      - orders
    sync_mode: full
    write_mode

Run it and read the result from any engine on the catalog.

SELECT COUNT(*) FROM shop.orders;
SELECT status, COUNT(*) FROM shop.orders GROUP BY status ORDER BY

SELECT COUNT(*) FROM shop.orders;
SELECT status, COUNT(*) FROM shop.orders GROUP BY status ORDER BY

SELECT COUNT(*) FROM shop.orders;
SELECT status, COUNT(*) FROM shop.orders GROUP BY status ORDER BY

Switch to change data capture once the full load is clean. CDC needs the Postgres-backed deployment, not the local SQLite context. Point the CLI there, recreate the source as a replica, and create the same lake sink, passing nested fields such as the catalog block as JSON.

filament source create shop-mysql \
  --source-connector mysql \
  --source-connection-method url \
  --source-dsn "filament:change-me@tcp(mysql.internal:3306)/shop?tls=true" \
  --source-replication cdc

filament pipeline create shop-to-lake-cdc \
  --source shop-mysql \
  --sink lake \
  --resources customers,orders \
  --sync-mode cdc \
  --write-mode merge \
  --sink-namespace

filament source create shop-mysql \
  --source-connector mysql \
  --source-connection-method url \
  --source-dsn "filament:change-me@tcp(mysql.internal:3306)/shop?tls=true" \
  --source-replication cdc

filament pipeline create shop-to-lake-cdc \
  --source shop-mysql \
  --sink lake \
  --resources customers,orders \
  --sync-mode cdc \
  --write-mode merge \
  --sink-namespace

filament source create shop-mysql \
  --source-connector mysql \
  --source-connection-method url \
  --source-dsn "filament:change-me@tcp(mysql.internal:3306)/shop?tls=true" \
  --source-replication cdc

filament pipeline create shop-to-lake-cdc \
  --source shop-mysql \
  --sink lake \
  --resources customers,orders \
  --sync-mode cdc \
  --write-mode merge \
  --sink-namespace

The first run snapshots each table from one consistent point and starts the stream from the binlog position captured just before, and later runs read only new events. The how-to says to pass --sync-mode cdc explicitly and to give each pipeline a unique --source-server-id when more than one replica reads the server.

Add a cron schedule in the web app or the Pipelines API and keep the interval well inside binlog retention, because a purged position means a re-snapshot.

Which MySQL to Iceberg path fits which workload?

Stream through Flink or Kafka if you already run one, buy Fivetran or Estuary if nobody will run a process, and run Filament for merged deletes on your own compute with no sidecar.

OLake wins on heavily partitioned lakes and on Unity or S3 Tables catalogs, which Filament’s sink does not name, and it beat Filament on the Postgres full load. Debezium and Flink win when a cluster already exists and sub-minute latency matters, since Filament’s CDC runs are bounded cycles. Fivetran wins when nobody will operate anything, and Filament takes merged deletes on your own compute through one process.

Related guides

The MySQL binlog replication explainer covers ROW format, GTIDs, and retention, and the Postgres to Apache Iceberg guide carries the Postgres side of the same cohort. The same source lands in a warehouse in MySQL to Snowflake and MySQL to ClickHouse, and the write modes explainer defines replace, upsert, and merge.

Frequently asked questions

Do you need Kafka to sync MySQL to Iceberg?

No. OLake, Debezium Server with the community Iceberg sink, Flink CDC, Airbyte, Estuary, ingestr, Sling, and Filament all read the binlog and write Iceberg without a broker. Kafka enters only when you pick the Apache Iceberg Kafka Connect sink, which needs Kafka 2.5 or later for its exactly-once commit coordination.

Which MySQL settings does Iceberg replication need?

Row-based binary logging, replication privileges for the capture user, and binlog retention longer than the longest gap between runs. MySQL 8.4 turns the binary log on with the ROW format by default and keeps 30 days of log. GTIDs stay off by default and are optional for most tools, though they make a failover painless.

How do MySQL updates and deletes land in Iceberg tables?

Through row-level delete files or rewritten data files. OLake, Airbyte, Sling, and ingestr write equality deletes on format v2 tables, Estuary rewrites the affected files by default, and the memiiso Debezium sink added v3 deletion vectors in October 2026. The Iceberg spec drops new equality deletes in v4, so check your reader before choosing.

Does the initial snapshot lock MySQL tables?

It depends on the tool. Debezium’s default snapshot takes a global read lock, and on RDS or Aurora it falls back to table locks that need the LOCK TABLES privilege. Flink CDC’s incremental snapshot avoids the lock entirely. Engines that bootstrap from a consistent snapshot hold a read lock only for the instant between capturing the binlog position and opening that snapshot.

What happens to a MySQL to Iceberg pipeline after a long outage?

If the outage outlasts binlog retention, the server purges the position the pipeline needs and the stream cannot resume. Filament reports that the binlog position is no longer available and asks for a re-snapshot, and ingestr fails the run rather than taking a partial snapshot. Keep retention well past your longest expected downtime.

Can you replicate MariaDB to Iceberg with the same tools?

Partly. OLake lists MariaDB as a supported binlog source, ingestr exposes a mariadb+cdc scheme, and Debezium split MariaDB into its own connector in release 3.6. A tool whose MySQL source is documented and tested only for MySQL 8.0 and later should be treated as untested on MariaDB, even when it accepts a mariadb URL.

Which tool loads MySQL into Iceberg fastest?

Only one public run at scale exists. In the August 31, 2026 cohort Filament loaded 298 million rows from MySQL into Iceberg in 235.7 seconds, and OLake took about twice as long on the same host and REST catalog. OLake’s own multi-billion-row test against Fivetran ran on different hardware, so the two sets of numbers do not compare.

How does Filament write MySQL changes into Iceberg?

A CDC pipeline snapshots each table from a consistent point, then streams binlog inserts, updates, and deletes as bounded catch-up runs on a schedule. The Iceberg sink buffers each table’s rows as Arrow batches, writes Parquet at commit, and applies merge by primary key, so a table becomes visible only when its own commit succeeds.