Best Data Movement Tools in 2026, Benchmarked

Intergalactic Data Labs
โ
โ
13 min read
Updated October 19, 2026
Data movement tools copy rows from a source system into a sink in one of three ways. A full load copies everything once, a batch job copies what changed on a schedule, and change data capture reads each insert, update, and delete from the database log. The public 2026-08-31 benchmark timed seven of the main tools, Filament, dlt, PeerDB, ingestr, Sling, Debezium, and Airbyte, on the same 298 million row Postgres to Postgres load, and this guide adds OLake, Fivetran, and Estuary for lake, managed, and streaming jobs. Filament posted the fastest load at 115 seconds with dlt and PeerDB next, while ingestr used the least CPU and Sling the least memory.
Tool | Wall time | Rows per second | CPU seconds | Peak memory | License |
|---|---|---|---|---|---|
1. Filament | 115 s | 2,597,498 | 377 | 2.97 GiB | Apache 2.0 |
2. dlt | 296 s | 1,007,711 | 728 | 45.5 GiB | Apache 2.0 |
3. PeerDB | 415 s | 718,368 | 3,270 | 0.74 GiB | AGPLv3 |
Baseline pg_dump to psql | 489 s | 610,267 | 63 | 0.01 GiB | PostgreSQL |
4. ingestr | 611 s | 488,233 | 157 | 4.83 GiB | FSL-1.1-ALv2 |
5. Sling | 1,053 s | 283,188 | 2,211 | 0.10 GiB | GPLv3 |
6. Debezium (snapshot) | 6,559 s | 45,476 | 11,658 | 15.7 GiB | Apache 2.0 |
7. Airbyte | 10,393 s | 28,698 | 13,703 | 12.8 GiB | ELv2 |
Every figure is the median run from the raw result files in the 2026-08-31 cohort of the galaxy-io/benchmarks repository, which loaded 298,270,427 NYC Taxi rows from one RDS Postgres into another. Licenses are quoted from each project's LICENSE file, not its README badge. The companion benchmark write-up walks through the same cohort in more depth.
How do you benchmark ETL tool performance?
Time one fixed dataset through every tool on identical hardware, repeat each run, and verify the sink matches the source. This benchmark ran each tool in its official container on the same host.
The methodology is explicit about scope. "The measured wall time starts when the harness triggers the prepared transfer. It ends when the adapter reports completion." Setup is excluded, and "the harness verifies row counts, not row contents."
The README discloses the conflict of interest up front. "We built filament, one of the systems measured here." It also flags that for Debezium "the full-load benchmark measures its initial snapshot, not the steady-state streaming workload it is primarily designed for."
Setting | Value |
|---|---|
Host | AWS c7i.16xlarge, 64 vCPU, 128 GiB |
Source and sink | Separate RDS db.m6i.8xlarge, same AZ |
Dataset | NYC Taxi, 12 months, 2 tables |
Repetitions | 3, except Debezium and Airbyte at 1 |
Cold start | RDS reboot before every timed run |
Verification | Row count parity |
What are the fastest data ingestion tools?
On the public Postgres to Postgres run, the fastest tools were Filament at 115 seconds, dlt at 296 seconds, and PeerDB at 415 seconds. The middle tier finished under eighteen minutes and the bottom tier took hours.
Takeaway. Filament beat the pg_dump baseline by 4.3 times, dlt by 2.6 times, and every other tool by more, but only on this one route.
Filament reads Postgres tables in up to 64 parallel ranges, per its source docs, and encodes rows "into PostgreSQL's binary COPY format directly from the Arrow columns," per its sink docs. dlt took second with the ConnectorX backend set in its result file, at the cost of the highest memory in the cohort.
A single pg_dump piped to psql landed fourth while using almost no CPU or memory, which makes it the number every tool should beat. Debezium and Airbyte are built for a different job than a one shot bulk copy.
The tools, ranked
Takeaway. Three of the seven ship under Apache 2.0, and every one of them wins at least one job.
1. Filament
Filament is a Go replication engine under Apache 2.0 that installs as a prebuilt binary or runs as a deployed server, per its CLI docs. It supports full, incremental, and change data capture reads from Postgres and MySQL, both beta. The connector page lists 29 sources and 13 sinks, most of them alpha, and the latest release is v0.1.1.
2. dlt
dlt is a Python library for loading data, and its README says it "can be dropped in anywhere," from a notebook to an Airflow DAG. That portability wins Python teams. The managed tier, dltHub, costs $12,000 per month, and the library stays Apache 2.0.
3. PeerDB
PeerDB is a Postgres change data capture engine that ClickHouse acquired in July 2024. Its README says the actively maintained paths are Postgres to ClickHouse, ClickHouse Cloud, and Postgres, and the Snowflake, BigQuery, and Kafka destinations are deprecated. The badge says ELv2 but the LICENSE file is AGPLv3.
4. ingestr
ingestr is a CLI from Bruin that copies data between more than a hundred sources and destinations, per its README. It used the least CPU of any tool in the cohort. The LICENSE file is FSL-1.1 with an Apache 2.0 future license, which forbids using it to offer a competing ingestion service.
5. Sling
Sling is a Go CLI under GPLv3 whose README describes it as built for "small to medium volume data pipelines." It had the smallest footprint of any tool, peaking at 0.10 GiB. Parallel streams need CLI Pro at $79 per month, and log based change data capture needs Pro Max at $149.
6. Debezium
Debezium is the reference open source change data capture platform, under Apache 2.0. Its Postgres connector docs say the connector "typically spends the vast majority of its time streaming changes." The benchmark timed only its initial snapshot.
7. Airbyte
Airbyte is open source and has the widest catalog in the group, with more than 700 connectors on its pricing page. Its connectors are under the Elastic License 2.0, with the protocol under MIT. The benchmark used Standard replication rather than change data capture.
What is the difference between real-time and batch data movement?
Batch tools copy data on a schedule and stop, micro-batch tools run short catch-up jobs every few minutes or less, and streaming tools stay connected and push each change as it happens. For dynamic data that changes all day, change data capture scales better than repeated full loads because it moves only changed rows.
Tool | Delivery model | Fastest documented cadence | Source |
|---|---|---|---|
Filament | Micro-batch CDC, streaming for Kafka and NATS | As often as you schedule each catch-up | |
Estuary | Streaming | Under 100 ms end to end, vendor claim | |
Debezium | Streaming | Continuous, event by event | |
PeerDB | Streaming CDC | Freshness of "a few 10s of seconds" | |
ingestr | Batch, streaming with a flag | Continuous Postgres CDC with --stream | |
Fivetran | Micro-batch | 1 minute on Enterprise and Business Critical | |
OLake | Micro-batch | Every minute in its scheduler | |
Sling | Batch, CDC runs on a schedule | Every 30 seconds in the docs example, Pro Max only | |
Airbyte | Micro-batch | 5 minutes on Pro, 15 on Plus, 1 hour on Standard | |
dlt | Batch | Whenever your scheduler runs it |
For Filament, "continuous CDC is a re-request loop, so it can run on a schedule," per its MySQL source docs.
What are the top real-time data movement tools?
The top real-time data movement tools in 2026 are Filament, Estuary, Debezium, and PeerDB, chosen by how fresh the sink must be. Filament fits change data capture with checksummed batches on a loop you schedule, Estuary and Debezium fit continuous streams, and PeerDB fits Postgres into ClickHouse. Fivetran's 1 minute tier and ingestr's stream flag cover the next band.
What are the best batch data movement tools for enterprise workloads?
For large scheduled loads, Filament, Fivetran, Airbyte, and dlt cover most enterprise needs. Filament had the fastest bulk load in the benchmark and its route checkpoints "advance only at a committed completion or committed pause," per the integrity docs. Fivetran runs syncs from 1 minute to 24 hours with no software to host, per its sync overview, Airbyte covers the SaaS long tail, and dlt fits teams that already schedule Python.
Which open-source tools handle large-scale data movement?
Filament, dlt, Debezium, OLake, and Airbyte are the main open source options at scale. Filament and dlt, both Apache 2.0, moved the 298 million rows fastest. Debezium streams changes through Kafka, OLake writes databases into Apache Iceberg and is Apache 2.0 per its LICENCE file, and Airbyte brings the widest connector catalog.
PeerDB and Sling are open source with copyleft terms, and ingestr is source available. The open source license breakdown covers each license in detail.
What are the most reliable data movement solutions?
Reliable tools checkpoint progress, resume after a crash without losing rows, and prove the sink matches the source. Filament computes a CRC32-C checksum for every batch that the sink recomputes before the write, and its recovery is "deliberately at-least-once," per the integrity docs. Debezium writes an ordered, replayable change log to Kafka, and Fivetran runs the whole pipeline as a managed service.
Which tool fits which job?
The ranking above measured one route, so match the tool to the job.
Tool | Best for | Trade-off | Cost |
|---|---|---|---|
Filament | Fast Postgres loads, Postgres and MySQL CDC with checksummed batches | Pre-1.0, most connectors alpha | No license fee, Apache 2.0 |
dlt | Python teams, custom sources, notebooks | Highest memory, Python runtime | Free library, dltHub $12,000 per month |
PeerDB | Postgres to ClickHouse CDC | AGPLv3, narrow destination list | Free OSS, also in ClickHouse Cloud |
ingestr | Low CPU copies, long source list | Source available, not OSI open source | Free CLI |
Sling | Tiny footprint, small pipelines | Parallel streams and CDC are paid | Free CLI, Pro $79, Pro Max $149 per month |
Debezium | Streaming CDC into Kafka | Slow snapshot, Java and Kafka ops | Free, Apache 2.0 |
Airbyte | Long tail SaaS connectors, managed sync | Slowest bulk load, hourly on Cloud Standard | Cloud from $20 per month plus credits |
OLake | Databases into Iceberg | Iceberg and Parquet only | Free, Apache 2.0 |
Fivetran | Managed, no ops, compliance | Closed source, cost scales with rows | Monthly active rows, 500k free |
Estuary | Sub-second streaming | BSL license, per connector fee | $0.50 per GB plus $100 per connector |
Fivetran and Estuary were not benchmarked, since both are hosted services, but Fivetran is the tool most top ranking listicles lead with. Its pricing docs count monthly active rows once per month, and "initial syncs don't incur a cost for the historical data they sync." Estuary charges $0.50 per GB plus $100 per month for each of the first six connectors, per its pricing docs.
How should you pick a data movement tool?
Start with the delivery model, then the limit that matters most in your stack.
If wall time on a bulk load matters most, Filament led the Postgres route and dlt was second.
If memory on a shared host matters most, Sling and PeerDB stayed under 1 GiB, and dlt with ConnectorX peaked at 45.5 GiB.
If a missing connector matters most, Airbyte, ingestr, and Fivetran cover hundreds of SaaS sources that the Go replication engines do not.
If freshness matters most, Estuary and Debezium stream continuously, and Filament runs change data capture on whatever loop you schedule.
For continuous change data capture the benchmark says nothing, so weigh maturity. Debezium has years of production history, while Filament and ingestr read the Postgres log through logical replication on younger code paths. The replication guide and the full load vs incremental vs CDC guide cover those trade-offs.
What does this benchmark not tell you?
It tells you nothing about incremental or change data capture throughput, failure recovery, or correctness beyond row counts. The benchmark limits page says so directly. "Results do not predict incremental or CDC performance."
Versions have moved since the run. dlt is now 1.31.0, Sling 1.6.5, and Debezium 3.7.0. Peak memory for PeerDB and Airbyte is the largest single container, and the sum would raise Airbyte to 14.4 GiB.
Galaxy, which maintains Filament, runs the harness and publishes the raw JSON and configs so anyone can rerun it. Until a neutral party does, treat the ordering as evidence for bulk loads only. The best change data capture tools and the Airbyte alternatives guides dig into the streaming and managed ends of the table.
Frequently asked questions
What is the best data movement tool in 2026?
There is no single best tool. In the public August 2026 benchmark Filament had the fastest Postgres to Postgres full load, ingestr used the least CPU, and Sling the least memory. Airbyte has the most connectors, Debezium the most mature streaming change data capture, and OLake is built for Iceberg. Pick by delivery model first, then by the limit that matters most.
What is data movement?
Data movement is copying rows from one system to another, either once as a migration, on a schedule as batch replication, or continuously as change data capture. The tools here differ in how they read the source, how they write the sink, and what they cost.
What are the top real-time data movement tools?
Filament, Estuary, Debezium, and PeerDB lead in 2026. Filament runs change data capture as checksummed catch-up runs on a loop you schedule. Estuary claims under 100 ms end to end, Debezium streams each change into Kafka, and PeerDB keeps Postgres to ClickHouse within tens of seconds. Fivetran reaches 1 minute syncs on its Enterprise and Business Critical plans.
What is the difference between real-time and batch data movement?
Batch data movement copies data on a schedule and stops between runs. Real-time data movement keeps a connection open and applies each change within seconds. Micro-batch sits between them, running short catch-up jobs every minute or so. Most analytics runs well on micro-batch.
What are the fastest data ingestion tools?
On the public 2026-08-31 Postgres to Postgres run, Filament copied 298,270,427 NYC Taxi rows in 115 seconds, dlt in 296 seconds, and PeerDB in 415 seconds. A plain pg_dump piped to psql took 489 seconds on the same hardware, and Airbyte took 10,393 seconds. The run measured a full load only, not incremental or change data capture throughput.
Is Filament faster than Airbyte?
On a Postgres to Postgres full load of 298 million rows, Filament finished in 115 seconds and Airbyte in 10,393 seconds, about 91 times longer. Airbyte ran in Standard replication mode with a 16 GiB limit per connector container. Airbyte still leads on connector count, with more than 700 connectors.
Is Debezium slow?
Not for its main job. The benchmark timed Debezium's initial snapshot, which took 6,559 seconds, because the harness only measures full loads. Debezium is built for streaming changes after that snapshot, and its docs say the connector spends most of its time streaming.
Which data movement tools are open source?
Filament, dlt, Debezium, and OLake ship under Apache 2.0 according to their LICENSE files. Airbyte is open source with connectors under the Elastic License 2.0. Sling is GPLv3 and PeerDB is AGPLv3. ingestr is source available under FSL-1.1 with an Apache 2.0 future license, and Estuary's engine is under the Business Source License.
Which tools offer high throughput and low latency together?
No single tool leads on both in public data. Filament had the highest throughput in the Postgres to Postgres benchmark at about 2.6 million rows per second, and its change data capture latency is set by how often you schedule catch-ups. Estuary claims the lowest latency but has no public bulk load result, and Debezium's snapshot took 6,559 seconds.
How do Fivetran and Airbyte price data movement?
Fivetran bills on monthly active rows, meaning distinct rows synced in a calendar month, with initial syncs free and 500,000 rows on the free plan. Airbyte Cloud Standard starts at $20 per month with 5 credits, and database sources use 4 credits per GB, per its credits docs. Self hosted Airbyte, Filament, dlt, Sling, and OLake have no per row charge.
More articles
Questions
Answered
FAQ
What does Galaxy do?
What is Filament?
What is enterprise context management?
What does working with Galaxy look like?
How do you handle security and compliance?
Why does Galaxy build in the open?
Company
Copyright ยฉ 2026 Galaxy. All rights reserved.