Best Way to Sync GitHub to Snowflake in 2026

This page is for the platform engineer who keeps engineering metrics in Snowflake and wants pull requests, issues, and workflow runs there without a per-row bill.

As of October 2026 there are four ways to land GitHub data in Snowflake. Managed ELT such as Fivetran or Estuary runs the sync and meters rows, credits, or gigabytes. A self-hosted engine such as Filament, Airbyte, or CloudQuery runs on your compute and writes typed tables. A library such as dlt or a webhook build leaves scheduling to you. Request budget, deletes, Snowflake auth, and pricing unit decide it.

What GitHub hands a sync tool

Every tool on this route reads the same REST API, with a budget of "5,000 requests per hour" per account and pages capped at 100 rows, per the rate limit docs and pagination docs.

The issues quirk corrupts dashboards. The docs say "GitHub's REST API considers every pull request an issue, but not every issue is a pull request." Land the issues resource without that marker and every pull request is counted twice.

What decides a GitHub to Snowflake sync?

Five criteria decide this route, and none of them is throughput, because the API budget caps every tool at the same ceiling.

• Incremental cursors. The tool should use since on issues and comments instead of re-reading every pull request on every run.

• Delete and churn handling. The REST API never announces a removed label or repository, so churning resources need a full replace or a keyed merge.

• Snowflake authentication and landing. Snowflake is enforcing strong authentication account by account through October 2026, so the tool should load through an internal stage with key pair auth, per the Airbyte Snowflake destination notes.

• Shape in Snowflake. Nested GitHub JSON needs typed columns for the fields you query plus a VARIANT copy of the rest.

• The unit you pay by. Rows, credits, events, gigabytes, and warehouse seconds each punish a different workload, and nightly pull request re-reads across hundreds of repositories punish a row meter most.

Which tools can sync GitHub to Snowflake?

Eleven tools name GitHub as a source and Snowflake as a destination, and they split cleanly on who runs them and what they meter.

Fivetran syncs "commits, users, pull-requests, and other GitHub data", hard deletes rows in its join tables, and connects to GitHub Enterprise Server, per its connector page and setup guide. The setup guide says to spread four or five tokens across user accounts to dodge the rate limit, and the free plan stops at 500,000 monthly active rows.

Airbyte's certified GitHub source has the longest stream list of the eleven. Its docs say only comments, commits, issues, and review comments are "pure incremental", and a failed attempt "now restarts the stream from the first one" instead of resuming. Airbyte Cloud starts at $20 a month with hourly syncs, and the source is licensed ELv2.

Estuary captures issues, pull requests, comments, and stargazers as real time resources and the rest as batch, per its GitHub capture docs. It bills "$0.50/GB + $100/mo per connector instance" above a free tier, per its pricing page, and the core is BSL licensed.

Stitch runs the Singer tap with key-based incremental tables from $100 a month, per its integration page and pricing. Hevo's API-based GitHub source is in Early Access, supports merge mode only, and bills by events above a free tier of one million a month, per its Edge docs and pricing.

The code-first tools cost nothing to run and ask more of you. dlt's verified source covers reactions and repo events for one repository, and its Snowflake destination supports key pair auth and merge strategies. ingestr reloads one repository's issues in full on every run, per its source page, under an FSL 1.1 license.

Sling's GitHub connector exists but "APIs require a CLI Pro token or Platform Plan", per its docs. CloudQuery's GitHub plugin reads whole organizations, and its free tier stops at one million rows.

Where Filament fits on GitHub to Snowflake

Filament reads one GitHub organization through an HTTP manifest and writes typed Snowflake tables through the user's internal stage, with the GitHub source at beta and the Snowflake sink at alpha on the roadmap. Disclosure, this site belongs to Galaxy, the company behind Filament.

Against the five criteria it scores on cursors, auth, shape, and cost. Issues, issue comments, and review comments read incrementally through since with one durable watermark per resource and a five minute re-read window. Every other resource is a full read that resumes after its last completed page if interrupted.

Snowflake auth is key pair or programmatic access token only. The sink stages Parquet under the user's @~ stage with no external storage integration, loads it with COPY INTO, and maps every logical type to a typed column with JSON as VARIANT. The tool meters nothing because it is Apache 2.0, so the bill is your compute plus warehouse seconds.

The limits are written down too. The docs say "Schema evolution is add-only. Existing columns are not dropped, renamed, or altered." Writes become visible batch by batch with no transaction over a whole run, and a failed replace leaves the target table empty. The public benchmark cohort has run no GitHub route, and a route without a public result file gets no throughput claim, so this page carries none.

How to sync GitHub to Snowflake with Filament

One Snowflake service user, one fine-grained token, and two pipelines cover an organization. The sink block is copied from the Postgres to Snowflake how-to, and the source flags apply its --source-<field> rule to the connector's two config keys, because the docs carry no GitHub CLI example yet.

Generate the key pair and create the Snowflake objects. The full grant script is on the sink page.

openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -out filament_key.p8 -nocrypt
openssl rsa -in filament_key.p8 -pubout -out filament_key.pub
grep -v '^-----' filament_key.pub | tr -d '\n'

openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -out filament_key.p8 -nocrypt
openssl rsa -in filament_key.p8 -pubout -out filament_key.pub
grep -v '^-----' filament_key.pub | tr -d '\n'

openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -out filament_key.p8 -nocrypt
openssl rsa -in filament_key.p8 -pubout -out filament_key.pub
grep -v '^-----' filament_key.pub | tr -d '\n'

USE ROLE SECURITYADMIN;
CREATE ROLE IF NOT EXISTS FILAMENT_ROLE;
GRANT ROLE FILAMENT_ROLE TO ROLE SYSADMIN;
USE ROLE SYSADMIN;
CREATE WAREHOUSE IF NOT EXISTS FILAMENT_WAREHOUSE
  WAREHOUSE_SIZE = XSMALL AUTO_SUSPEND = 60 AUTO_RESUME = TRUE INITIALLY_SUSPENDED = TRUE;
CREATE DATABASE IF NOT EXISTS FILAMENT_DATABASE;
GRANT USAGE ON WAREHOUSE FILAMENT_WAREHOUSE TO ROLE FILAMENT_ROLE;
GRANT USAGE, CREATE SCHEMA ON DATABASE FILAMENT_DATABASE TO ROLE FILAMENT_ROLE;
USE ROLE SECURITYADMIN;
CREATE USER IF NOT EXISTS FILAMENT_USER
  TYPE = SERVICE DEFAULT_ROLE = FILAMENT_ROLE DEFAULT_WAREHOUSE = FILAMENT_WAREHOUSE;
GRANT ROLE FILAMENT_ROLE TO USER FILAMENT_USER;
ALTER USER FILAMENT_USER SET BINARY_INPUT_FORMAT = 'BASE64';
ALTER USER FILAMENT_USER SET TIMESTAMP_INPUT_FORMAT = 'AUTO';
ALTER USER FILAMENT_USER SET RSA_PUBLIC_KEY = '<PUBLIC_KEY_BODY>';

USE ROLE SECURITYADMIN;
CREATE ROLE IF NOT EXISTS FILAMENT_ROLE;
GRANT ROLE FILAMENT_ROLE TO ROLE SYSADMIN;
USE ROLE SYSADMIN;
CREATE WAREHOUSE IF NOT EXISTS FILAMENT_WAREHOUSE
  WAREHOUSE_SIZE = XSMALL AUTO_SUSPEND = 60 AUTO_RESUME = TRUE INITIALLY_SUSPENDED = TRUE;
CREATE DATABASE IF NOT EXISTS FILAMENT_DATABASE;
GRANT USAGE ON WAREHOUSE FILAMENT_WAREHOUSE TO ROLE FILAMENT_ROLE;
GRANT USAGE, CREATE SCHEMA ON DATABASE FILAMENT_DATABASE TO ROLE FILAMENT_ROLE;
USE ROLE SECURITYADMIN;
CREATE USER IF NOT EXISTS FILAMENT_USER
  TYPE = SERVICE DEFAULT_ROLE = FILAMENT_ROLE DEFAULT_WAREHOUSE = FILAMENT_WAREHOUSE;
GRANT ROLE FILAMENT_ROLE TO USER FILAMENT_USER;
ALTER USER FILAMENT_USER SET BINARY_INPUT_FORMAT = 'BASE64';
ALTER USER FILAMENT_USER SET TIMESTAMP_INPUT_FORMAT = 'AUTO';
ALTER USER FILAMENT_USER SET RSA_PUBLIC_KEY = '<PUBLIC_KEY_BODY>';

USE ROLE SECURITYADMIN;
CREATE ROLE IF NOT EXISTS FILAMENT_ROLE;
GRANT ROLE FILAMENT_ROLE TO ROLE SYSADMIN;
USE ROLE SYSADMIN;
CREATE WAREHOUSE IF NOT EXISTS FILAMENT_WAREHOUSE
  WAREHOUSE_SIZE = XSMALL AUTO_SUSPEND = 60 AUTO_RESUME = TRUE INITIALLY_SUSPENDED = TRUE;
CREATE DATABASE IF NOT EXISTS FILAMENT_DATABASE;
GRANT USAGE ON WAREHOUSE FILAMENT_WAREHOUSE TO ROLE FILAMENT_ROLE;
GRANT USAGE, CREATE SCHEMA ON DATABASE FILAMENT_DATABASE TO ROLE FILAMENT_ROLE;
USE ROLE SECURITYADMIN;
CREATE USER IF NOT EXISTS FILAMENT_USER
  TYPE = SERVICE DEFAULT_ROLE = FILAMENT_ROLE DEFAULT_WAREHOUSE = FILAMENT_WAREHOUSE;
GRANT ROLE FILAMENT_ROLE TO USER FILAMENT_USER;
ALTER USER FILAMENT_USER SET BINARY_INPUT_FORMAT = 'BASE64';
ALTER USER FILAMENT_USER SET TIMESTAMP_INPUT_FORMAT = 'AUTO';
ALTER USER FILAMENT_USER SET RSA_PUBLIC_KEY = '<PUBLIC_KEY_BODY>';

Install the CLI with brew install galaxy-io/tap/filament or the install script, then create the source with a fine-grained token. Teams need organization members read and stars need repository admin.

filament source create acme-github \
  --source-connector github \
  --source-token "$GITHUB_TOKEN" \
  --source-organization acme

filament sink create snow-eng \
  --sink-connector snowflake \
  --sink-account myorg-myaccount \
  --sink-username FILAMENT_USER \
  --sink-warehouse FILAMENT_WAREHOUSE \
  --sink-database-name FILAMENT_DATABASE \
  --sink-role FILAMENT_ROLE \
  --sink-auth "$(jq -n --rawfile k filament_key.p8 '{type: "key_pair", private_key: $k}')"

filament source create acme-github \
  --source-connector github \
  --source-token "$GITHUB_TOKEN" \
  --source-organization acme

filament sink create snow-eng \
  --sink-connector snowflake \
  --sink-account myorg-myaccount \
  --sink-username FILAMENT_USER \
  --sink-warehouse FILAMENT_WAREHOUSE \
  --sink-database-name FILAMENT_DATABASE \
  --sink-role FILAMENT_ROLE \
  --sink-auth "$(jq -n --rawfile k filament_key.p8 '{type: "key_pair", private_key: $k}')"

filament source create acme-github \
  --source-connector github \
  --source-token "$GITHUB_TOKEN" \
  --source-organization acme

filament sink create snow-eng \
  --sink-connector snowflake \
  --sink-account myorg-myaccount \
  --sink-username FILAMENT_USER \
  --sink-warehouse FILAMENT_WAREHOUSE \
  --sink-database-name FILAMENT_DATABASE \
  --sink-role FILAMENT_ROLE \
  --sink-auth "$(jq -n --rawfile k filament_key.p8 '{type: "key_pair", private_key: $k}')"

Discover the resources, then create one incremental pipeline for the three since streams and one full pipeline for the catalog tables. Upsert needs a primary key, which discover shows.

filament source discover acme-github

filament pipeline create github-activity \
  --source acme-github \
  --sink snow-eng \
  --resources issues,issue_comments,pull_request_review_comments \
  --sync-mode incremental \
  --write-mode upsert \
  --sink-schema GITHUB

filament pipeline create github-catalog \
  --source acme-github \
  --sink snow-eng \
  --resources repositories,repository_details,pull_requests,releases,traffic_views,traffic_clones \
  --sync-mode full \
  --write-mode upsert \
  --sink-schema GITHUB

filament run github-catalog
filament run github-activity

filament source discover acme-github

filament pipeline create github-activity \
  --source acme-github \
  --sink snow-eng \
  --resources issues,issue_comments,pull_request_review_comments \
  --sync-mode incremental \
  --write-mode upsert \
  --sink-schema GITHUB

filament pipeline create github-catalog \
  --source acme-github \
  --sink snow-eng \
  --resources repositories,repository_details,pull_requests,releases,traffic_views,traffic_clones \
  --sync-mode full \
  --write-mode upsert \
  --sink-schema GITHUB

filament run github-catalog
filament run github-activity

filament source discover acme-github

filament pipeline create github-activity \
  --source acme-github \
  --sink snow-eng \
  --resources issues,issue_comments,pull_request_review_comments \
  --sync-mode incremental \
  --write-mode upsert \
  --sink-schema GITHUB

filament pipeline create github-catalog \
  --source acme-github \
  --sink snow-eng \
  --resources repositories,repository_details,pull_requests,releases,traffic_views,traffic_clones \
  --sync-mode full \
  --write-mode upsert \
  --sink-schema GITHUB

filament run github-catalog
filament run github-activity

Stars and watchers are current membership, so the docs give them a separate full replace pipeline. Traffic views and clones sit in the upsert pipeline because GitHub keeps 14 days and the upsert keeps the history. Verify in Snowflake, then list runs with filament run ls github-activity.

SELECT count(*) FROM FILAMENT_DATABASE.GITHUB.ISSUES;
SELECT count(*) FROM FILAMENT_DATABASE.GITHUB.ISSUES WHERE pull_request IS NOT NULL;
SELECT max(_filament_run_started_at) FROM FILAMENT_DATABASE.GITHUB.ISSUES;

SELECT count(*) FROM FILAMENT_DATABASE.GITHUB.ISSUES;
SELECT count(*) FROM FILAMENT_DATABASE.GITHUB.ISSUES WHERE pull_request IS NOT NULL;
SELECT max(_filament_run_started_at) FROM FILAMENT_DATABASE.GITHUB.ISSUES;

SELECT count(*) FROM FILAMENT_DATABASE.GITHUB.ISSUES;
SELECT count(*) FROM FILAMENT_DATABASE.GITHUB.ISSUES WHERE pull_request IS NOT NULL;
SELECT max(_filament_run_started_at) FROM FILAMENT_DATABASE.GITHUB.ISSUES;

The second query is the double count check, and every row also carries _filament_run_id and _filament_operation so a dashboard can exclude a failed run. Schedule the activity pipeline hourly and the catalog pipeline nightly in the web app or the Pipelines API. Snowflake uses "per-second billing (with a 60-second minimum each time the warehouse starts)", per its warehouse docs, so the schedule sets the Snowflake bill.

Matching the path to the metrics you need

Organization scope and tolerance for a row meter decide the pick.

Fivetran wins when the team already pays for it and wants join table deletes handled. Airbyte wins on stream breadth when Kubernetes is already there. Estuary wins for near real time changes under its free gigabytes, and dlt for one repository a Python engineer owns. Filament takes the organization-wide case where data stays on your compute and the only meter is the warehouse.

Related guides

• Best Way to Sync GitHub to Postgres, the same source into Postgres.

• Postgres to Snowflake replication guide, the database route into the same sink.

• Best Way to Sync Stripe to Snowflake, a second SaaS API with the same Snowflake setup.

• MySQL to Snowflake replication guide, binlog capture into Snowflake.

• Best Way to Sync Notion to Snowflake, another polled API with no outbound sync.

• Data Integrity Verification in Replication Pipelines, what a batch checksum proves and misses.

• Filament GitHub source, Snowflake sink, and HTTP manifests.

Frequently asked questions

Can Snowflake pull data from GitHub directly?

No. Snowflake has no GitHub connector and GitHub has no outbound export to a warehouse. Something has to call the GitHub REST API, write files, and load them. That is a managed ELT service, a self-hosted engine, a code library, or a webhook receiver that pushes events through Snowpipe Streaming.

How many GitHub API requests does a full organization sync take?

Budget one request per 100 rows per resource per repository, plus one per repository for details. A token gets 5,000 requests an hour, so an organization with a couple hundred repositories and a few hundred thousand issues and comments takes several hours on the first pass. Incremental cursors cut later runs to a few hundred requests.

Does a GitHub sync capture deleted issues and repositories in Snowflake?

Only tools that re-read the full list and run a replace or a merge with delete detection catch removals. Issues are closed rather than deleted, so most syncs never see a true delete. Repository deletion, label removal, and team changes vanish from the API and stay in Snowflake until a full refresh overwrites them.

Why are stargazers empty in my GitHub to Snowflake sync?

GitHub restricted the stargazer listing endpoints to repository admins and collaborators in July 2026. A token from an outside account now gets an empty list instead of an error, so every tool that reads stars shows zero rows. Use a token from a repository admin or read the star count from the repository details instead.

Should GitHub JSON land in Snowflake as VARIANT or as typed columns?

Land the projected fields as typed columns and keep the full response in one VARIANT column. Typed columns make issue counts, cycle time, and review latency cheap to query, and the VARIANT copy keeps fields the connector never projected, so a later question about reactions or labels needs no re-sync.

How does Filament sync GitHub issues to Snowflake incrementally?

Filament reads issues, issue comments, and review comments with the GitHub since parameter on the updated_at field, keeps one durable watermark per resource, and re-reads a five minute window on every run. Rows that repeat inside that window are merged by primary key in Snowflake, so the table converges rather than duplicating.

What Snowflake permissions does Filament need?

A service user with key pair or programmatic access token authentication, since password login is not exposed. Its role needs USAGE on the warehouse, USAGE and CREATE SCHEMA on the database, and USAGE plus CREATE TABLE on any schema that already exists. Filament creates its own schema when allowed, which keeps table level grants out of your hands.

How often should a GitHub to Snowflake pipeline run?

Hourly for pull request and issue tables that feed cycle time dashboards, nightly for repositories, releases, and traffic. Each run resumes the Snowflake warehouse and bills a 60 second minimum, so a five minute schedule costs more in warehouse credits than in GitHub requests. Match the schedule to how often the dashboard is read.