How to Migrate from SQLServer to BigQuery in PostgreSQL

How do I migrate data from SQL Server to BigQuery?

Move data from Microsoft SQL Server into Google BigQuery using export-to-CSV, Cloud Storage staging, and bq load commands.

Welcome to the Galaxy, Guardian!
You'll be receiving a confirmation email

Follow us on twitter :)

Oops! Something went wrong while submitting the form.

Description

Example H2

Example H3

What are the core migration steps?

Extract tables from SQL Server, upload the files to Google Cloud Storage, create equivalent BigQuery tables, load the staged files, then validate row counts and spot-check values.

How do I export SQL Server tables quickly?

Use the bcp utility.It streams data to flat files without locking tables, making it ideal for large datasets.

Export Customers

bcp "SELECT * FROM dbo.Customers" queryout Customers.csv -c -t , -S sqlsrv-prod -U sa -P *****

Export Orders

bcp "SELECT * FROM dbo.Orders" queryout Orders.csv -c -t , -S sqlsrv-prod -U sa -P *****

How do I stage files in Cloud Storage?

Create a bucket and copy each CSV file.

gsutil mb gs://sqlserver-migration
gsutil cp *.csv gs://sqlserver-migration/

How do I create matching tables in BigQuery?

Run DDL in the BigQuery console or bq CLI.

CREATE TABLE ecommerce.Customers ( id INT64, name STRING, email STRING, created_at DATETIME ); CREATE TABLE ecommerce.Orders ( id INT64, customer_id INT64, order_date DATE, total_amount NUMERIC );

How do I load CSVs into BigQuery?

Use bq load with schema, format, and skipping header rows.

bq --location=US load --source_format=CSV --skip_leading_rows=1 \ ecommerce.Customers gs://sqlserver-migration/Customers.csv \ id:INT64,name:STRING,email:STRING,created_at:DATETIME

How do I validate migrated data?

Compare row counts and sample aggregates between systems.

-- BigQuery SELECT COUNT(*) FROM ecommerce.Customers; -- SQL Server SELECT COUNT(*) FROM dbo.Customers;

What best practices ensure a smooth cut-over?

Freeze writes during final sync, script everything for repeatability, enable BigQuery partitioning for large fact tables, and document data-type mappings.

Why How to Migrate from SQLServer to BigQuery in PostgreSQL is important

How to Migrate from SQLServer to BigQuery in PostgreSQL Example Usage


-- After loading, join Customer and Order counts
SELECT c.id, c.name, COUNT(o.id) AS order_count, SUM(o.total_amount) AS lifetime_value
FROM ecommerce.Customers AS c
LEFT JOIN ecommerce.Orders AS o ON c.id = o.customer_id
GROUP BY c.id, c.name
ORDER BY lifetime_value DESC
LIMIT 10;

How to Migrate from SQLServer to BigQuery in PostgreSQL Syntax


1. Export from SQL Server
   bcp "SELECT * FROM dbo.<table>" queryout <table>.csv -c -t , -S <server> -U <user> -P <password>

2. Upload to Cloud Storage
   gsutil cp <table>.csv gs://<bucket>/

3. Create BigQuery tables
   CREATE TABLE dataset.<table> ( ... );

4. Load into BigQuery
   bq --location=<region> load --source_format=CSV --skip_leading_rows=1 \
      dataset.<table> gs://<bucket>/<table>.csv <schema>

Example for Customers:
   bcp "SELECT * FROM dbo.Customers" queryout Customers.csv -c -t , -S sqlsrv-prod -U sa -P *****
   gsutil cp Customers.csv gs://sqlserver-migration/
   CREATE TABLE ecommerce.Customers ( id INT64, name STRING, email STRING, created_at DATETIME );
   bq load --source_format=CSV --skip_leading_rows=1 ecommerce.Customers \
      gs://sqlserver-migration/Customers.csv \
      id:INT64,name:STRING,email:STRING,created_at:DATETIME

Common Mistakes

Mistake: Ignoring SQL Server DATETIME2 precision. Why wrong: BigQuery’s DATETIME has microsecond precision; truncation may occur. Fix: CAST columns to DATETIME before export or map to TIMESTAMP in BigQuery.
Mistake: Loading very large tables without partitioning. Why wrong: Queries become slow and costly. Fix: Create partitioned tables in BigQuery using date or integer RANGE partitions during the CREATE TABLE step.