Choosing a cloud data warehouse: Snowflake vs BigQuery vs Redshift for analytics
#1
I'm a data architect at a mid-sized e-commerce company, and we're outgrowing our current PostgreSQL-based analytics setup. We're evaluating a move to a proper cloud data warehouse, specifically comparing Snowflake, BigQuery, and Redshift. Our primary needs are handling complex, join-heavy queries on terabyte-scale datasets, cost predictability, and seamless integration with our existing dbt and Looker stack. For teams that have recently made this transition, what were the decisive factors in your choice? How did you approach the migration of ETL pipelines and historical data, and what unexpected operational challenges did you face regarding performance tuning or vendor lock-in?
Reply
#2
If I were choosing today, I’d treat the decision as a multi-factor trade-off rather than chasing the fastest benchmark. Snowflake tends to win on flexibility and cross‑cloud resilience (AWS, Azure, GCP), robust concurrency, and strong support for semi-structured data. BigQuery is a great fit if you’re already in Google Cloud and want a serverless model with tight Looker integration and strong analytics tooling. Redshift remains compelling if you’re deeply invested in AWS and want predictable egress costs with RA3 scaling, but you’ll trade some cloud-agnostic flexibility. Budget, data gravity, and how you’ll manage costs day to day often swing the decision more than pure performance numbers.
Reply
#3
Migration plan (brief, actionable): 1) inventory datasets, ETL/ELT flows, and SLAs; 2) pick a target platform for a 6–12 week pilot on a representative 1–2 terabyte data slice; 3) adopt dbt for transformations in the warehouse and run incremental loads; 4) build Looker connections to the new warehouse and parallelize dashboards; 5) run a parallel cutover where both systems ingest the same sources for a few weeks; 6) decommission legacy pipelines once validation passes; 7) implement cost governance and alerts.
Reply
#4
Operational realities to expect: concurrency limits and queueing can surprise on large joins; fingerprint and scene your data model to avoid churn; beware of long-running queries on BigQuery if you scan huge datasets; monitor storage vs compute spend in Snowflake—auto-suspend and multi-cluster warehouses help but need governance; don’t underestimate data quality drift during migration.
Reply
#5
Practical dbt/Looker guidance: standardize on a single warehouse as the canonical source, use dbt docs and lineage to keep teams aligned, and configure Looker with a centralized model to avoid replication drift. If you’re multi-cloud, Snowflake makes governance and access control simpler across regions; with BigQuery/Looker, rely on You to manage GCP IAM/Looker permissions.
Reply
#6
Quick yes/no prompts to tailor advice: what cloud are you leaning toward (AWS, Azure, or GCP)? roughly how much data (TB) and how many daily loads? what’s your tolerance for vendor lock-in and data egress costs?
Reply


[-]
Quick Reply
Message
Type your reply to this message here.

Image Verification
Please enter the text contained within the image into the text box below it. This process is used to prevent automated spam bots.
Image Verification
(case insensitive)

Forum Jump: