The Instagram Graph API has a limitation that catches most teams out: it only exposes a rolling window of Insights data. Once a period falls outside that window, it’s gone — there’s no backfill, no archive, no way to recover it later.
If you want year-over-year comparisons on your organic Instagram performance, the only solution is to capture the data as it arrives and store it yourself. BigQuery is the natural destination, especially if your Shopify and GA4 data already live there.
This guide covers the methods, how to design the tables, and the limits to plan around.
Before anything: the account requirements
The Graph API only returns Insights data if:
- Your Instagram account is a Business or Creator account — personal accounts return nothing.
- That account is linked to a Facebook Page.
- Your app has been granted the relevant permissions during authentication.
If any of these is missing, your pipeline will authenticate successfully and return empty datasets — a confusing failure that looks like a bug rather than a configuration issue. We cover this and the broader Insights picture in our guide on connecting Instagram Insights to Looker Studio.
Method 1: Managed ELT tools
Airbyte, Fivetran and similar platforms offer Instagram connectors with BigQuery as a destination. You authenticate once, pick a sync frequency, and the tool handles schema creation, incremental loading and API changes.
Advantages: no code to maintain, schema evolution handled for you, and these tools already hold approved API access.
Limits: pricing scales with data volume or connector count, and you’re dependent on the provider’s implementation — if they don’t expose a metric you need, you can’t add it yourself.
Best for: teams that want history without owning a pipeline.
Method 2: Reporting connectors with a BigQuery destination
Some connectors built for Looker Studio (Supermetrics and similar) also offer BigQuery as a destination rather than only a live dashboard connection. If you’re already paying for one of these for your ad platforms, check whether Instagram Insights to BigQuery is included — it’s often the cheapest route because you’re not adding a new vendor.
Method 3: A custom pipeline
A Cloud Function (or Cloud Run job) calls the Graph API on a schedule set by Cloud Scheduler, and writes results into BigQuery.
Advantages: complete control over which metrics you capture and how you model them, and costs that stay negligible at this data volume.
Limits: you maintain it. Graph API versions deprecate on a regular cycle, so expect to update the code periodically or watch the pipeline fail silently.
Best for: teams with engineering capacity, or anyone already running custom pipelines for Shopify and ad data.
Designing the tables
The single most important design decision: Instagram Insights data is a mix of snapshots and events, and conflating them creates problems later.
Account-level metrics are snapshots. Follower count, profile views, reach — these are point-in-time values. Store one row per account per day, with the date as a column. Never overwrite: the whole purpose is accumulating history.
Media-level metrics are cumulative per object. A post’s reach, saves and shares keep growing after publication, especially on Reels which can resurface weeks later. Two valid approaches: store a daily snapshot per post (heavier, but lets you see how a post accumulated performance over time) or store the latest value per post (lighter, loses the accumulation curve). Daily snapshots are usually worth the storage at this volume.
Keep the raw payload. Store the API response alongside your parsed columns, in a JSON column. When you later realize you need a metric you didn’t parse, you’ll have it for the historical period rather than starting from zero again.
A reasonable starting structure: one table for daily account snapshots, one for media metadata (posted once per post), and one for daily media metrics snapshots.
The limits to plan around
Rate limits. The Graph API applies hourly call limits. Pulling per-post metrics for a large back catalog every day can hit them. Practical approach: sync all posts for the first 30 days after publication, then taper to weekly for older ones, since engagement mostly settles.
Story data expires fast. Stories have a much shorter availability window than feed posts through the API. If stories matter to you, sync at least daily — queried a few days later, the data is simply gone.
API versioning. Meta deprecates Graph API versions on a published schedule. Pin a version explicitly in your calls rather than relying on a default, and set a calendar reminder to review before your pinned version sunsets.
Demographic thresholds. Audience breakdowns (city, age, gender) only return above minimum volume thresholds, for privacy reasons. Smaller accounts will see gaps here, and that’s expected rather than broken.
Using the data once it’s in BigQuery
With Instagram Insights in BigQuery alongside your Shopify orders and GA4 export, two analyses become possible that neither platform offers on its own:
Content-to-revenue correlation. Compare organic reach by week against direct and branded traffic, and against revenue. You won’t get clean attribution — organic social rarely produces trackable clicks — but you can see whether the two move together over time.
True historical benchmarking. Compare this quarter’s engagement rate against the same quarter last year, with real data rather than whatever the API still remembers.
Looker Studio then reads from BigQuery with no API quotas and no sampling, following the architecture in our Looker Studio dashboard guide.
FAQ
Why move Instagram Insights into BigQuery at all? Because the Instagram Graph API only exposes a rolling window of historical data. Once a period falls outside that window, it’s gone — you can’t backfill it later. Storing snapshots in BigQuery as the data arrives is the only way to build genuine year-over-year history.
Do I need to write code to do this? Not necessarily. Managed ELT tools have Instagram connectors that write directly to BigQuery, and some reporting connectors offer a BigQuery destination. A custom Cloud Function plus Cloud Scheduler gives you more control, at the cost of maintaining it yourself.
What account setup is required? A Business or Creator Instagram account linked to a Facebook Page, plus an app with the relevant permissions granted. Personal accounts return no Insights data at all — that’s the most common blocker people hit before writing a single query.