GA4: Traffic Acquisition: Session Source / Medium Report is Suddenly VERY Inaccurate

[tpawlowski]

I use Fivetran's GA4 connector to sync a Traffic Acquisition: Session Source / Medium report to our data warehouse, and then I feed that data into a BI reporting tool that can refresh and store the dataset from the data warehouse on demand. I noticed that between two of the report refreshes (one on 3/2/2023 and one on 3/17/2023), the GA4 Sessions count drastically decreased. Because I had an older version of the file still stored, I was able to pull out the datasets to confirm. Here's a summary of the change:

Upon logging into GA4 and checking the numbers in platform, I find the data from 3/2/2023 much closer to what GA4 is displaying in platform. I understand that variances between the GA4 reporting API and GA4 in platform counts are to be expected, but in previous months those variances were around 1%, much less than the 13% seen in the 3/17/2023 dataset. 

To help rule out user error, I set up a new GA4 connector with the same traffic acquisition report, and it's displaying the same (lower) numbers I see from the 3/17 data pull.

To try to understand the variance better, I drilled into the session source/medium for the Feb 2023 period where I'm seeing the large variance between the two pulls. On a single session/source medium, I'm seeing what looks like data points missing for whole days:

Is anyone else experiencing weird issues with the GA4 connector? We have been using this connector since October 2022 without issues. I find it very odd that the connector seemingly had the "correct" data as of the 3/2 data pull, and has seemingly lost data points as of 3/17. Not sure if this is a Fivetran issue or a GA4 API issue.

[tpawlowski · Accepted solution]

I created a support case with Fivetran on this issue, and they were very helpful.

These are the dimensions and metrics that were being requested (essentially a pre-built Traffic Acquisition: Session Source/Medium prebuilt report in the Google Analytics 4 Fivetran connector):

Dimensions:

date

sessionSourceMedium

Metrics:

conversions

engagedSessions

newUsers

sessionConversionRate

sessions

totalUsers

Fivetran support helped me discover that Fivetran is accurately returning/syncing the results of that API call. However, through my own testing I've now found that the presence of the totalUsers metric appears to alter the number of date results returned in the API call. When I remove totalUsers from the API call, I get back the number of results I expect, which yields the correct sum of Sessions.

I've actually confirmed that this behavior is seen within the GA4 web user interface as well, not only in the API. So this is not a Fivetran issue. It's either a bug with GA4, or perhaps a data sampling issue, or it's just pointing toward a limitation in my own understanding of how combinations of dimensions and metrics work in GA4 reports. What I find odd is that this behavior never popped up before the last few weeks. I configured this GA4 connector, with these settings, sometime in 2022, and never noticed this issue. So perhaps something changed with how GA4 reporting works, or the data pattern within our property changed (perhaps grew in volume?) to start triggering this behavior.