When to update Yocto sstate-cache on CI server

Our Yocto CI script restores the sstate-cache on every build and updates it when bitbake succeeds via rsync. Our intention is to speed-up the CI builds by not rebuilding packages we already have.

The issue is that the sstate-cache is updated regardless of which branch was used to run bitbake.

This means that any successful build updates the sstate-cache that is used for all CI builds. So if the CI run on a temporary/experimental branch that updates a lot of packages, then the sstate-cache is updated accordingly and if the next build does not use the same version of packages, it is much slower and once again updates the sstate-cache with its version of the packages.

I am wondering what is the best practise to update the sstate-cache in this scenario?