What options are there for managing Google Play System (Mainline) Updates?
Our very own
https://bayton.org/android/gpsu-system-update
Jason is often ahead of the pack when it comes to understanding what new versions of Android bring, and I am incredibly thankful for the research he does and the education he provides to the community. In this case, an article he wrote several years ago is only just hitting my doorstep now. This is largely driven by the fact that new versions of Android, thankfully, do not propagate immediately into the production world of dedicated enterprise use. We are only really starting to see Android 14 show up in any material way now.
With that, we are now being exposed to the fact that even more components and modules have been included in the Mainline system through Google Play System Updates. One of these modules appears to be the cert store containing the system certs. Recently, we encountered a major production outage caused by a root cert rotation (Sectigo...), where the only resolution was getting the system cert lists on the devices updated. Given that the devices were on Android 14, the only way to do that was through Google Play System Updates. We found devices spread across a range of different versions of these updates and were only able to verify that manually, deep within the Settings app. Verification was manual and painful, and forcing the updates was as well, requiring multiple forced installs and reboots on the devices. This has exposed yet another vulnerability in these environments, where once again Google seems to be to blame. It is a very confusing position to find yourself in, having an automatic update that you cannot control or schedule be a critical component that the devices rely on to function, and then in an emergency having no way to automate that install. The very Mainline system was supposed to solve these sorts of issues, and yet here I find that it is making things worse in certain situations.
I am a bit perplexed by how we can enter this world of more and more core system components migrating to Mainline while simultaneously not having deployment control over those updates. Why can’t I force them? Why can’t I stop them? Why can’t I roll them back? These are questions I am asking, and any CIO with a fleet of mission-critical Android devices will be asking them as well the next time operations are down, either because Google forced an update out automatically or failed to push the update out automatically.
How is it that we do not even have a proper mechanism to report on and monitor what version of Google Play System Updates is installed on a device, and the versions of each of the potential subcomponents included within? Am I missing something? Do some EMMs report on these properties?
The uncontrolled world of updates to system components like WebView was already stressful enough, but I was at least able to report on those versions. Google Play System Updates are becoming more critical and yet remain more of a black box than ever before. I cannot report on them. I cannot automate them. I cannot force them to happen. I cannot undo them if something goes wrong.
There is this community article that was suggested to me when I started drafting this up. This however simply describes what System Updates are. It doesn’t really explain how you can actually manage them.
https://www.androidenterprise.community/best-practices-33/managing-google-system-updates-with-android-enterprise-211?tid=211&fid=33
What does the community have to say? How are you managing these in your environments?
