I am currently travelling with my laptop, after I’ve recently switched it to Fedora Atomic Workstation and then rebased to rawhide.
While I am sitting by the pool, I have some time to ponder: how has it been going ?
I have upgraded to a newer image a few times now, and I won’t lie: rawhide is still rawhide. Sometimes, the new image was a disappointment and would not give me a working system. In one case, it hung before getting to the login screen. The typical rawhide experience.
In the past, hitting such a bad compose meant a painful struggle to recover: Boot into single user-mode, do some live debugging, try to find a few problematic packages to downgrade, work around breakage, etc. You end up with a half-working system in a mixed state, and hope that the next update will get you back on track.
An ostree-based system like Fedora Atomic Workstation makes the recovery painless. In the case I mentioned, I simply rebooted, selected my previous, working image from the grub menu, and was back to a working system in 3 minutes. No time lost.
The tree that didn’t work is still present on my system. If I wanted, I could make changes to it in an attempt to understand and fix the problem and try booting it again. Having both trees on the system is useful even if I just want to report the problem, since rpm-ostree can tell me what exactly changed between the working and the broken tree:
$rpm-ostree db diff d15a65950972 21ac24694b6e ostree diff commit old: d15a65950972 ostree diff commit new: 21ac24694b6e Upgraded: apr 1.6.3-5.fc28 -> 1.6.3-6.fc29 atomic 1.21.1-1.fc28 -> 1.22.1-1.fc29 ... Removed: NetworkManager-glib-1:1.10.2-1.fc28.x86_64 libnm-gtk-1.8.10-2.fc28.2.x86_64 ... Added: authselect-0.3-3.fc29.x86_64 authselect-libs-0.3-3.fc29.x86_64 ...
(I found the checksums to pass to rpm-ostree db diff by looking for the Commit fields in the output of rpm-ostree status)
Or I can simply wait for the next compose to see if rawhide ‘fixed itself’. Even if it takes more than one compose before things get back to working, this is a safe thing to do: ostree will never replace the current image that I am booted into. This makes it entirely safe to try newer composes until I find one that works for me.
It would of course be nice if broken composes never made it to my system. The current efforts at ‘gating’ updates in Fedora should get us to a place where we detect non-booting composes before they get sent out to users. But even in this (hopefully not too distant) future, the easy rollback with Atomic Workstation provides a useful safety net.
Automated QA will never be able to detect every problem that could affect my workflow. Maybe the system boots fine, but frequently loses network connections, or maybe it can’t handle my dual-monitor setup, or something else… I may only discover the problem hours later. But up until I run rpm-ostree upgrade again, I can just reboot back into my previous image.
ostree provides fearless upgrades.