So, after my epic 20+ minute offline update of 245 packages, last night I decided to look at some profiling numbers. All my testing was done using git master PackageKit (for the new strace support) on an otherwise unmodified Fedora 20 of a snapshot from last week. For the strace I chose to update two packages, otherwise the strace -tt output went maaaasive. Some salient points:
- yum opens and closes the rpmdb 6692 times (that’s about 6690 more than it needs to) –we’re investigating why
- fdatasync and fsync are killing us:
| duration(ms) | system call |
| 805.749 | fdatasync(17) |
| 752.828 | fsync(27) |
| 658.659 | fdatasync(9) |
| 614.367 | fdatasync(15) |
| 598.182 | fdatasync(33) |
| 535.642 | wait4(903, [{WIFEXITED(s) && WEXITSTATUS(s) = |
| 423.247 | wait4(911, [{WIFEXITED(s) && WEXITSTATUS(s) = |
| 368.85 | fsync(22) |
| 309.556 | stat(“/var/lib/yum/yumdb/g/gvfs-fuse-1.18.3-1.fc20-x86_64/checksum_type” |
| 217.877 | fdatasync(18) |
| 179.002 | close(23) |
The full strace log is here (warning, huge) if you’re interested. I’ve got some other work to be doing today, but I’ll continue to work on this at the weekend.
Shocker! Yum is shit.
just troling here, but, apt-get/dpkg is file (txt) based. Rpm is sqlite based right? So it should be … a LOT faster, right?
One thing I’ve often wondered is why package managers don’t do the write-to-temp-file-sync-and-rename trick for entire updates.
Write every file in a update/new package to temp file names in the proper directories, sync(2), and then rename. The only danger point then for a crash would be midway through the rename, which you could easily fix early in boot process. This way we’d get completely crash safe updates too – on *any* file system.
Thanks a bunch for working on this. Hope you can get a freeze exception for anything that makes this faster, otherwise the first update after installing F20 will be really horrible.
> yum opens and closes the rpmdb 6692 times (that’s about 6690 more than it needs to) –we’re investigating why
That’s pretty bad. :-/
Any idea if dnf (mis)behaves similarly?
Thank you for the work you’re doing on this by the way. This is some important work to speed up our tools, from which everybody will benefit, even those who don’t use the offline updates. :-)
I’m no more yum lover than the next guy, but to straighten the impression one gets here a bit…
a) The crazy rpmdb reopens only occur when using a specific yum API feature (automatic rpmdb closing after query) in a way it was not intended to be used. This is specific to PackageKit, offline or online (but has been fixed in PK upstream now).
b) Yes fdatasync() is expensive, you can blame rpm for trying to keep your system data safe ;) That has nothing to do with yum, or offline-updates being slower than online updates.
The bottom line is that there’s a huge slowdown in rpm transactions that is specific to the offline-updates application. I asked for a strace to see if there’s something unusual going, such as constant timeouts due to missing services in the minimal environment. There are no such things happening, only lots of “unexpected” gettimeofday() calls (perhaps from glib mainloop) and progress-report communication over stdout. I’m fairly positive the slowdown is related to the progress-reporting, and that the same issue is present with the hawkey backend as well. I’d suggest simply testing with progress reporting disabled entirely – if (and I’d go so far as saying when) it then runs with similar speed to regular yum cli transactions, there you have it. Worrying about rpm’s fdatasync() and such is pointless as long as there’s a factor of two slowdown that is specific to the offline-updates application.