As others mentiones, create an online service with a well-defined API to both download and upload (merge) distro-specific data. As for the offline clients (we won’t really care about these) make it possible for devs to download an SQLite 3 snapshot with packages just for that single distro. Icons are not important – if there’s no network link you have bigger concerns than missing OpenOffice icons in the installer.

I see hughsie updated his numbers to 20Mb for the icons. Downloading this on the fly will take approximately 5 minutes on a fairly common 0.5 Mbit connection or easily an hour on dial-up. Not really feasible imo, even if you do something clever about the order in which you download these icons. This needs to appear instant to users.

If the initial 20Mb of icons are pre-cached in a package (possible to remove if you don’t want them), then updating them every time there is a new package online is not a problem. Even in the most extreme case that you install Fedora 9 now and have 1.5 years of package additions to update for it will still only be about 4Mb. This can download in the background while you start the service, filling in the blanks as you go along. It should only take 1 minute on a 0.5 Mbit connection, or not much more than 10 minutes on dialup.

I wanted to avoid wrapping this in XML, this has to be quick to insert into a database. You are correct I’ll have to define the encoding and stuff, but that should be pretty simple.

About the mirror choke point. I’m not expecting to update this more frequently than every 6 months after a stable distro release like F10. For distros like rawhide it’ll be more often, but that’s not unusual.

Also, the 80Mb was calculated having 4 installed, as the actual data in each repo is only about 20Mb in size.

A per package approach is a really good way to go. Besides, the current approach is really not working for everyone:
https://bugzilla.redhat.com/show_bug.cgi?id=487614