File operations in nautilus-gio and adventures in the land of PolicyKit

This week I’ve been working on adding file operations to the gio-branch of nautilus. Today I finished enough to hook up a mostly finished copy operation into the UI. This is completely new code, since gio doesn’t have the gnome_vfs_xfer API that implemented complicated file operations in gnome-vfs.

So, while this is doing the same thing as the old code its structured in a completely different way which is much nicer to work with. And along the way I made some other changes that was easy to do with the new improved design. Check out this screencast of the new features:

nautilus-gio-copy.png

Next week I will continue working on the rest of the file operations. It will be a lot easier now that the general structure and a lot of common code is written.

Also, next week I will be starting to merge libgio from the gio-standalone module into the glib module (as a separate libgio-2.0.so). The gio code has all features required by nautilus (as we can see from the almost finished port), and has been pretty API and ABI stable for a while. So, this is a good time to merge it and get more people looking at the code and using it in their applications.

PolicyKit vs Nautilus

But all that is not whats got me most excited right now. Instead its this idea I got about using PolicyKit and gio in order to implement system administrator features in Nautilus. Basically, you’d do a file operation on some system file, which gives you an error dialog saying you don’t have permissions to do this. But the error dialog has a button that lets you authenticate (with e.g. the root password) and continue the operation with root rights.

This is quite simple to implement in nautilus-gio, because all I/O operations go through the GFile interface. You just create an implementation of GFile that proxies all operations to a setuid helper program (which uses PolicyKit to authenticate). With this done, all the Nautilus code will work unmodified with this new backend, all you have to do is make sure it uses this new GFile implementation.

This is much better than running your entire Nautilus process (or even desktop) as root, for multiple reasons. First of all your Nautilus application will integrate nicely with your desktop (using your theme/prefs/etc instead of the root ones). Secondly a lot less code is running as root, which means that the risk for a security breach or accidents as root are much smaller. Third, you can configure this to use sudo-style authentication which means you don’t need to give out the root password (or even have one) .

Still, this work will have to wait, as the main priority right now is getting the gio-branch into a useful shape so that it can be merged into svn HEAD. I think this will happen soon. Maybe next week even.

37 Responses to “File operations in nautilus-gio and adventures in the land of PolicyKit”

  1. dr88dr88 says:

    PolicyKit vs Nautilus i have waited years for this future. Will this future be in the next version of nautilus.

  2. alexl says:

    I can’t guarantee it will be in Gnome 2.22. Fixing all regressions and getting gio into shape has priority. However, hopefully there will be time for it.

  3. [...] questo è molto promettente, qui trovate il post originale di Alexander, con il video a piena risoluzione [...]

  4. Stu says:

    I should really submit this as a feature request, but since your on file copying… I’ve always thought the file copying should all be in one dialogue (a bit like the firefox download box)

    copying
    file 1 [||||| 20% ]
    file 2 [|| 5% ]

    queued files |>

    It could be intelligent about device speed too…

    There could be some sort of intelligent queuing involved –

    If I copy 100 files from a cd to the disk, and in the middle drag another one over, it could wait until the current file has finished copying then copy the one just dragged as the next one in the queue.

    Also, if copying between say my floppy and usb stick, these wouldn’t have to be queued, just because I’m doing a big copy between two hard disks.

  5. Paul says:

    Great can’t wait to test and help with bug-hunting :)
    Alex, is these awesome status-icon intended to be something like Methusalem? Can other apps like cd-burner, epiphany or tracker use these API to plug in their long-running-tasks in?

    Thx

  6. meastp says:

    Will we have undo for file operations with gio?

  7. zith says:

    Looks great!

    How about having some kind of dynamic icon for files being transferred that would show the user that the file is still not complete, or even some kind of progressbar in the icon itself? This could also be used for epiphanys downloader and various bittorrent client or whatnot. I think it would add a cool touch.

  8. Xav says:

    The notification icon and progress drialog remind me a lot of http://live.gnome.org/Mathusalem – which I like very much. Would it be possible to either:
    – use Mathusalem ?
    – replace Mathusalem (i.e. make the dialog usable by other apps) ?

    Thanks,
    Xav

  9. Rui says:

    Hey, very nice the shared copy files progress dialog!

    But I’d like to suggest that if the source physical disk or the destiny is the same it would be nice to queue the operation than do it in parallel since it would be faster due to reduced seeking. Ofcourse this should only apply only to HDDs and likewise physical devices. USB mem sticks, for instance, should be able to handle the parallel operations without degraded performance. Maybe this could be provided as an hint by the gio API so that the final decision is made by the toplevel application (nautilus in this case).

  10. .fosk. says:

    Great!!!

    Amazing work!
    Cant wait for the posibilities it opens!

  11. Rafael Jannone says:

    This is awesome! :D
    Congratulations and keep up with the good work :)

  12. [...]En este caso, Alexander Larsson, desarrollador de Gnome, está trabajando desde hace tiempo en Gio[...]

  13. loupgaroublond says:

    That shared copy dialog just made me very happy.

  14. Xav says:

    BTW, if you ever want a screenshot of Mathusalem in action:
    http://tw.apinc.org/weblog/2006/08/13#mathusalem-gets-useful
    I really think this thing would be awesome if it was well-integrated in the desktop.

    Xav

  15. alexl says:

    Lots of questions… I’ll try to reply.

    The status icon right now is just something internal to nautilus to make it work. However, the implementation has a clean cut using a ProgressInfo object that connects the operations to the progress display. This could easily be connected to Mathusalem, or similar approaches in the future.

    Bunch of feature requests… Some good ideas, some are hard to implement. Right now I’m focusing on getting back to the pre-gio state, and to fix bugs introduced.

    Specifically about undo. There is someone working on that, and it seems to be progressing. Its a lot of work though to get a solid implementation of it though, so we’ll have to see when is finished.

  16. Sven says:

    Queued file transfers, as requested here several times already, is also the main feature request I have for nautilus.

  17. Gianni says:

    Feature request : copy a folder without the hidden files/folders in it (I think it needs to edit a lot of code under the hood, but still, maybe it’s already implemented somewhere and I just don’t know how to do it)

    Question : is this related to gnomevfs modification ?

    Nautilus has never been so much improved. Keep the good work.

  18. Anton says:

    Awesome work!
    Will this code also result in ACL support in Gnome when you are finished th port or is that still a long way off?

  19. Yes, queued file transfer definitely would be a killer feature putting Nautilus far ahead of other widely used file managers.

  20. Please check out http://code.google.com/p/libproxy/ for your proxy implementation in gio. We could always use some testing and (after our 0.2 release) we are going to focus on performance.

  21. Simon Holm Thøgersen says:

    Nice work, but I guess there needs to be cancel of transfer button somewhere?

  22. This feature is similar to how Windows Vista does it these days with UAP enabled. If you don’t have rights to do a file operation, it’ll prompt for the administrator password to do it. I’d really like this feature in GNOME as well!

  23. Jerome Haltom says:

    So the basic idea is that only the side of the process which NEEDS to run as an elevated user will? If I copy from ~ to /etc, I’d expect the source side to run as me, and the destination side to run elevated.

  24. Martin says:

    Queued file transfers? Why on earth would you want that? :)

  25. alexl says:

    Yes, clearly the progress dialog needs a cancel button. Thats just not implemented yet.

    Nathaniel: Thats another type of proxy (http proxy). We don’t have an http backend in gvfs yet, but the one we’re working on will use soup, so it will get proxies from that. Not sure what it uses.

    ACL support is quite possible, as all the metadata like ACLs can be nicely accessed via gio. But it still needs coding.

    Queued transfers could be added without *that* much work I guess. I’m a little surprised that people see them as so important though. What do you want them for?

  26. Björn says:

    I’d like to know, if the move to gio also reduces startup time and memory consumption of nautilus.

  27. Xav says:

    Queued transfers are of great psychological help when transferring many big files over a slow link. Try using WinSCP to transfer a bunch of movies from your linux box to a friends windows box over an ADSL link for example. If you ask for a copy one by one, if they all copy without queuing the individual progress is tied to the global progress, and you’re under the impression it’s really slow. But if they are queued, each one will transfer faster (even if globally it’s the same) and you can even watch one before the rest is finished. In WinSCP you can choose to queue and it helps.

    Personnally I can live without that, but I have whished it existed more than once. Sometimes little “psychological help”, like the drawer icon opening when dropping a file onto them, render an app much more usable.

  28. > Queued transfers could be added without *that* much work I guess. I’m a little surprised that people see them as so important though. What do you want them for?

    File transfers only are fast, when you avoid disk seeks. That’s somewhat true for hard-disk transfers, but even more true, when copying data from CD/DVD. Interestingly the problem doesn’t show up in your demo. Probably since you copy many small files. Well, but when copying large files I’ve observed overall throughput dropping to 50% and less on harddisks, but more dramatically to just 10% on optical media for just copying two large files in parallel.

    So the queue-rule should go like this: Only one active file-transfer per physical device. Guess fetching from distant HTTP and (S)FTP servers doesn’t fall under this rule, as for such transfers network bandwidth, not disk seek time is the limiting factor.

  29. Oh, seems Xav wrote at the same time like me, so I didn’t see him mentioning the psychological aspect justify queue also for network transfers. Seems the queue heuristic for network transfers needs some more finetuning yet. Also like the local queue heuristic could be tuned (many small vs. few large files). So I guess in terms of code the queue policy should get some good encapsulation for better fine tuning over the time.

  30. alexl says:

    Björn: Long term, yes, since we can drop a bunch of libraries, etc.

    However, short-term we still link to these libraries via some dependencies (like libgnomeui). As project ridlrey continues we’ll soon reach a point where we can dump a lot of crap though.

  31. Frej Soya says:

    Isn’t the transfer status (the small bar) in the WM nautilus folder icon good enough?
    Instead of having a seperate ‘progress notification icon+window’ then it’s just the folder icon (in top corner) that shows progress?

    Just trying to keep stuff out of notification area. It’s not that important :)

  32. pirast says:

    i had that idea some time ago, also :) see http://bugzilla.gnome.org/show_bug.cgi?id=474261

  33. alex r says:

    On the subject of a cancel button: the dialog really needs a pause button. That way, if you are uploading a large file over a slow connection, you can temporarily pause it, do some useful work, and then start it going again. Pausing might not make sense for local file transfers, but for large transfers over the network, it is absolutely essential.

  34. alexl says:

    A Pause button seems like an interesting idea too

  35. amano says:

    *woot*
    Can’t wait for this sudo button on the error dialog. I hope that this will make it into 2.22. A feature that many have called for: http://ubuntuforums.org/showthread.php?t=588243

    Great that these things seem to be technically possible right now.

  36. Vadim P. says:

    Thank you so much for that policykit thing!

  37. thanks for this initiative.I likes Nautilus FM.please make it load faster :D