Ignacy Kuchciński's Blog

There are many things that we take for granted in this world, and one of them is undoubtedly the ability to clean up your files – imagine a world where you can’t just throw all those disk space hungry things that you no longer find useful. Though that might sound impossible, turns out some people have encountered a particularly interesting bug, that resulted in silent sweeping the Trash under the rug instead of emptying it in Nautilus. Since I was blessed to run into that issue myself, I decided to fix it and shed some light on the fun.

Trash after emptying in Nautilus, are the files really gone?

It all started with a 2009 Ubuntu launchpad ticket, reported against Nautilus. The user found 70 GB worth of files using disk analyzer in the ~/.local/share/Trash/expunged directory, even though they had emptied it with graphical interface. They did realize the offending files belonged to another user, however, they couldn’t reproduce it easily at first. After all, when you try to move to trash a file or a directory not belonging to you, you would usually be correctly informed that you don’t have necessary permissions, and perhaps even offer to permanently delete them instead. So what was so special about this case?

First let’s get a better view of when we can and when we can’t permanently delete files, something that is done at the end of a successful trash emptying operation. We’ll focus only on the owners of relevant files, since other factors, such as file read/write/execute permissions, can be adjusted freely by their owners, and that’s what trash implementations will do for you. Here are cases where you CAN delete files:

– when a file is in a directory owned by you, you can always delete it
– when a directory is in a directory owned by you and it’s owned by you, you can obviously delete it
– when a directory is in a directory owned by you but you don’t own it, and it’s empty, you can surprisingly delete it as well

So to summarize, no matter who the owner of the file or a directory is, if it’s in a directory owned by you, you can get rid of it. There is one exception to this – the directory must be empty, otherwise, you will not be able to remove neither it, nor its including files. Which takes us to an analogous list for cases where you CANNOT delete files:

– when a directory is in a directory owned by you but you don’t own it, and it’s not empty, you can’t delete it.
– when a file is in a directory NOT owned by you, you can’t delete it
– when a directory is in a directory NOT owned by you, you can’t delete it either

In contrast with removing files in a directory you own, when you are not the owner of the parent directory, you cannot delete any of the child files and directories, without exceptions. This is actually the reason for the one case where you can’t remove something from a directory you own – to remove a non-empty directory, first you need to recursively delete all of its including files and directories, and you can’t do that if the directory is not owned by you.

Now let’s look inside the trash can, or rather how it functions – the reason for separating permanently deleting and trashing operations, is obvious – users are expected to change their mind and be able to get their files back on a whim, so there’s a need for a middle step. That’s where the Trash specification comes, providing a common way in which all “Trash can” implementation should store, list, and restore trashed files, even across different filesystems – Nautilus Trash feature is one of the possible implementations. The way the trashing works is actually moving files to the $XDG_DATA_HOME/Trash/files directory and setting up some metadata to track their original location, to be able to restore them if needed. Only when the user empties the trash, are they actually deleted. If it’s all about moving files, specifically outside their previous parent directory (i.e. to Trash), let’s look at cases where you CAN move files:

– when a file is in a directory owned by you, you can move it
– when a directory is in a directory owned by you and you own it, you can obviously move it

We can see that the only exception when moving files in a directory you own, is when the directory you’re moving doesn’t belong to you, in which case you will be correctly informed you don’t have permissions. In the remaining cases, users are able to move files and therefore trash them. Now what about the cases where you CANNOT move files?

– when a directory is in a directory owned by you but you don’t own it, you can’t move it
– when a file is in a directory NOT owned by you, you can’t move it either
– when a directory is in a directory NOT owned by you, you still can’t move it

In those cases Nautilus will either not expose the ability to trash files, or will tell user about the error, and the system is working well – even if moving them was possible, permanently deleting files in a directory not owned by you is not supported anyway.

So, where’s the catch? What are we missing? We’ve got two different operations that can succeed or fail given different circumstances, moving (trashing) and deleting. We need to find a situation, where moving a file is possible, and such overlap exists, by chaining the following two rules:

– when a directory A is in a directory owned by you and it’s owned by you, you can obviously move it
– when a directory B is in a directory A owned by you but you don’t own it, and it’s not empty, you can’t delete it.

So a simple way to reproduce was found, precisely:

mkdir -p test/root
touch test/root/file
sudo chown root:root test/root

Afterwards trashing and emptying in Nautilus or gio trash command will result in the files not being deleted, and left in the ~/.local/share/Trash/expunged, which is used by the gvfsd-trash as an intermediary during emptying operation. The situations where that can happen are very rare, but they do exist – personally I have encountered this when manually cleaning container files created by podman in ~/.local/share/containers, which I arguably I shouldn’t be doing in the first place, and rather leave it up to the podman itself. Nevertheless, it’s still possible from the user perspective, and should be handled and prevented correctly. That’s exactly what was done, a ticket was submitted and moved to appropriate place, which turned out to be glib itself, and I have submitted a MR that was merged – now both Nautilus and gio trash will recursively check for this case, and prevent you from doing this. You can expect it in the next glib release 2.85.1.

On the ending notes I want to thank the glib maintainer Philip Withnall who has walked me through on the required changes and reviewed them, and ask you one thing: is your ~/.local/share/Trash/expunged really empty? 🙂

Introduction

Throughout this summer I’ve been working on making the New Documents feature discoverable in Nautilus, a file manager for GNOME as part of the GSoC project. This post is an overview with links of the work I did together with my mentor Antonio Fernandes.

Results

For the project I was supposed to resolve the discoverability problem of this feature – when there are no templates in the Templates directory, the new document menu is not shown, and many users don’t know about its existence. Below is a list of steps I’ve taken to fulfill that quest:

 

1. Planning – first we had to establish a schedule for our work
2. Research – then we had to do some research about how others implement this feature
3. Design – afterwards we had to design a mockup for the improved feature
4. Code – at last a prototype was made, and after several design iterations, a final MR was submitted and merged to a feature branch (not master yet), during that phase I also presented at GUADEC

 

Future

While the short term solution is implemented in a feature branch, it’s still not a final one – we need to keep iterating on the design, perform usability testing, get more feedback, before we implement it in master branch and it can reach the users. There is also a long term plan detailed in the mockups – it’s an ambitious one that requires changes across the app ecosystem, but some day the work towards it needs to start, and I’m going to aim towards it. I’m also planning on keeping contributing to Files, as well as other core GNOME apps written in C.

Conclusion

This GSoC project has been an amazing journey – I expanded my knowledge of C, Glib, GTK, Libadwaita, and the internals of Files. I also learned how to do proper research with Boxes, use Inkscape for creating mockups, ask for designer feedback, and how to use Builder to create magic.

I would like to thank my mentor Antonio Fernandes for answering my questions and guiding me through the project, as well as Tobias Bernard, Allan Day and Michaël Bertaux for designer feedback.

Introduction

I published my last blog post about the design phase of my GSoC project on 17 July. It’s been a month, and it’s been far from uneventful. I’m going to talk about the first prototype and my short presentation at GUADEC, as well as the review from designers and the result of it – the new mockup and the second prototype.

First prototype and GUADEC

Following the old mockup from my last update, I prepared the first prototype of the revamped New File menu; it wasn’t really functional but served its main purpose as something that the designers at mini-GUADEC could test and review. Speaking of GUADEC, I took part in the Intern Lightning Talks where I presented the old mockup – you can watch my presentation on YouTube.

Me presenting the old mockup at GUADEC Intern Lightning Talks

Review at mini-GUADEC

Equipped with the old mockup and the first prototype, my mentor Antonio Fernandes presented the work at mini-GUADEC satelite event to the designers, and we got following feedback:

 

• Templates functionality in the file manager is an exotic feature – as the feature matrix from the research phase shows, no other file browser is doing it (except elementary). At the same time, completely removing this feature was discarded because some people rely on it a lot, so the consensus was to keep it but not promote it too much, which means dropping the “no templates” empty state
• The “+” button in the headerbar was discarded
• The ideal situation would be for every editor app to accept a path to a new file and take care of everything. Then, in Files, when creating a new file in a folder, we would only have to ask the user which app they wanted to create the new file with. However, this requires changes across the ecosystem, so it’s not a short-term plan

 

New mockup and second prototype

After additional discussions in the project chatroom, we’ve come up with the following new mockup adhering to the feedback:

For more information check out the Whiteboards issue.

Here are the screenshots of the second prototype (merge request) implementing the above mockup:

Conclusion

During this month we’ve iterated on the mockup and made two prototypes, I presented the project at GUADEC, and even made my first review in an effort to help with the Nautilus beta release – I’m very pleased with how the project is going and how it helped me find my place in the GNOME community 🙂 As for what’s next, the MR needs to be reviewed and we’ve got usability testings to do, so hang tight until the future updates!

Introduction

It’s been a while since my last update, in which I’ve shared my research about the underlying problem and use cases of the “New Document” feature, regarding its discoverability and ease of use, as part of my Nautilus GSoC Project. Since then I’ve been focusing on the following phase of the project: “Design a mockup based on aforementioned research”, and I’m here today to share the results with you.

Use cases

Figuring out hypothetical user profiles and their use cases could prove really helpful in designing and refining prototypes, so together with my mentor Antonio Fernandes we’ve come up with the following examples:

User profile 1 – Programmer

A programmer or system administrator with a repertoire of many pre-made code snippets/basic scripts for a variety of needs.

Use case: has to create source/script files at specific locations with the premade content.

User profile 2 – Computer science student

An aspiring programmer, who attends classes where they learn about the  basics of programming or operating systems.

Use case: writes simple code in text editors, and needs to create many empty text files with a specific extension, i.e. “.c”, in specific directories representing classes, i.e. fundamentals-of-programming/lesson-1

User profile 3 – Teacher

A teacher at school with a collection of many different materials for different classes and groups of students.

Use case: has to manage worksheets, documents and presentations in order of subjects, classes or groups of students; and accomplish that by creating directories with a topic name, and creating empty documents in that directory which they immediately want to edit.

Discussing the templates layout

While we were pondering over how to best tackle the discoverability of the “New Documents” feature in the design chat room, one interesting idea strongly stood out – changing the implementation of templates so that they have a layered structure (in the same spirit as systemd unit files), and so not only users could be able to create them, but also distributions, apps, and even administrators:

 

• /usr/share/xdg/templates – distro/app packaging, when applications install template files on the system, this is the location where they are placed by default
• /etc/xdg/templates – site customization, templates found in this directory take precedence over any of the other locations on the filesystem, this is where administrators create new templates, but also where they can create ones that override those provided by applications
• $HOME/.Templates, or $HOME/.local/share/xdg/templates, or $HOME/.local/ – basically, nautilus should ignore XDG_TEMPLATES_DIR (Templates), and mark it as deprecated

 

One would of course need to figure out how to handle sandboxed applications in that case, so the overall concept is definitely a long term goal.

Mockup

After considering amount of discussion and pointers gained at the gnome-design chat room and the tremendously helpful input from Michael Bertaux in the whiteboard issue, we’ve currently developed the following mockup:

Do note that it’s by no means final – it could be iterated on many times after I prepare a code prototype or perform the usability testing, etc. I’ve actually presented two ideas there, one building up on the other – the top half part of the mockup is the short term plan, that I aim to implement during the project, and the bottom half shows the long term plan, that could be a follow up in the future. There are many things happening even considering the short term idea, so I’ll try to go over some of the most important ones:

Combining “New Folder” and “New Document” features in one single “New” menu

The research showed that almost all of the implementations moved the “New Folder” entry into a “New” submenu, which makes sense, considering there are often more features in that menu, like “New Link” (or even “New Zip” in one case), which would otherwise clutter the main context menu. So I followed that suggestion in my mockup, and in the image of the web apps, added a “+” button on the header bar, to make sure that the number of mouse clicks needed to access the “New Folder” feature stays the same, for it to become even more discoverable.

Default templates for most common types of files

Another conclusion we’ve drawn from the research is that the user may expect to be able to create new files without configuring anything prior themself. With that in mind, we would show special, default, empty file types in the menu if the user has appropriate applications installed – a text editor, and office suite.

Custom templates dialog

In order to fix the discoverability problem of the custom templates feature, we’ve come up with a dialog with a helpful empty state, in which users can easily import new templates. Such templates can also be searched, and we’re planning to extend the mockup to include the “edit” mode that will allow for removing or editing them as well. Each template would have an icon based on its MIME file type, and users would be able to instantly rename them after or when they  create them (same for the common empty file types).

Long term vision

Considering future expansion, the design proposal includes the app templates, which allows for more than just the text editor and office suite to provide empty templates for each file type. There’s also the matter of custom non-empty app templates, but we’re currently wondering whether that’s something to be handled only on the applications side.

Adjusting the timeline

Since most of the design pointers exchange has been asynchronous, the design process took a little longer than we expected – that’s why we’ve updated the timeline of the project:

 

1. Research the underlying problem and use cases (3 weeks) – 12.06-03.07
2. Design a mockup based on aforementioned research (½ week) – 03.07-07.07
3. Code prototype iteration in a development branch (2 weeks) – 07.07-19.07
4. Test and review the prototype iteration (1½ week) – 20.07-31.07
5. Open a Merge Request to merge the development branch to the master branch (4 days) – 31.07-04.08

 

I’ve neglected the blog a little though among all of the tasks, as I’m already in the “code prototype” phase, but better late than never :’D

Conclusion

Thanks to the research and help we got both in the design chat room as well as the Whiteboard issue, we’ve managed to prepare a solid mockup, part of which I’ll implement in the next (current) stage of the project. We’ve also prepared some user profiles that will be helpful in the future review, discussed changing the layout of templates overall, and adjusted the schedule. Now it’s finally time to write some code, see you in the next update 🙂

Introduction

Two weeks have passed since I started the first phase of my “Make New Documents feature discoverable” Nautilus GSoC project. It’s called “Researching the underlying problem and use cases”, and according to the timeline that was set up in my last planning post, I’m here to share our findings and results.

Why do the research

Before we start revamping/fixing issues with our implementation of the “New Document” feature, it’s essential that we look into other operating systems, file managers, and web apps, to see how they approach allowing the user to create files. Do not be mistaken – the intention is not to blindly copy them, but to take inspiration from them, identify potential problems with specific solutions, and find out what users may expect from us. If there’s a clear trend in some approach, there may be a valid reason to implement it. We don’t exist in a vacuum, and we are definitely able to learn from the accomplishments or mistakes of others.. Everybody, in some way or the other, builds up on their predecessors work, and continues the everlasting chain of inspiration and creation. While doing the research one needs to take into account the different types of problems they are trying to solve and specific kinds of users that they need to focus on.

List of software to test

And so I put on my spying disguise and began my journey through the multiverse of different Operating system’s file managers and web apps:

• Other Operating Systems
  • Windows 11 File Explorer
  • macOS 12 Finder
  • ChromeOS Files
• Other Linux distributions
  • KDE’s Dolphin
  • Deepin File Manager
  • elementaryOS Files
• Web apps
  • Google Drive
  • Dropbox
  • Microsoft Onedrive
  • Apple iCloud
  • Nextcloud
  • GitLab file tree web UI  — templates are offered by the text editor after creating new file

Some of them, like macOS 12 Finder, Apple iCloud and CrOS don’t have the “New Document” feature at all – they rely on an application-centric model, therefore they are ignored in the rest of the post.

Enter the matrix, (KDE) Neo(n)!

Putting a stop to the movie references here – we’ve quickly identified several common features across different implementations, and to avoid repetition, we’ve summarized them in the following matrix:

Pretty pictures

If all you wanted from this post are fancy screenshots, here are the goodies. I’ve divided the screenshots not by the implementation itself, but rather by the features I’ll be referring to.

“New” menu containing a “New Folder” submenu

All of the tested implementations except for Deepin Files moved the “New Folder” entry into a “New” submenu, which could be something we might consider doing as well. If we move the “New Folder” item into a “New” menu, we should also add a [ + ] menu button to the headerbar, since all of the web apps have one.

Windows 11 File Explorer groups the “New Folder” feature together with creating other files in one single “New” menu. Among others, it also allows creating empty zip archives no-questions-asked, perhaps because they are treated as normal folders, i.e. user can double click and “enter” them in the file explorer.

elementaryOS Files “New” menu which allows creating folders

KDE Dolphin “Create New” feature which offers “New Folder” option, among others

Gitlab file tree web UI also groups “New File” with “New directory” in one specific “+” menu, as all of the other web solutions do.

Links

KDE Neon 5.25 Dolphin, Windows 11 File Explorer and Dropbox add a “New Link” entry to the “New” menu which allows you to create a link to a file. Nautilus doesn’t even show the option to create links by default, perhaps this way of creating links is more intuitive than copy->paste link. It could be interesting to consider adding it to the “New” menu, since there may be a discoverability problem with links as well. It would require a whole new dialog, so unfortunately it’s out of scope of this project.

KDE Neon 5.25 Dolphin allows creating links from the “Create New” menu.

Windows 11 File Explorer opens a link wizard after selecting the “Create Shortcut” option.

Default entries

One of the conclusions is that the new text files and new office documents are both common features users can fairly expect us to have.

Deepin Files offering default templates for office documents and plain text files

Google Drive also creates a shortcut to each editor application’s “choose template” functionality, and there are even more default templates in the “more” submenu

OneDrive showing default templates in its specific format

Dropbox allowing to choose a specific format when selecting a general default file type

Nextcloud showing many default templates

Templates directory

Deepin 20 File Manager preserves the Templates directory functionality, but hides the directory as .Templates, while KDE Neon 5.25 Dolphin ignores the Templates directory altogether and requires creating .desktop files in .local/share/templates directory pointing to the templates, wherever they are. Although there are implementations using it, the XDG Templates directory seems not to be that much of a standard after all – XDG_DIR_TEMPLATES is a standard location, but it doesn’t look like it mandates any special behaviour at all. We are free to change how the Templates integration works!

elementaryOS Files uses the XDG Templates directory to allow the user to add more “New File” entries, just like GNOME Files

Deepin Files hides the Templates (.Templates) directory, but the functionality remains.

KDE Dolphin ignores the XDG Templates directory altogether, requiring the user to create .desktop files in .local/share/templates directory pointing to the templates.

Adding more app templates

Google Drive allows the user to add more items than the default ones by guiding users through the installation of apps, which after installing add the templates to the “New” menu, which is something we could consider doing as well.

Renaming on creation

All of the implementations (except for Google Drive and Dropbox, which open the document handling app) immediately allow the user to rename files after creation, except for Nautilus. It’s outside of the scope of this project, but important to note nonetheless.

elementaryOS Files and Deeping Files allowing the user to immediately rename the created file

KDE Dolphin opening a dialog after creating a file in which user can change the filename

Immediately opening new files in the app

All of the web apps immediately open new files in their appropriate editor applications, while all of the non-web apps don’t.

Google Docs allowing the user to choose from different templates while creating a new file

Nextcloud allowing the user to choose from different templates or a blank file before opening it in a web app

Conclusion

The research highlighted many interesting things, including how unique GNOME Files is with not grouping the “New File” and “New Directory” features, how popular it is to include default templates for office and text files in other implementations, how XDG Templates “standard” is neglected, how often the other implementations allow for renaming the files on creation or how consistent web solutions are with providing a “+” button. There’s also the matter of discoverability of the “New Link” feature, but it’s out of the scope for my project. The next phase is “Designing a mockup based on aforementioned research”, which includes asking the design team for pointers – that’s what I’ll be doing now, so I can prepare a design later. I also want to tremendously thank my mentor Antonio Fernandes, who’s credited for the awesome feature matrix, and most of the conclusions from the research. See you in the next update 🙂

Introduction

This summer I’m contributing to Nautilus as part of GSoC, focusing on improving the discoverability of the new document feature. In this post I will describe how the project was split between me and Utkarsh, briefly go over the schedule established for my work, and briefly mention my current research in GNOME Boxes.

The split

The initial short project idea assumed that only one student was going to work on it, so when both me and Utkarsh Gandhi were accepted, we had quite an unexpected situation. Fortunately, the fact that the project had many stretch goals allowed us to split it so that both of us can work independently. The unexpected situation has taught us to share tasks in a meaningful way, which has made us all the more able to grow at our assignments, furthermore we have learned how to work without blocking each other’s progress. Most of the initial tasks that aim to revamp the UI and the code of the New Document menu go to Utkarsh, while I’m going to focus on the discoverability side and the default experience of the user, meaning what happens when there are no templates in the Templates directory.

Make “New Documents” feature discoverable

Finally, the subject of my project turned out to be about resolving the accessibility issue of this feature – when there are no templates in the Templates directory, the new document menu is not shown, and many users don’t know about its existence. They completely ignore the Templates directory, not knowing what it does and just assuming it’s one of those “just there” directories and files. Another thing I’ll take a look at is the ability to easily add templates, without the cumbersome process of creating and copying files. I’ll also reconsider the pros and cons of default templates.

Timeline

While it’s not final, because we’ve lost our crystal balls, here’s the current anticipated schedule I’ll be following:

1. Research the underlying problem and use cases by looking at other implementations (operating systems, file managers, web apps) (2 weeks) – 12.06-26.06 (current)
2. Design a mockup based on above research, adhering to GNOME HIG and designers review (1 week) – 26.06-03.07
3. Code prototype iteration in a development branch that provides a meaningful empty state, makes sure the “new documents” menu item is always shown, and the user can add more templates (2 weeks) – 03.07-17.07
4. Test and review the prototype iteration, refine the prototype based on feedback and repeat if necessary (2 weeks) – 17.07-31.07
5. Open a Merge Request to merge the development branch to the master branch (4 days) – 31.07-04.08

Beginning of research

As I have started the first point of my timeline/schedule, I found myself in need of many virtual machines. Equipped with powerful yet simple and elegant GNOME Boxes, I managed to run 2 different operating systems on it:

• ChromeOS Flex – knowledge gained on how to work with libvirt xml files allowed me to figure out how to get this web centric system running in Boxes. I have documented the necessary steps in this guide, but it definitely deserves a separate blog post.
• Windows 11 thanks to this excellent guide.

Next to try are file managers from other linuxes, and web apps; already tested macOS Finder. My next GSoC update definitely won’t lack colorful pictures.

Conclusion

The project is coming along quite smoothly, with reasonable objectives and deliverables. We’ve managed to figure out how to split the project, establish a schedule for our work, and I’ve learnt how to use GNOME Boxes to test different implementations of the “New Document” feature. I found the community very helpful and welcoming, just like my mentor Antonio Fernandes who’s very understanding and patient 🙂