June 14, 2011
So, when a user turns on their computer, boot ups, and then enters their username, password, etc, under the covers GDM is using a framework called PAM to do the heavy lifting of processing the login request. PAM is a fairly old, and established library that reads a “service” file to determine what plugins to load, and in what order to load them. The plugins determine what questions to ask the user, verifies the answers the user gives are acceptable, and things like that. As an example, one pam module is pam_unix which will get GDM to ask for a username and password that get checked against /etc/passwd and /etc/shadow (in a lot of cases anyway). Another example, is a module called pam_rps that makes the user play rock-paper-scissors instead of entering a password. Some PAM modules may not want text answers at all, though. For instance pam_fprintd just waits for the user to swipe their finger.
Traditionally, applications that use PAM only ever have one conversation running at a time. That one service file is loaded up and that service file’s stack of modules are run through in order as defined by the policy in the file. If a user wants fingerprint and password support, then they have to use a service file that mentions both pam_unix and pam_fprintd, and the modules have to provide some way to “fall through” to the next module. That means potentially giving up on waiting for fingerprint after 30 seconds and then jumping to pam_unix to ask for password, or something suboptimal along those lines.
Of course, ideally a user doesn’t want this serialized behavior. Instead, they want to either be able to type their password right away or swipe their finger right away and have the right thing happen.
One solution, that’s been slowly trickling in from a branch in git for quite a while, is to run multiple PAM conversations concurrently. I’ve deemed this ‘multi-stack’ or ‘split authentication’. So we can have the “password” stack of pam modules and the “fingerprint” stack both cheefully running along, independent of each other. If the user satisfies the requirements of one of the conversations (say by swiping their finger or entering their password), we cancel the other and proceed with login.
It’s a useful feature, and I finally decided to cut the remainly bulk of the work over to master yesterday in time for the 3.1.2 release. distros will have to enable it with –enable-split-authentication=yes passed to configure, and probably need to modify their pam configurations to accomodate.
I wouldn’t say it’s in perfect shape (in particular the greeter plugin interface is pretty unpleasant), but I wanted to land it now since it’s been lingering on the branch for so long. I’m sure there will be follow commits to deal with unexpected breakage as distros pick it up.
Next thing I’d like to do for 3.2 is get more GNOME 3 integrated greeter. I’ll try to do a follow up post about that soon.
May 24, 2011
October 25, 2010
So in an effort to help make GNOME 3 turn out as good as possible, I’m helping figure out the story for system modal dialogs in the presence of the shell.
Anyway, I don’t really want to make this blog post about GNOME 3 as a development platform, but more about system modal dialogs. These dialogs are the things that should be dealt with “right away”. These are things that require the full attention of the user. As such, they should
- should appear front and center on screen
- should be clearly coming “from the OS” versus from an application
- should “take the stage” until dismissed
The idea is these are things that really can’t wait, be side lined, etc. In general, only the os “core” should ever need to present a system modal dialog.
Most dialogs should not be system modal, since most dialogs come from applications. Also, system modal dialogs have a very high “cost” to the user since by their very nature, they’re modal. That is to say, they preempt the user from whatever auxiliary tasks the user was doing and prevents those tasks from being available again until after the dialog is dealt with. In this regard, system modal dialogs should never be unexpected. They should always be the direct consequence of something the user was doing. For many use cases the message tray is a better option than system modal dialogs.
One example where system modal dialogs are very relevant is password dialogs. We really need password dialogs to have a consistent look that matches the look for the shell. This is important so that we can “train” the user into learning what a password dialog should look like, so they think twice before inputting their password into a dialog that’s not provided by the OS. This paves the way for a trusted path future. Of course, it’s only a step in the right direction. Actually getting trusted path is a hard problem that’s going to have to get solved at some point in the future. WHen that feature is seriously looked at, the final solution may involve one or more of: secure attention keys, wayland, factory mode login screens, integration with the users cell phone, or other possibilities
Another example of a dialog that should be system modal is the logout dialog.
Now, I just want to give a few details about where I’m going on the implementation side of things. Since we know that system modal dialogs need to “look like” the shell, they probably need to run in the shell process. (there’s no plan to make GTK+ have a “Look like the shell” mode etc). On the other hand, these dialogs are very tightly bound to the service that wants it. For instance, gnome-session is the only thing that should ever be able to summon the logout dialog for the shell. Everything else should talk to gnome-session.
Given how closely bound a dialog is to the particular service needing the dialog, ideally, its implementation should be easily adaptable to changes the service needs as the services needs them. While the look and feel both need to align with the shell, the contents of the dialog need are details almost entirely specific to the service requesting the dialog. The interface between the dialog and the service should be private and without any sort of backward compatibility constraints, etc. This means it would be best if we could ship the implementation for the dialog with the service. But how do we ship the implementation with the service and run that implementation in the shell?
There are some unresolved details on how these “private” extensions should differ from the more public, user installable extensions the extension system was originally designed for (like should the extension interface be different, should the private extensions show up in the extension list in the UI, etc).
This blog post is already getting a bit long, so I’m going to cut it off here. I’ll do another post at some point with some results from the above considerations.
July 30, 2010
So Mo and I finally got engaged.
November 28, 2009
So one of the big polish bits in the boot process is the transition from plymouth to X. In Fedora, we worked to make it as seemless as possible when we first started on the Better Startup stuff a few releases ago.
If you haven’t seen it, when boot up finishes, plymouth settles down the boot splash to a transitionable animation frame, then the mouse pointer shows up, and GDM’s background cross fades in while the login window maps and expands to show frequently logged in users. In the best case, this transition all happens without any flicker, resolution changes, black intermediate screens, or console text showing up.
The way we accomplished this was:
- I gave “plymouth quit” a “–retain-splash” option which told plymouth to keep the boot splash’s contents on screen even after plymouth exits
- krh and ajax added a -nr option to the X server to make X not clear the screen to black (or a weave) at start up
- krh, airlied, and darktama (Ben Skeggs) added the driver backend support for -nr to intel, ati, and nouveau drivers respectiviely. Also, Bill Nottingham made -nr work with the fbdev X driver.
- ajax made the ugly big “X” mouse cursor not show up by default
- I made gdm stuff the screen contents to a pixmap referenced by the pseudo-standard _XROOTPMAP_ID root window property
- I also made gnome-settings-daemon cross fade from the _XROOTPMAP_ID pixmap when it loads its background. This step meant we also get a nice transition from gdm to the user’s session, because it also causes a crossfade to happen from gdm’s background to the user’s background during login.
Another big piece to the puzzle was kernel modesetting. This is what makes sure the right mode is set at boot up from the start. Jesse Barnes and krh did most of the work for that in the Intel driver (based on top of the memory manager work anholt, keithp, and other Intel crew did), while airlied did it for the ati driver, and darktama did it for nouveau.
This all works pretty well these days, but one big wrinkle in the whole process is the hand off from plymouth to gdm. Plymouth exits, leaving the system in KD_GRAPHICS mode and hopes that X starts and picks up the peices. This is not a very robust design, because if X doesn’t start, the system will be stuck in KD_GRAPHICS mode and the user won’t be able to switch VTs or really do anything at all, but reboot.
Furthermore, when plymouth exits, the kernel sets the scan out buffer to the frame buffer of the console. This is because the previous scan out buffer was owned by plymouth and goes away when plymouth exits. That means in order to keep things seemless we need to copy the last frame of the animation from plymouth’s framebuffer to the console frame buffer. This extra copy is unfortunate, because it means the kernel console has to be configured the same way as plymouth, but in multihead set ups it’s not. The kernel console “clones” the ttys across all heads, forcing every head to be the same resolution. But plymouth (and X) want every head to run in native resolution. This means in multihead setups we can’t do the copy because the pixel layouts aren’t the same, so we end up filling with black instead.
I’ve wanted to improve this situation for a while, but haven’t really been able to prioritize it above other various bits of work that need to be done. Since I’m on Thanksgiving break I can, in good conscience, ignore my priority list and just work on what I want, so I decided to work on this problem a bit :-)
I just merged the “no-fbcon” branch to plymouth master. It adds a new way to “deactivate” plymouth such that it’s still on screen and idly running, but X can start and take over anyway. This combined with some gdm changes to tell plymouth to go away in the appropriate way after X is started or fails to start means we solve the robustness issues and avoid the extra fbcon copy.
It actually only took a few hours to do. I probably should have made time for it before break.
November 17, 2009
October 11, 2009
The DVR hardware in the setup ouputs four avi files–one for each camera. The first file has the audio encoded in it. Having four files is cumbersome, though. It’s much better to see the camera focused on the users face at the same time as video focused on the users hands, and at the same time as the view that shows the users screen.
That’s where gstreamer comes in. It’s possible to write a pipeline that can take the 4 videos and compose them together into one 4-way split screen.
In Mo’s post she showed an earlier pipeline I came up with, but it was very slow and lacked audio.
I’ve been reading up on gstreamer, searching the internets for example pipelines, etc, and now have a better pipeline. Someone here at the summit asked for me to check it into git, so I did that today in the usability-lab module.
September 29, 2009
So it’s been a while since I’ve blogged.
This is just a short blog post to say that I landed multi-head support on plymouth master yesterday. More details here:
What this means is the boot splash redraws itself once per monitor, in a way that looks good for the native resolution each monitor. Previously, we’d draw right on the fb console, which is cloned across all monitors. That made the splash have large black margins on some monitors, or look stretched.
One implication of this change is we now use libdrm directly when possible instead of going through the /dev/fb interface.
We still don’t do any accelerated rendering. We don’t really need it, since plymouth splashes are normally designed to be very simple (Think “cell phone splash” not “3D video grame”).