Some people seem to have an outrageous number of containers on their system. That can create pretty bad performance with Ptyxis when it is using a GtkPopoverMenu to show you the container list.
Nightly will now use a custom popover backed by GtkListView which should help with that. Additionally, you can filter them now easily. To bring up the menu, alt+, can be used.
Now that libfoundry use has proliferated I need to get all the core abstractions in place for the proverbial 1.0.
There is already a device manager and provider abstraction in libfoundry with the typical back-ends. There are providers for the local system (so native architecture) and deviced which connects to a device on the local network.
Builder supports cross-architecture building and running even when you do not have a cross-toolchain available. So this must be added to Foundry too. The mechanics are handled by qemu-user-static and binfmt when properly packaged on your distribution. Fedora manages to have this setup correctly if you dnf install qemu-user-static.
Practically speaking, that means if you install a Flatpak SDK for another architecture you can use it to build/run your application (at a performance penalty). Qemu-user-static uses a combination of syscall-translation and instruction-translation which can have significant overhead, but it does work.
You can use Foundry now to do this rather easily.
$ cd project/ $ foundry enter $ foundry device list ID Active Name Chassis System qemu:riscv64 No My Computer (riscv64 Emulation) workstation riscv64 qemu:x86_64 No My Computer (x64_64 Emulation) workstation x86_64 native Yes My Computer workstation aarch64-linux-gnu $ foundry device switch qemu:x86_64 $ foundry run ...
Since I’m running on an aarch64 laptop right now, qemu:x86_64 device emulation is available.
If you have a cached build you might want to purge that so it doesn’t try to incrementally rebuild your project.
$ foundry pipeline purge $ foundry run
If you want to export a Flatpak to test on another system, you can export as normal. However this time it will be for your alternate architecture.
$ foundry export ... Artifacts: /path/to/x86_64-main/app.devsuite.Test.Devel.flatpak
Hopefully that makes things easier for people who want to test other devices/architectures such as GNOME on a handheld device.
D-Spy was getting pretty long in the tooth using so much infrastructure in GTK that has been deprecated. Particularly GtkTreeView. Makes sense given that it was originally a GTK 3 application.
Now that we have libadwaita and lots of easy GtkListView data binding, I took the liberty to revamp it. I’m sure it could use design review and lots of paper-cut fixes. But that is why we land this stuff early in the cycle.
Now that people are testing out GNOME running on mobile devices, I made it shrink to that form factor as well.
Have at it!
I know that I will never convince everyone to use Builder for development. Even I have a hard time getting myself out of the terminal at times.
Foundry comes in two forms, an executable and a library. The executable is just a bunch of commands and libfoundry is meant for building your own IDE (like Builder) or specialized tooling.
# initialize .foundry directory if never done before cd my-project/ foundry init
If you go into most GNOME applications, they likely have a Flatpak manifest. Builder uses this to auto-discover a lot about a project. Foundry is no different.
# Build the project, just like Builder would do foundry build
You can run this from any directory in your project as it will scan upwards to locate the .foundry/ directory that contains project state. It will also do incremental building just like Builder does.
# Run the project, just like Builder would do foundry run # Run a specific command in place of the default program. # Useful to test something with "runtime" instead of # build environment. foundry run -- gtk4-demo
Running the project will setup the necessary incremental flatpak pipelines and auxiliary tooling like an a11y bus, font integration, and what not. Your application will be run in the Flatpak build container based on the finish-args.
This does require building the application to ensure the runtime environment is setup correctly.
# Pull updated project dependencies. foundry dependencies update
Updating your deps is pretty painless. This is not done automatically unless the dependency is missing so that we don’t have to constantly hammer remote servers on every build request.
# Run a shell in the build pipeline, launches in builddir foundry devenv # Run a specific command in build pipeline foundry devenv -- ps aux
You can get a terminal shell in the build pipeline (much like ctrl+alt+shift+t in Builder.
# Lots of commands to inspect the build pipeline foundry pipeline info # Alias to foundry build foundry pipeline build # Invalidate all pipeline stages foundry pipeline invalidate # Purge (remove build files/artifacts/etc) to force # a clean build foundry pipeline purge # clean, configure, export, rebuild, which, install # all provide additional pipeline operations
You can interact with the active build pipeline using the above commands.
Sometimes you might want to know what compiler flags will get used with tooling on a specific file. That is made available via the pipeline as well.
foundry pipeline flags path/to/file.c
Maybe you need a language server for integrating with another editor. That is pretty easy to do for the supported languages by using the lsp command.
$ foundry lsp list Name Languages zls 'zig' vhdl-language-server 'vhdl' vala-language-server 'vala' 'genie' ts-language-server 'js' 'jsx' 'typescript' 'typescript-jsx' sourcekit-lsp 'swift' serve-d 'd' rust-analyzer 'rust' ruff 'python3' 'python' pylsp 'python3' 'python' mesonlsp 'meson' lua-language-server 'lua' jedi-language-server 'python' jdtls 'java' intelephense 'php' gopls 'go' glsl-language-server 'glsl' elixir-ls 'elixir' clangd 'c' 'cpp' 'chdr' 'cpphdr' 'objc' blueprint 'blueprint' bash-language-server 'sh' # use stdin/stdout to communicate with an LSP for python3 $ foundry lsp run python3
If you have multiple project configurations, you can look through them and select the active one. Just switch configs and run, Foundry will do the rest.
foundry config list ... foundry config switch flatpak:app.devsuite.Manuals.Devel.json foundry run
Want to get a .flatpak you can copy to another system to test?
foundry export ... Artifacts: file:///.../apps.devsuite.Manuals.Devel.flatpak
You can switch devices in case you have deviced running somewhere like a phone or tablet. Then running should deploy to that system and run it there.
foundry device list ... foundry device switch some_device foundry run
You can get the URI to documentation which could allow you to integrate with other editors. Like many commands, you can use --format=json to get the output in an easy-to-parse format for editor plugins.
# list available doc bundles to install foundry doc bundle list ... # install docs for GNOME 48 foundry doc bundle install flatpak/org.gnome.Sdk.Docs/48/user # Find GtkWidget documentation foundry doc query --format=json GtkWidget
Many settings can be tweaked using foundry settings set .... Just tab-complete around to explore.
Anyway, hopefully that serves as a quick intro into some of the things you can do with Foundry already as it progresses towards being a fully-fledged replacement for Builder’s internals.
I finally got around this past week to getting Manuals ported to libfoundry.
That was something I wanted to do early so that I can rapidly get away from 3 versions of the documentation engine and Flatpak SDK management code. Currently it existed in Manuals, Builder, and Foundry, and soon we’ll get it to just Foundry.
That also makes Manuals the first application to use libfoundry which resulted in lots of tree-shaking and things are looking bright! I very much look forward to getting Builder rebased on libfoundry.
While I was at it, I ticked off a number of requested design issues as part of the Incubator submission. This afternoon it also got support for narrow views meaning you can run it on a GNOME-enabled phone.
Probably not how most people will use it, but hey, maybe you have actually good public transportation where you are and some free time.
Back in December (before I caught the flu working at a farmers market, then Covid two weeks later, then two months of long-Covid) I mentioned that we’d discuss the various subsystems needed in libfoundry to build an IDE as a library.
I used the little bit of energy I had to work on some core abstractions. In an effort to live up to my word lets talk a bit about what went into libfoundry last night.
There is now a DocumentationManager sub-system which handles documentation installed on the host system, chroots, and Flatpak SDKs. It’s a bit tricky to make this all work without blurring the lines of abstraction so lets cover how that works.
Generally speaking, we try to avoid plugins depending on other plugins. Sometimes it happens but usually it is an opportunity to make a better abstraction in libfoundry. Lets look at what is needed around documentation.
So lets dive into the abstractions.
This is the core abstraction you start interfacing with. It is a service of the FoundryContext and therefore can be accessed with Foundry.Context:documentation-manager property.
The documentation manager manages the Foundry.DocumentationProvider plug-in abstractions. Plug-ins that which to contribute to the documentation pipeline must subclass this in their plug-in.
To query documentation, use Foundry.DocumentationManager.query(). As I noted earlier, I don’t want new capabilities to break the API so a Foundry.DocumentationQuery object is used rather than a sequence of parameters which would need to be modified.
Since we want to be able to support other documentation formats in the future, it is important that we do not force anything about devhelp2 XML into the core abstraction.
The core result object from queries is a simple Foundry.Documentation object. Like above, we want to avoid breaking API/ABI when new capabilities are added so this object serves as our abstraction to do so. Navigating a tree structure will live here and can be implemented by plug-ins through subclassing.
Additionally, a “devhelp” plug-in provides support for crawling the devhelp2-style directories on disk. But this plug-in knows nothing about where to find documentation as that is relevant only to the SDKs.
This is where the Foundry.DocumentationRoot object becomes useful. SDK plug-ins can implement DocumentationProvider in their plug-in to expose documentation roots. The host-sdk, jhbuild, and Flatpak plug-ins all do this to expose the location of their documentation.
Now the devhelp plug-in can be provided the information it needs for crawling without any knowledge of SDKs.
The old adage is that the only way to go faster on a computer is to do less work. This is particularly important in search systems where doing an entire query of a database means a lot of wasted CPU, memory, and storage I/O.
To make querying fast the devhelp plug-in indexes information about SDKs in SQLite. Way back in Builder we’d avoid this and just make an optimized fuzzy search index, mmap that, and search it. But now days we’ve gone from one set of documentation to multiple sets of documentation across SDK versions. The problem domain explodes quite a bit. SQLite seemed like a nice way to do this while also allowing us to be lazy in our searching.
By lazy what I mean is that while we’ll start your query, we only retrieve the first few results from the cursor. The rest are lazily fetched as the GListModel is scanned by scrolling. As that is not a very common operation compared to typing, you can throw away a lot of work naturally while still sitting behind the comfortable GListModel interface.
Since libfoundry already supports SDK management (including Flatpak) you could probably re-implement Manuals in a week-end. Hopefully this also breaks down a bit of the knowledge used to build such an application and the deceptive complexity behind doing it well.
This should also, hopefully soon, allow us to share a documentation implementation across Builder, Manuals, and an upcoming project I have which will benefit from easy access to documentation of object properties.
With GNOME 48 I released libdex 0.10 on the march towards a 1.0. One of the major improved features there was around fiber cancellation.
I’m not going to go into detail about the differences between threads and fibers as wikipedia or your local CS department can probably help you there. But what I will say is that combining __attribute__((cleanup)) (e.g. g_autoptr()) with futures and fibers makes such a nicer experience when writing C.
Thread cancellation is a rather non-portable part of the threading stack across platforms. Some POSIX platforms support it, some don’t. Having safe places to cancel can be a real challenge even if you are depending on a threading implementation that can do it.
With fibers, we have a natural cancellation point due to the cooperative nature of scheduling. All (well behaved) fibers are either making progress or awaiting completion of a future. We use the natural await() points to implement cancellation. If everything that was awaiting the future of the fiber has been cancelled, then the fiber can naturally cancel too. The next time it awaits that will just happen and natural exit paths will occur.
When you don’t want cancellation to propagate, you still use dex_future_disown() like always (as the fiber itself is a future).
Just to give a quick example of how fibers and futures makes writing C code nicer, here is an excerpt from libfoundry that asynchronously implements the necessary phases to build/run your project with a specific tool, possibly on a non-local system. In the GNOME Builder IDE, this is a series of async callbacks that is extremely difficult to read/debug. But with Foundry using libdex, it’s just a few lines of code and every bit as non-blocking.
From foundry-run-manager.c.
g_autoptr(FoundryDeployStrategy) deploy_strategy = NULL;
g_autoptr(FoundryBuildProgress) progress = NULL;
g_autoptr(GSubprocess) subprocess = NULL;
GError *error = NULL;
if (!(deploy_strategy = dex_await_object (foundry_deploy_strategy_new (state->pipeline), &error)) ||
!dex_await (foundry_deploy_strategy_deploy (deploy_strategy, state->build_pty_fd, state->cancellable), &error) ||
!dex_await (foundry_deploy_strategy_prepare (deploy_strategy, state->launcher, state->pipeline, state->build_pty_fd, state->cancellable), &error) ||
!dex_await (foundry_run_tool_prepare (state->run_tool, state->pipeline, state->command, state->launcher, state->run_pty_fd), &error) ||
!(subprocess = foundry_process_launcher_spawn (state->launcher, &error)))
return dex_future_new_for_error (error);
At each dex_await*() function call the fiber is suspended and we return to the main loop for additional processing.
In a better world we’d be able to do these without fibers and instead do stackless coroutines. But maybe with a little compiler help we can have that too.
Not all of my projects this December are code related. In fact a lot of them have been house maintenance things, joy of home ownership and all.
This week was spent building my new office and music space. I wanted a way to have my amplifiers and guitars more accessible while also creating a sort of “dark academia” sort of feeling for working.
The first step was to get the guitars mounted on the walls. I was looking for something blending artistic showpiece and functional use.
After that I quickly framed them in. Just quarter round with the hard edge inwards, 45° miter, some caulking, easy peasy.
My last office had Hale Navy as the color, but the sheen was too much that it made it difficult to actually see the color. This time I went flat and color drenched the space (so ceilings, trim, etc all in matching tone).
Then somewhat final result is here. I still want to have a lighting story for these that doesn’t involve a battery so some electrical fish taping is likely in my future.
I also converted the wide closet into a workstation area with the studio monitors for recording. But that is still partially finished as I need to plane all the slats for the slat wall, frame the builtin, and attach the countertop.
Early on Builder had the concept of layered settings. You had an application default layer the user could control. You also had a project layer which allowed the user to change settings just for that project. But that was about the extent of it. Additionally, these settings were just stored in your normal GSettings data repository so there is no sharing of settings with other project collaborators. Boo!
With Foundry, I’d like to have a bit more flexibility and control. Specifically, I want three layers. One layer for the user’s preferences at the application level. Then project settings which can be bundled with the project by the maintainer for needs specific to the project. Lastly, a layer of user overrides which takes maximum preference.
Of course, it should still continue to use GSettings under the hood because that makes writing application UI rather easy. As mentioned previously, we’ll have a .foundry directory we place within the project with storage for both user and project data. That means we can use a GKeyFile back-end to GSettings and place the data there.
You can git commit your project settings if you’re the maintainer and ensure that your projects conventions are shared to your collaborators.
Of course, since this is all command-line based right now, there are tab-completable commands for this which again, makes unit testing this stuff easier.
# Reads the app.devsuite.foundry.project config-id gsetting
# taking into account all layers
$ foundry settings get project config-id
# Sets the config-id setting for just this user
$ foundry settings set project config-id "'org.example.app.json'"
# Sets the config-id for the project default which might
# be useful if you ship multiple flatpak manifest like GTK does
$ foundry settings set --project project config-id "'org.example.app.json'"
# Or maybe set a default for the app
$ foundry settings set --global project stop-signal SIGKILL
That code is now wired up to the FoundryContext via foundry_context_load_settings().
Next time I hope to cover the various sub-systems you might need in an IDE and how those services are broken down in Foundry.
A core tenant of Foundry is a pleasurable command-line experience. And one of the most creature-comforts there is tab-completion.
But how you go about doing that is pretty different across every shell. In Flatpak, they use a hidden internal command called “complete” which takes a few arguments and then does magic to figure out what you wanted.
Implementing that when you have one layer of commands is not too difficult even to brute force. But imagine for a second that every command may have sub-commands and it can get much more difficult. Especially if each of those sub-commands have options that must be applied before diving into the next sub-command.
Such is the case with foundry, because I much prefer foundry config switch over foundry config-switch. Particularly because you may have other commands like foundry config list. It feels much more spatially aware to me.
There will be a large number of commands implemented over time, so keeping the code at the call-site rather small is necessary. Even more so when the commands could be getting proxied from another process or awaiting for futures to complete.
With all those requirements in mind, I came up with FoundryCliCommandTree. The tree is built as an n-ary tree using GNode where you register a command vtable with the command parts like ["foundry", "config", "switch"].
At each layer you can have GOptionEntry like you normally use with GLib-based projects but in this case they will end up in a FoundryCliOptions very similar to what GApplicationClass.local_command_line() does.
So now foundry has a builtin “complete” command like Flatpak and works fairly similarly though with the added complexity to support my ideal ergonomics.