Rust: state of GUI, December 2022

There was a recent call for blogs about Rust GUI. So, Are we GUI yet?

Contents:

Categorised listing of toolkits

Lets start by categorising entries from Are we GUI yet, ignoring those which appear abandoned or not very functional.

Bindings

Wrappers around platform-specific toolkits:

  • Mac OS / iOS - cacao - Rust bindings for AppKit and UIKit
  • Mac OS / iOS - core-foundation - Bindings to Core Foundation for macOS
  • Win32 - winsafe - Windows API and GUI in safe, idiomatic Rust

Wrappers around multi-platform non-Rust toolkits:

Web/DOM based

Frameworks built over web technologies:

  • Dioxus - React-like Rust toolkit for deploying DOM-based apps
  • Tauri - Deploy an app written in HTML or one of several web frontends to desktop or mobile platforms

Note that Dioxus uses Tauri for desktop deployments.

Rust toolkits

Finally, Rust-first toolkits (excluding the web-based ones above):

  • Druid - Data-oriented Rust UI design toolkit
  • egui - an easy-to-use GUI in pure Rust
  • fui - MVVM UI Framework
  • Iced - A cross-platform GUI library inspired by Elm
  • KAS - A pure-Rust GUI toolkit with stateful widgets
  • OrbTk - The Orbital Widget Toolkit
  • Relm4 - An idiomatic GUI library inspired by Elm
  • Slint - a toolkit to efficiently develop fluid graphical user interfaces for any display

Details

Lets take a deeper look at a few of the above.

Tauri is a framework for bringing web apps (supporting several major web frontend frameworks) to the desktop and (eventually) mobile. Dioxus is a React-like frontend written in Rust with built-in support for Tauri. Tauri has its own rendering library (WRY) over platform-specific WebView libraries along with a windowing library (TAO), a fork of winit.

Druid, a data-first Rust toolkit, is interesting in a few ways. First is the UI design around a shared data model, giving widgets focussed views through a system of lenses. Second is that it uses its own platform-integration library, druid-shell, along with its own graphics library, Piet. Finally, all widgets (from the user or library) implement only a simple interface, but must be stored within a special WidgetPod. Watch a recent talk by Raph Levien on the state and design of Druid, including the current Xilem re-design.

egui, an immediate-mode pure-Rust toolkit, succeeds both in being extremely easy to use and in having an impressive feature set while being fast. It has some limitations. Widgets may be implemented with a single function. Windowing may use winit, glutin or integration; rendering may use glow, Glium or wgpu.

Iced is an Elm-like toolkit. It supports integration, winit or glutin windowing with wgpu or glow for rendering or web deployment.

Relm4 is an Elm-like Rust toolkit over GTK 4.

Slint is a UI built around its own markup language (.slint) in the style of QML, with support for apps written in Rust, C++ and JavaScript. The markup language makes description of simple UIs easy, but as a result user-defined widgets have completely different syntax and limited capabilities compared to library-provided widgets. Slint uses either Qt or winit + femtovg + various platform-specific libraries.

KAS, by myself, is an efficient retained-state toolkit. It supports windowing via winit and rendering via wgpu.

State of KAS

Being a toolkit author myself, I ought to say a few words about the status of KAS. In case you hadn't guessed already, I like lists:

  • In September 2021, KAS v0.10 reorganised crates, added support for dynamic linking, improved the default theme, and standardised keyboard navigation.
  • Exactly a year later, v0.11 heavily revised the Widget traits and supporting macros, supported declarative complex static layout within widgets, plus many more small changes all aimed at making KAS easier to use.
  • I have just released v0.12 which uses Rust's recent support for Generic Associated Types to update temporary APIs.

To state the obvious, KAS isn't very popular. Despite this, it compares well with other Rust toolkits on features, missing a few (such as screen-reader support and dynamic declarative layout), but also supporting some less common ones, such as complex text formatting (currently limited to web frameworks, bindings, Druid and KAS) and fast momentum touch scrolling over large data models.

My main concerns regarding KAS are:

  • Without a userbase and with a (now) healthy competition, motivation to further develop the library is limited.
  • Immediate-mode GUIs, the Elm model and Druid all make dynamic layout easy. In KAS, all widgets must be declared with static layout (though hiding, paging etc. is possible and the declaration is often implicit).
  • Several aspects of KAS from glyph fallback and font discovery to the drawing interface and themes are hacks. Producing good implementations from these is possible, but alternatives should also be considered (such as switching to COSMIC Text or Piet). Either way, significant breaking changes should be expected before KAS v1.0.
  • Some redesign to EventState is needed to properly support overlay layers.

From another point of view, KAS has had several successes:

  • A powerful size model, if a little more complex to use than most toolkits
  • Robust and powerful event-handling model
  • Very fast and scalable
  • Not too hard to use for non-dynamic layout (if you tried KAS v0.10 or earlier, usability is much improved, and certainly much better than traditional toolkits like GTK or Win32)
  • The easy-cast library
  • I learned a plenty about proc-macros, resulting in a rather interesting method of implementing the Widget trait and the impl-tools library.

Future work, if I feel so inspired, may involve the following:

  • Dynamic declarative layout. Druid's Xilem project has given me ideas for a solution which should work over KAS's current model, though likely less efficient. This is not the right place to elaborate on this.
  • Async support? Currently, KAS only supports async code via threads and a proxy which must be created from the toolkit before the UI starts. Real async support allows non-blocking slow event handlers in a single thread. KAS could support this, e.g. via a button-press handler generating a future which yields a message on completion, then resuming the current event-handling stack with that message. However, this may not be the best approach (especially not if it requires every event-handling widget to directly support resuming a future). A variant of this approach may be more viable together with the dynamic declarative layout idea mentioned above.

State of GUI

The purpose of this post is not just to talk about KAS, but about Rust GUIs in general. To quote a reddit comment:

GUI will never be "solved" because it has become a problem too complex to be solved with one solution to rule them all. GUI is not one problem anymore, it is a family of problems.

As listed above, we now have many (nascent) solutions. These share a few common requirements:

  • A need to create (at least one) window
  • A need to render, often with GPU acceleration
  • A need to type-set complex text
  • A need to support accessibility (a11y) and internationalisation (i18n)

So lets talk a little about libraries supporting those things.

Windowing

winit is, perhaps, the de-facto Rust windowing/platform integration library. It handles window creation and input from at least keyboard, mouse and touchscreen devices. Making things more difficult is that there is room for considerable scope for feature creep in this project: Input Method Editors, theme (dark mode) detection, gamepad/joystick support, clipboard integration, overlay/pop-up layers, virtual-keyboard invocation, and probably much more.

Designing a cross-platform windowing and system-integration library is a hard problem, and though Winit already has a lot of success, it also has a lot of open issues and in-progress features. Winit's nearest equivalent is probably SDL2, which focussess mainly on game requirements (window creation and game controller support). Winit attempts to be significantly more (particularly regarding text input).

So, is Winit the right answer, or is attempting to solve all windowing and input requirements on all platforms for all types of applications a doomed prospect? My understanding is that Winit's biggest issue is a lack of maintainers with adequate time and knowledge of the various target platforms.

Thanks to raw-window-handle, there is a degree of flexibility between the renderer and window manager used, supporting some alternatives to Winit:

  • Tauri maintains TAO, a fork of winit with deeper GTK integration, most notably to support WRY via WebKitGTK.
  • Druid has its own druid-shell library designed for use with Piet (also supporting raw-window-handle)
  • Glazier is a new project (also by the Druid team) for a GUI-oriented alternative to Winit
  • We don't have to use Rust! Some of toolkits above, even those considered Rust toolkits like Slint and Relm4, make use of GTK or Qt for platform integration and rendering.

Rendering (backends)

Quite a few solutions are available for drawing to windows. Pick one, or like a few of the above toolkits, support several:

  • softbuffer facilitates pure-CPU rendering over a raw-window-handle
  • glutin supports the creation of an OpenGL context over a raw-window-handle, thus supporting glow (GL on Whatever; the library may also be used with SDL2) or Glium (an "elegant and safe OpenGL wrapper")
  • Several (low- and high-level) bindings are available to Vulkan, Metal and Direct3D; to my knowledge none of these are used directly by any significant Rust GUI toolkits
  • wgpu is "a cross-platform, safe, pure-rust graphics api. It runs natively on Vulkan, Metal, D3D12, D3D11, and OpenGLES; and on top of WebGPU on wasm." As a high-level, portarble and modern accelerated graphics API, it is (optionally) used by multiple Rust GUI toolkits (eGUI, Iced, KAS) despite adding a considerable layer of complexity between the toolkit renderer and the GPU.

Rendering (high-level)

Several Rust toolkits add their own high-level rendering libraries:

  • WRY is a rendering layer over platform-dependent WebView backends
  • Piet is Druid's rendering layer, leveraging system libraries especially for complex font support
  • Vello, formerly known as Piet-GPU, is a research project to construct a high-level GPU-accelerated rendering layer
  • Kurbo is used by Piet for drawing curves and paths
  • epaint is "a bare-bones 2D graphics library for turning simple 2D shapes and text into textured triangles"

Unaffiliated high-level rendering libraries used by GUI toolkits include:

  • Lyon, a path tessellation library (supports GPU-accelerated rendering of SVG content)
  • resvg is a high-quality CPU-rendering library for static SVGs using tiny-skia
  • tiny-skia is "a tiny Skia subset ported to Rust"
  • femtovg is an "antialiased 2D vector drawing library written in Rust"

Text and type-setting

To say that type-setting text is a complex problem is a gross understatement. Sub-problems for fonts include discovering system fonts and configuration, matching a "font family", use of "fallback" fonts for missing glyphs, glyph rendering, hinting, sub-pixel rendering and font synthesis. Sub-problems for text include breaking into level runs (i.e. same left/right direction), breaking these into same-font runs, shaping (or at least kerning), potentially multiple times where multiple fonts are required, line-breaking, hyphenating, indenting, dealing with ligatures, combining diacritics, navigation, and then there's still everyone's favourite: emojis. Then there is drawing highlighting and underlines, and if you're still looking for things to do, correctly justifying Arabic texts (by extending words) and gapping underlines around descenders.

Right, what do we have? We really have to thank Yevhenii Reizner (@RazrFalcon) for his efforts including ttf-parser, fontdb and rustybuzz (not to mention tiny-skia!). We have several libraries for rendering and caching glyphs including ab_glyph and Fontdue which perform basic rendering; more recently Swash has significantly raised the bar on the quality of typesetting and rendering.

The above are all low-level components. If you want a high-level interface able to type-set and render text, what libraries are out there?

  • glyph_brush is around four years old, covering rendering and basic line wrapping. Though this is enough for quite a few use-cases, it cannot handle complex text nor does it properly handle navigation (which is presumably why Iced still does not support multi-line text editing).
  • Piet (or more accurately piet-common), around three years old, supports at rich text, leveraging system libraries (Cairo on Linux).
  • KAS Text, around two years old and written by myself, supports (at least partially) rich text and bi-directional text, while missing a few features such as emojis and using a few crude hacks (indentation and font discovery).
  • femtovg, at least since around a year, supports moderately complex text.
  • COSMIC Text is a (very) recent project by System76 (creators of Pop! OS) for complex text layout in Rust, leveraging fontdb, rustybuzz, Swash and other Rust font libraries. It seems we can finally have high-quality pure-Rust type-setting. Thanks System76!
  • parley is a nascent project by Chad Brokaw (the author of Swash) for rich text layout, intended for usage with Piet

Accessibility and internationalisation

Admittedly this is not a topic I have researched. Accessibility for GUIs means at least supporting keyboard control (now common) and screen readers (less widely supported). Internationalisation, though a little more complicated than just string substitution, is less reliant on toolkit support. We now have a few libraries on these topics:

  • AccessKit, "UI accessibility infrastructure across platforms and programming languages"
  • TTS "provides a high-level Text-To-Speech (TTS) interface"
  • rust-i18n is "a crate for loading localized text from a set of YAML mapping files", inspired by ruby-i18n and Rails i18n
  • Fluent is "a Rust implemetnation of Project Fluent"
  • gettext-rs, "Safe bindings for gettext"