> I don't understand why desktop app user interfaces aren't blazingly fast and 240fps.
Telemetry back to HQ takes resources.
I began to notice my disk drive light flickering regularly even when I was doing nothing with the computer, and the computer started seeming sluggish. I'd look at the process list, and started randomly forcing them to quit. Suddenly, the drive light stopped flashing. I'd found my man.
Googling the name of the process, it belonged to a program I'd bought that served media files to my Roku box. But I wasn't asking it to stream anything at the time.
I didn't know what the hell it was doing, but couldn't justify it, and removed the program.
> Googling the name of the process, it belonged to a program I'd bought that served media files to my Roku box. But I wasn't asking it to stream anything at the time.
Unfortunately, even if you have very fast drawing, there are plenty of other ways to make software slow. But high performance GUI software is the main thing I'm working on at the moment and I expect to have interesting results in the coming months. If nothing else, it should be a data point of how fast GUI software can be.
I’m super excited to hear about this. I’ve been working on a programming language for UI designers, and I ran into some massive issues when I tried to build my own renderer. Gave up and now I’m just transpiling to other languages instead.
Maybe basic productivity applications shouldn't try to tackle open research problems and instead just do the thing which has worked fine for the last 30 years which is CPU rendering? Like, do you think notepad is going to burn your battery because it redraws 20x20 pixels on every mouse event ?
text editors and terminals don’t render vector graphics though, do they? they rely on libraries to rasterize glyphs of a font into a bitmap once and use the bitmaps over and over.
> they rely on libraries to rasterize glyphs of a font into a bitmap once
Unfortunately it's not this simple. Unicode and multilingual support means you can't possibly generate a single bitmap to contain all possible characters. Modern UIs (including text editors and terminals) generally support various zoom levels, which requires re-rasterizing glyphs at different zoom levels. This means you typically have to rasterize vector graphics (your font) on the fly as needed and then update your bitmap.
Edit: Also, emojis and certain scripts require composting multiple glyphs together. This means you have to figure out which glyphs are required for your codepoint and use your font metrics to composite the glyphs appropriately. Then throw in mathematical notation and all bets are off.
My point is, rendering text is not a simple done deal, but it's also not an impossible feat and it can be done efficiently.
> vector graphics are not rendered directly to the screen.
What difference does it make if you're caching the rasterization or not? You're still rendering vector graphics. And you're not just rasterizing once.
Modern font renderers can't get away with rasterizing glyphs once and reusing them anymore than arbitrary vector graphics renderers can. I sure hope my vector graphics editors cache the rasterizations instead of re-rasterizing needlessly.
a huge amount of emoji are embedded png in font files. newer ones, however, do require more work.
> figure out which glyphs are required for your codepoint and use your font metrics to composite the glyphs appropriately.
this is all done in shaping though, the renderers just see "put this thing there" and (at least in GTK) we just maintain texture atlases that are updated on the fly.
you're correct that it doesn't solve the same glyph at lots of scales though. but glyphy does (and at high FPS too).
GTK's implementation of text rendering is is admittedly broken in several subtle ways that nobody seems to know how to properly fix. [1] Also, this is while _not_ supporting fractional scaling, which is something that users definitely want.
To be clear, I'm not bashing on GTK. I'm saying that it definitely isn't as simple as the above comment implies ;)
That issue is literally why I have a branch integrating GLyphy. But yes, your casual drive by comment is definitely more knowledgeable than the people who work on the stuff.
harfbuzz creator made https://github.com/behdad/glyphy/ which i integrated into GTK a while back (but we don't ship currently, because we still do the bitmap stuff).
the bitmap stuff still has major drawbacks though, like maintaining grid alignments and pixel boundaries where as this stuff (mostly) goes away using glyphy.
still work to be done around hinting though (hence not merged).
Yeah, but they're not actually completely flat since they're translucent.
Also, on Windows, the "acrylic" thing is a circus show. Every app behaves differently. It also manages to sometimes lag on an RX5600. I know it's an "old" gpu, but still...
But in most contexts, it is not necessary to render vector graphics 'live'; caching is acceptable, so this should not be a bottleneck for most desktop software.
Only very recently. Like, all that old Aqua stuff wasn't vectors, it was done in Photoshop.
Bitmaps are much faster to draw and can be made pixel-precise up front. With vectors it's easy to have pixel cracks or blurry lines because it's offset .5px.
> Bitmaps are much faster to draw and can be made pixel-precise up front.
yes, but that alone doesn't require that bitmap graphics be used.
> With vectors it's easy to have pixel cracks or blurry lines because it's offset .5px.
only if your renderer or your artwork isn't done correctly; inkscape used to be notorious for floating point rounding errors showing up with very few manipulations of a shape.
it's also been very easy to very quickly rasterize vector graphics for some time. maybe not fast enough in the past I guess, but computers are certainly fast enough now.
I look forward to a time when UIs are just all vector all the time, and it's the monitor itself which rasterizes, thanks to a program that the attached device uploads as it boots up or as the monitor is connected.
it's going to get more and more difficult to continue increasing display resolutions and refresh rates over long inexpensive cables if we continue to rasterize on a computer and push raw pixels to the display device.
Like? SVG UIs everywhere? Could you make a UI that's like SVG and onclick handlers do the "magic". We could have a VB6 like SVG UI designer and program "actions" in $LANG?
WPF would probably be that. In any case, modern UI toolkits often are vector-based anyway because of UI scaling. Heck, even ancient ones are, if you look at the Marlett font. That contained scalable UI elements for Windows 95.
> That being said, I think current iOS and macOS can run their UIs at 120fps (and Apple devices still seem to have decent battery life.)
The same critique for Apple applies all the time, when you are completely vertically integrated and make your own chips up to the very software running on them, it's kinda easier.
That aside, there's no display offering above 60Hz IIRC in the Apple line. Correct me if I am wrong.
Streaming in hundreds of assets as you walk around an open world does surely take a different pipeline, but as a show of computing competence it is staggering. You should see how fast some of these truly massive games load nowadays. The fact that the notepad app takes forever to load when all it needs to load is the UI is just a sad state of affairs.
A mesh may have a lot of triangles, but it's still a single object you can memory map. UIs are typically a bunch of heap allocations and pointer jumping all created at runtime. That can be surprisingly inefficient.
Arena allocators only work when everything has the same lifetime.
There are game-like UI libraries (https://github.com/ocornut/imgui) but to use them you have to write your program like you're a gamedev and Conway's Law usually fights that.
1. Avoidable: The main one here is shader caching. Shaders could be precompiled for every supported card/driver, but that's more work than anyone is willing to do. Even so, some games (like Rust) have an excessive amount of shaders, which adds a lot of extra waiting time.
2. Unavoidable: The main one here is textures. If you want it to be quickly accessible in VRAM, then it's going to be uncompressed, which means either having some very large files to copy over, or having some semi-large files to decompress. You can save quite a bit of time by micromanaging the texture resolution of each asset to be just high enough. Apart from textures, you have vertex data and other assets; all of which have basically the same considerations.
When it comes to regular software, most UI designs don't have big assets. They aren't loading billions of triangles or thousands of textures. This means there really isn't much need for long startup times.
I don't understand why desktop app user interfaces aren't blazingly fast and 240fps.
That being said, I think current iOS and macOS can run their UIs at 120fps (and Apple devices still seem to have decent battery life.)