FYI, virtual desktop (and steam link) on quest 3 recently got beta updates for wired modes (they just tunnel ethernet afaik, but its better than nothing for your usecase probably) since the OS has seemingly recently unblocked this.
But sadly not (officially) in 3D! SuperDepth3D is such a cool concept, I don't need most of my games to be full VR immersive camera experiences, but a nice big virtual full no-limits 3D monitor would be sick.
That’s how it works with the Quest 3 on an updated OS. If you’re using hand tracking then you gaze + tap thumb and finger together to select, or pinch and hold to grab a corner for resizing or a border for dragging.
That’s not how it works, there is no gaze support. I just double checked and couldn’t find it in any Accessibility or Experimental settings. Apologies for the misinfo.
(Also no inward-facing cameras ofc)
What I was thinking of was on iPhone. It was Eye Tracking + Dwell for selecting.
Hands act as a pointer on the Quest, with a pinch action to select and drag.
afaik the AVP has a dedicated Lidar sensor for gesture detection. I'd say Apple has managed to patent more basic stuff than that (pinch-to-zoom, rounded corners,...)
Oh, that'd make sense. I think the quest 3 has a depth sensor, but they mostly use that for room setup. I don't even think it has IR emitters for low light.
But that's the age old question about patents, if they're being granted too much they stifle innovation and competition instead of the opposite... (which is the point you were making as well).
On the quest 3, I've felt like the screen is just a bit too low res to use comfortably as a monitor anyways, so I imagine the Frame won't be much different. The better weight distribution should help a lot though.
It's absolutely wild to me that they didn't include, and have devs ship profiles for some of their games, SuperDepth3D or some other depth map shader officially in some form in the Steam UI for this. If a big focus is playing your pancake library on a huge screen (due to there not being not many native VR games), having that huge screen be 3D would be a huge value add. And they're big enough to get some developers to ship profiles for their games officially guaranteeing some form of workingness(?).
This. This is what I am looking for. I remember using iz3d years ago, with their anaglyph renderer. Playing Half Life 2, Halo and others in stereoscopic 3D was a great experience. Dead Space was particularly nice, as the game uses no HUD.
You can already do a lot of it with ReShade shaders like the aforementioned SuperDepth3D, but it requires a game profile, tons of fiddling and non-incompatibility from the game. Having that shipped as dev-tested profiles with games would be the exact sort of thing that makes proton nice to use on the steam deck
They seem to be getting pretty far with only vision, to be fair. Same with openpilot.
Still, I cant help but agree. If you have access to beyond-human sensors and are able to do sensor fusion, then please do it. What's the point otherwise? saving a few hundred bucks per car?
It's kind of insane how having this tech at this speed 5 years ago would have probably still been seen as insanely useful and revolutionary. If LLMs were more capable but dramatically slower, I wonder how it would impact how we use it? Dramatically more thought being put into prompts, much more preparation probably
The future of inference is likely in ASICs, so we'll get the inverse, a bit less capable than frontier but super fast models. Like this 14k tok/s beast https://chatjimmy.ai/ from Taalas (who got acquired by AMD recently).
GPT-6-astra runs at like ~40 tok/s, I have a hard time imagining what could be accomplished with that type of model at 10k+ tok/s when in the hands of the public. Will certainly make cybersecurity a challenge for older systems.
To the extent that cryptocurrency moved off ASICs, it was because of interest shifting to different cryptocurrencies that were specifically designed to be harder to mine on an ASIC than Bitcoin's compute-heavy, memory-light hashing.
I'm not sure there's any reason to expect a similar shift from LLMs. The hardware used for training doesn't dictate what hardware needs to be used for inference, and nobody's going to design an LLM architecture with an overt intention to make it better suited to GPUs and hard to target with ASICs.
Yet it doesn't seem that ASICs will have any particular advantage over consumer hardware since AI is very memory heavy, which is (right now) expensive no matter how you package it. And the compute is just simple matrix multiplication, which is almost entirely what GPUs were meant to do anyway.
ASIC vs GPU doesn't make a ton of difference when both are relying on commodity DRAM; in that sense, LLMs are more like the anti-ASIC cryptocurrencies. But the actually interesting ASICs are the ones that ditch the commodity discrete DRAM chips. They lose out on the memory density and thus struggle to scale up to the largest models, but for what does fit onto a Cerebras wafer or a Taalas chip, the speed is phenomenal. They have a real shot at securing the "smart enough, and really fast" segment of the market.
And it seems more plausible to me that an ASIC architecture rather than GPUs would be able to best make use of something like wafer-bonded custom memory to approach the density of discrete DRAM while retaining the extremely high bandwidth that comes with arbitrarily wide interfaces and minimal PHYs.
Go back and correct your idea that consumer hardware made asics obsolete. Then we can figure out if asic or asic like devices for inference will have no advantage.
Yeah! Nobody needs chatjimmy.ai. Nobody needs their results to come back instantly instead of at 10 tokens per second. Nobody needs a CPU faster than a megahertz.
My thought was that ASICs turned out not to be worth it for crypto mining because consumer hardware evolved fast enough to do it, while also being cheaper and having some resale value, while ASICs are useless besides mining and have no resale value.
So I'm extrapolating this same idea to LLM inference.
For the problems ASICs exist they vastly outperform general hardware. Typically both in absolute speed and efficiency.
But it's only possible to make custom ASICs when you have a specific problem to solve. For newer crypto systems they can vary enough parameters that building a flexible enough ASIC to recoup the investment before the algorithm changes and makes your hardware useless.
For problems where the problem to solve remain in the problem space the ASIC can solve there is no point to use a thing else.
At 1t/s it's still faster than humans for a lot of tasks, basically doing overnight what could take humans half a week. Plus you can always parallelize.
This is what people forget when they see slow performance: at 1 t/s it's still roughly the equivalent of having another person work for you at no extra cost besides the initial purchase/sign-on-bonus.
Frontier models are amazing, but what will really be useful for us is having models and hardware so efficient that you can run useful LLMs locally. One of my favourite LLMs to this day is still my jail-broken gemma4 12b because it's small enough to run on my computer, but also 100% local and free as in liberty.
I don't really do anything at my job that could be done in a single prompt, and certainly not something that would take me 7 hours. A 7 hour task would take multiple iterations with how LLMs are right now.
You said it would do overnight what would take a human half a week, so I'm curious what tasks you are doing where AI is 3x faster than you even at 1t/s
you could give it an entire spec, let it rip through everything that it is assigned and let another model correct the failures. Or similarly there's uses as well as an asynchronous adversarial review agent which basically ends up being an extra pair of "hands" to help out where performance isn't critical.
Your prompts are probably very underspecified then. Frontier models one shot the majority of my prompts. UI is kind of the exception, there I do have to ask for a lot of tweaks.
Maybe. Do you have an example of a prompt you've done recently?
I don't think it's valuable at all to try and craft a specific prompt to "one-shot" a task, I'm way faster just doing small asks and guiding it. At 1t/s, how would you even know your prompt was insufficient? It would take you 7 hours to see it.
Stuff along the lines of implement controller service and tests for the following endpoints:
- list of many endpoints with the JSON they receive and return and description of what they need to achieve
Stuff you could probably do in a single work day if you lock in and enter flow state, but in a typical job takes like half a week. And the vast majority of times the AI one shots it with no bugs, where I would have copy paste errors or dumb stuff I'd need to fix before it's shippable.
Like I said, in my experience I would rather do this with a bunch of tiny prompts than try to craft something that oneshots it. Chances are it will do several things I don't want, no matter how clear I think I'm being.
Write the program, punch the cards, send the cards to another building to be loaded, program runs, printout comes out in another building, somehow this takes 2-3 days
That's because 5 years ago it was still brand new for a computer to be able to speak English. 5 years later, we have accepted that LLMs can speak English and we expect them to do useful things.
running my own locally. I just set the tasks to start when systems go idle over x. Read and copy only to external drive projects, codes, ect for review. I review the reports the changes and apply them myself or correct them. Is it slower than say throwing it into fable yes. But I don't have to be monitoring it 24/7
The using a J lens is super cheap compared to inference. You basically add a single matrix multiply per layer. You probably wouldn't even notice the overhead in a good implementation.
You should even be able to create a j lens from scratch, but it might take a while. I was able to do it in a few hours on an H100. Creating the J lens is basically the equivalent of calculating a few thousand training steps for a model (256,000 backprops in my case). I've got more details in a blog post:
I've been experimenting with using J-Space (and final hidden layers) to extract the semantic meaning of words to improve TTS output accuracy using Qwen and Gemma models. My goal is to either map to an alternative token set where heteronyms are preserved or to output parenthesized IPA annotations for ambiguous words (with standard tokens). It's interesting to me that LLMs preserve this data throughout their processing but discard it in the final output.
I've also looked into extracting actions from J-Space to short-circuit a local assistant on low-end hardware. Are there any resources on how to do this training with inexpensive H100 instances (~$8/hr)? I would be opening up the final weights for Qwen/Gemma layers.
A reminder that google is the only major lab without a meaningful opt-out of training on your data. The only way to opt out is to disable message history entirely, which seems like a darkest of dark patterns to get users to leave "opt in" to training on, because next to nobody wants to use it without message history.
Within the general theme of token subsidies slowing down, this seems like the logical step.
In my mind the timeline goes something like this product wise:
- Pepole figure out that having a general purpose not-just-coding agent actually works with newer models, openclaw and its buddies spawn
- How do you get this to the general consumer? Offer integration with services even stronger than before and give the chat a cloud vm with persistent storage as one path, claude computer use as the other.
- - Perplexity computer releases, and they slowly start nudging users from plan-included chat usage to more stongly limit using or even extra credit usage billed computer tasks
- - chatgpt and other main labs do the same with work, but they can afford to subsidize it a bit more still by just having it use codex quota
We'll see where it goes from here, but i do see the general trend of pushing people towards strongly billed features without explicitly taking away the previous chat experience because that'd make them look bad
reply