> We now believe Astra meets the Critical cybersecurity capability threshold under our Preparedness Framework, meaning that with the right tools and access, it can find previously unknown security flaws and develop ways to exploit them across many well-protected systems without a person guiding each step.
> We plan to make Astra available soon[, but access to its most advanced cybersecurity capabilities will be more limited].
The code in compilers is the closest to your typical app you can get in a benchmark like SPEC, eveerything else is actually far more specialized. Compiler code is full of small basic blocks, lots of branches, indirect memory access; it's actually harder to get good performance for such code, both for CPUs and compilers (that was part of the death of Itanium too).
That's true of most of the applications in SPECint (SPECfp is a different matter); there's nothing special about compilers there.
Where compiler code is going to get really unusual, I suspect, is that compilers tend to be a little mono-focused on relatively few data structures. I know I was able to get measurable (single-digit percent!) performance differences in LLVM making very small tweaks to layout in llvm::Value. By contrast, when I was working on Thunderbird, the only similarly small change I could think to make that kind of difference would be to "oops, all string functions are now a cross-DLL call" (and even then, only because string handling is so dominant in that kind of application). Another kind of difference is that the compiler-based benchmarks are going to be quite light in virtual or indirect function calls (there's more of an emphasis on switch-based dispatching than vtable-based dispatching in most compiler implementations), which is going to make it a poorer proxy for some kinds of applications.
Seconding this - having worked on LLVM and Firefox, the performance tuning of each application was very different. Even measuring the performance of an application like Firefox (in a meaningful way) is non-trivial, wheras compilers are much more approachable with traditional profilers (either tracing or sampling).
I mildly disagree - depending on your definition of "typical app". Most applications have much greater use of multi-processing and concurrent cross-thread (or cross-process) communication. Compilers, aside from high-level parallelism across modules, tend to be quite single-threaded applications.
If you're solely interested in single-core performance, then I would agree that they are a good stress test, but I think for a processor that is being sold on it's parallelism, they are not a great benchmark.
In the history of the SPEC benchmarks, the compiler benchmarks, like gcc, have been the best predictor of CPU performance for the applications that cannot benefit from array operations, so they cannot use the vector or matrix instruction set extensions.
The reason is that for the other benchmarks the CPU vendors have always succeeded sooner or later, to tweak their compilers and compiling options, or even the hardware of the CPUs, in order to get improved benchmark results that nonetheless are not indicative of the improvements in other applications.
On the other hand, the compiling benchmarks, like with gcc, and now also with clang, are too diverse in CPU resource usage and no special feature of the CPU has a significantly greater weight than others, so special tricks to enhance the benchmark results have never been found.
When looking at the past SPECint results, the values of the gcc benchmark remain the most reliable relative performance estimator.
I doubt that this will change in the near future.
Moreover, the multi-threaded compiling benchmark is also very useful, because it matches exactly a real-world workload that is extremely frequently encountered. Due to the great clock frequency difference between running a benchmark on a single thread and running it on all available threads, the single-threaded results have a very poor correlation with the multi-threaded results.
Death of Itanium was that it was a) VLIW and b) Intel was too arrogant. So it went to the same destination as later Larrabee and ATI/AMD attempts at VLIW GPUs.
That is, nowhere.
Also you are wrong and anyone sizing up an arch to put their loads onto must first try that load on it and not rely on "bah, compilers compile on it".
VLIW works for DSP applications, it's not an instant dead end. It's a good fit in cases where code path and memory accesses are predictable, like shader code.
Essentially every cell phone out there has a VLIW DSP like Qualcomm's Hexagon cores (though AFAIK Qualcomm is the only one who lets you run your on programs on their DSP).
In a hypothetical world where AMD wasn't allowed to do AMD64, and Intel stayed committed to Itanium: Itanium would still have sucked, and both PowerPC and SPARC would have out-sold Itanium by an even wider margin than they did in this reality. Itanium could only have succeeded if AMD64 wasn't possible and literally all of the competing 64-bit architectures were killed off by their owners so they could jump on the Itanium bandwagon. Itanium managed to kill off PA-RISC and Alpha and (high-end) MIPS roadmaps, but it still had competitors that were not just viable but actually more successful.
If you ignore VLIW, you just have a very unexciting RISC ISA, but because of the VLIW, you get extra scheduling info that most RISC designs don't provide which might be advantageous. The real question is actually about the code density of 41-bit instructions and if it can be offset by the 128-bit package (and perhaps something like allowing new 24-bit compressed instructions).
Poulson already somewhat proved part of this as it added back a traditional frontend and even added some OoO capabilities and 4-way SMT. It wasn't earth-shattering, but it wasn't absolute garbage either.
The contemporaneous IBM POWER ISA was implemented in superscalar CPU cores with out-of-order execution and with SMT and it would provide superior performance in any equivalent fabrication technology.
The Itanium ISA actually had a few nice features, but it also had other bad features that outweighed the good features. Besides the static instruction scheduling in bundles, there was also the handicap of using SPARC style register windows, which slowed-down context switches.
The second version of HP PA-RISC, which was too quickly replaced by Itanium, would have had good chances of providing superior performance in comparison with Itanium, had it not been abandoned without a fight.
Um...while not "OG Windows" I guess, PowerPC was an officially supported and shipping Windows NT 3.51 & 4 target[1]. It was released a couple of years before XP 64-bit for itanic. SPARC was a planned NT port, and while it never happened, it could have[2][3].
I have the opposite problem with Code, it barely writes any comments even for fairly complex C++ code blocks (systems programming stuff), have to prompt it to add extra details about why it did something. Request in agents.md doesn't seem to help much.
/code-review in Claude Code spawns a lot of sub-agents (counted like 8 once), each looking at the code from some certain aspect (like correctness, maintainability, duplication, testing, etc). It eats tokens like crazy doing that, but also covers quite a lot. The default code review in Codex does far less (feels like it's only correctness) and doesn't uses subagents. Actually I made a skill for Codex that does a review closer to what Claude does by default, but using like 4-5 agents and some being cheaper models/less than xhigh reasoning. I'm getting pretty nice reviews with that that cover more than just correctness.
MSVC (Microsoft's C++ compiler) had an pretty advanced inter-procedural (LTO) way of doing devirtualization, but it was so buggy and slow that it eventually got disabled. It was trying to prove that a pointer can only target a certain class all the time (or maybe a couple), but things get really messy with typical C/C++ code and even worse once you have DLLs which may inject new derived classes.
Sadly for them, Nvidia didn't stay still in the meantime and created the next generation of CUDA, CuTile for Python and soon for C++, through CUDA Tile IR (using a similar compiler stack based on MLIR).
Event though it's not portable, it will likely have far greater usage than Mojo just by being heavely promoted by Nvidia, integrated in dev tools and working alongside existing CUDA code.
Tile IR was more likely a response to the threat of Triton rather than Mojo, at least from the pov of how easy is to write a decently performing LLM kernel.
And for not staying behind, Intel and AMD are doing similar efforts, and then we have the whole CPython JIT finally happening after so many attempts.
Not to mention efforts like GraalPy and PyPy.
And all these efforts work today in Windows, which is quite relevant in companies where that is the assigned device to most employees, even if the servers run Linux distros.
I keep wondering if this isn't going to be another Swift for Tensorflow kind of outcome.
People keep mistaking Mojo as good syntax for writing GPU code, and so imagine Nvidia's Python frameworks already do that. But... would CuTile work on AMD GPUs and Apple Silicon? Whatever Nvidia does will still have vendor lock-in.
Indeed, but Intel and AMD are also upping their Python JIT game, and in the end Mojo code isn't portable anyway.
You always need to touch the hardware/platform APIs at some level, because even if the same code executes the same, the observed performance, or in the case of GPUs the numeric accuracy has visible side effects.
It is portable in that you can write code to target multiple platforms in the same codebase. Mojo has powerful compile-time metaprogramming that allows you to tell the compiler how to specialise using a compile-time conditional, e.g. https://github.com/modular/modular/blob/9b9fc007378f16148cfa...
Of course, this won't be necessary in most cases if you're building on top of abstractions provided by Modular.
You don't get this choice using vendor-specific libraries; you're locked into this or that.
Yes you do, you get PyTorch or whatever else, built on top of those vendor-specific libraries.
That is the thing with Mojo, when it arrives as 1.0, the LLM progress and the investment that is being done in GPU JITs for Python, make it largely irrelevant for large scale adoption.
Sure some customers might stay around, and keep Modular going, the gold question is how many.
Have you ever wondered how much work would have been saved by the Pytorch team if they could have used just Cuda for all the platforms they support? If they didn't have to write compatibility abstractions or layers, and instead just focused on the problem of training neural networks? What if all the primitives they used from Cuda and cuDNN worked just as well on AMD GPUs, Apple GPUs, and probably Google's TPUs as they did on Nvidia GPUs?
Mojo and Modular's Max platform would do to heterogeneous compute what LLVM did to programming language development. People who dismiss the real value offering here know nothing. Modular have already raised $350m+ from industry giants (including Nvidia and Google) to solve this, and I believe they will.
Yes, because one of my hobbies was graphics programming for a long time, and I keep observing all the time how FOSS folks misunderstand the games industry, what gets talked at GDC and IGDA events, isn't one API to rule them all.
> What if all the primitives they used from Cuda and cuDNN worked just as well on AMD GPUs, Apple GPUs, and probably Google's TPUs as they did on Nvidia GPUs?
Why should they? CUDA is a GPGPU paradigm, AMD/Apple/Intel all ship diverse raster-focused hardware, and TPUs are a systolic array. How much can you realistically expect to abstract with unified primitives? How much performance do you perceive to be left on the table with native CUDA-based implimentations?
Pytorch's abstractions answer this by ignoring raster hardware conventions entirely. The underlying ATen library is basically a CUDA wrapper, which is not much of a surprise since nobody else is willing to standardize a better alternative. We learned as much when OpenCL died, and now that Khronos is riding into the sunset it's unlikely we'll even see that level of paltry early-2010s cooperation. Mojo really should have taken Vulkan's lessons to heart; you need stakeholders to succeed, simply "disrupting" the proprietary status quo is a recipe for coming dead last in adoption rates.
> People who dismiss the real value offering here know nothing.
So explain the value, then. This is not an "optimize this IR for MIPS and x86" problem, the Lattner Fairy can't shoehorn shaders into every CUDA Compute Capability to make raster GPUs a viable GPGPU platform. If you followed Geohot's gradual descent into (sadly, quite literal) insanity then this would have been glaringly obvious from the offset. Tinygrad has an IR, industry-scale support, multiplat deployment, and it's still a dumpster fire. The project exacerbated all of the issues in ROCm and Metal, without contributing to any form of upstream cooperation between CUDA's competitors. If Mojo goes the same route with a more ambitious goal, they'll end up entrenching CUDA and obsoleting themselves. As much as people hate to admit it, CUDA is less of a software moat and more of a hardware one.
> Why should they? CUDA is a GPGPU paradigm, AMD/Apple/Intel all ship diverse raster-focused hardware, and TPUs are a systolic array. How much can you realistically expect to abstract with unified primitives?
Ah, it seems impossible to you. These are very different hardwares... It is hard enough to make compatibility among different hardwares of the same vendor. Very difficult to imagine building primitives for hardwares with completely different memory layouts.
> How much performance do you perceive to be left on the table with native CUDA-based implimentations?
Zero is the idea. And I wasn't saying there should be a native cuda-based implementation, I'm asking you to imagine how much easier everything would have been if Cuda was cross-platform without any performance or ergonomic penalties.
Mojo is a foundational step here. The big HOW is powerful parametric programming. So much information could be passed during compile time which the compiler uses to specialize.
> Ah, it seems impossible to you. These are very different hardwares...
In effect, they are completely different hardware. The only thing any of them have in common is rasterization primitives, so unless you're focused on render workloads you're nearly better off software-accelerating the language for CPUs instead. As a point of comparison, go look at early ray tracing implementations on GPUs that have no dedicated RT blocks or hardware-accelerated denoising. They were oftentimes slower than software-accelerating the same thing on a cheaper CPU.
> I'm asking you to imagine how much easier everything would have been if Cuda was cross-platform
...you are aware of what CUDA actually does, right? Mojo is not a cross-platform version of it. At risk of repeating myself, creating an OpenCL-style library without a Khronos-style consortium does not address the problem. CUDA is a hardware solution, you need to define standards around GPGPU programming because that's what Nvidia does internally. Ignoring industry-wide standardization is the express bet that Nvidia and Google are making, investing in Mojo. Without any hardware stakeholders, Mojo's only opportunity is to become a DirectML/ONNX-style middleware like @pjmlp suggested. Spoiler: that's not a super disruptive or successful goal. Certainly not an LLVM scale opportunity.
> So much information could be passed during compile time which the compiler uses to specialize.
Like what? None of this is a rhetorical question, explain to me what information would help bring GPUs and TPUs up to parity with CUDA. What information are you thinking of that isn't currently expressed in SPIR-V and MLIR? What optimizations do you have in mind?
> In effect, they are completely different hardware. The only thing any of them have in common is rasterization primitives
They can also do matrix multiplication, which is the relevant thing here.
Mojo is not an OpenCL-style library; it is a programming language, and it doesn't really abstract much. However, it is designed to support compilation to exotic hardware. Cuda CPP can also be thought of as a language; however, the compilation model and core primitives are fundamentally tied to Nvidia hardware.
> Like what? None of this is a rhetorical question, explain to me what information would help bring GPUs and TPUs up to parity with CUDA.
My understanding from speaking with a few Tile IR devs on dates is that its primary motivation was providing better portability for programming tensor cores than PTX offers. Nobody ever told me they saw it as a response to anything other than customer feedback.
Maybe OP was thinking about compilers "cracking" certain SPEC benchmarks: implementing exactly the optimization needed to boost a benchmark quite a lot, but that opt. probably won't apply to any other code out there (usually it's so targeted and risky with general C/C++ code that intentionally it doesn't work on anything else). That happened a couple of times over the years, I know about the Intel compiler cases for ex. I can certainly see LLM providers adding tricks that help a certain class of benchmarks, but doesn't help much for anything else.
They won't, Windows isn't even compiled with AVX2 extensions on, even though every CPU in the last 10y+ supports it, for the fear of it running on some machine that doesn't have AVX2. The whole "CPU unsupported" thing is from the marketing side, trying to push purchase of new hardware, not from the actual devs.
reply