I strongly dislike CUDA. Once you have allowed that proprietary cr*p into your C++ codebase, it is very hard to get rid, and you end up with code that is either tied to a single vendor or an #ifdef hell, probably both.
The best way to program GPUs is face up to the reality that they are not the same machine as the CPU, write your kernels in separate files, and launch them manually, like in Metal, OpenCL, and D3D12, etc.
These days we even have DSLs like Triton that make kernel writing much more ergonomic than anything you would hope to achieve in Rust.
I highly recommend Julia for (scientific) GPU programming but it would be nice if there was a larger community and/or funding behind the GPU side of things. It has very few core devs for what it is.
Julia has had a great CUDA story for a few years now, and this about 9 days old. Rust rejects buffer aliasing at compile time using Rust's borrow checker, but shared memory in cuda-oxide currently requires unsafe, but then there's HuggingFace's Grout and mistral.rs, so yeah, Rust is picking up ground here on Julia. How is OpenCL's performance these days?
I fell in love with MATLAB (or GNU Octave for free since you really pay for toolboxes/packages) back around 2004, despite it warts. So I second Julia, which is similar, but is a more modern functional language instead of imperative.
I asked Google's Gemini if Julia can run on GPU unmodified without annotations, pragmas, intrinsics or similar manually-managed friction, and it said yes, but that data types must be swapped out for GPU-backed types:
If your code is written using vector/matrix operations, broadcasting, or standard linear algebra functions, it can run on the GPU entirely unmodified. You only need to change the input data type to a GPU-backed array (e.g., swapping a CPU Array for a CuArray from CUDA.jl).
# A standard Julia function — completely agnostic to hardware
function custom_math!(C, A, B)
@. C = sin(A) + 2 * B # Normal broadcasted operation
end
# Running on the CPU:
A_cpu = rand(1000)
B_cpu = rand(1000)
C_cpu = similar(A_cpu)
custom_math!(C_cpu, A_cpu, B_cpu)
# Running on the GPU (Unmodified function!):
using CUDA
A_gpu = CuArray(A_cpu)
B_gpu = CuArray(B_cpu)
C_gpu = similar(A_gpu)
custom_math!(C_gpu, A_gpu, B_gpu) # Automatically compiles to native PTX!
This is the direction we should be going. So while Nvidia's Rust port is an important first step, it's an evolutionary rather than revolutionary achievement. But that's all Nvidia can really do now, since it's locked into its own paradigm like Intel/Microsoft and has gotten too big to think outside the box.
Edit: PTX in its example stands for Parallel Thread Execution, the Virtual Machine (VM) Instruction Set Architecture (ISA) created by NVIDIA for its GPUs, which works similarly to Java byte code.
Edit 2: Broadcasting is a feature in Julia that allows you to apply a function or mathematical operation element-by-element across arrays of different shapes and sizes, without writing manual loops. In Julia, broadcasting is syntactically indicated by a dot (.) placed before an operator or function name (e.g., sin.(x) or .+). <- I was today years old when I learned the term for this
jacobgorm · · focus · HN ↗
The best way to program GPUs is face up to the reality that they are not the same machine as the CPU, write your kernels in separate files, and launch them manually, like in Metal, OpenCL, and D3D12, etc. These days we even have DSLs like Triton that make kernel writing much more ergonomic than anything you would hope to achieve in Rust.
melodyogonna · · focus · HN ↗
adgjlsfhk1 · · focus · HN ↗
patagurbon · · focus · HN ↗
eggy · · focus · HN ↗
zackmorris · · focus · HN ↗
I asked Google's Gemini if Julia can run on GPU unmodified without annotations, pragmas, intrinsics or similar manually-managed friction, and it said yes, but that data types must be swapped out for GPU-backed types:
If your code is written using vector/matrix operations, broadcasting, or standard linear algebra functions, it can run on the GPU entirely unmodified. You only need to change the input data type to a GPU-backed array (e.g., swapping a CPU Array for a CuArray from CUDA.jl).
<a href="https://cuda.juliagpu.org/stable/" rel="nofollow">https://cuda.juliagpu.org/stable/This is the direction we should be going. So while Nvidia's Rust port is an important first step, it's an evolutionary rather than revolutionary achievement. But that's all Nvidia can really do now, since it's locked into its own paradigm like Intel/Microsoft and has gotten too big to think outside the box.
Edit: PTX in its example stands for Parallel Thread Execution, the Virtual Machine (VM) Instruction Set Architecture (ISA) created by NVIDIA for its GPUs, which works similarly to Java byte code.
Edit 2: Broadcasting is a feature in Julia that allows you to apply a function or mathematical operation element-by-element across arrays of different shapes and sizes, without writing manual loops. In Julia, broadcasting is syntactically indicated by a dot (.) placed before an operator or function name (e.g., sin.(x) or .+). <- I was today years old when I learned the term for this