Comment by dheera
"Most people" didn't figure this out either, the top 0.01% did.
"Most people" didn't figure this out either, the top 0.01% did.
You're just moving the goal post & not addressing the question I asked. Why isn't AI optimizing the kernels in its own code the way people have been optimizing it like in the posted paper?
I read the paper. All the prerequisites are already available in existing literature & they basically profiled & optimized around the bottlenecks to avoid pipeline stalls w/ instructions that utilize the available tensor & CUDA cores. Seems like something these super duper AIs that don't get tired should be able to do pretty easily.
I also wouldn't be surprised if they used AI to assist themselves in small ways