Comment by UncleOxidant
Comment by UncleOxidant 3 days ago
How long does this typically take? It sounds time consuming. Also, it seems like this could be similar to doing a GA?
Comment by UncleOxidant 3 days ago
How long does this typically take? It sounds time consuming. Also, it seems like this could be similar to doing a GA?
Not today but we will implement memoization of kernels for each hardware backend, yes.
You can also set a time budget for how long you'd like the search to run for to avoid wasting time on diminishing returns.
That depends on the model architecture and how it was written since that informs the size of the search space.
The typical range is 10 mins to 10 hours. It won't be fast but you only have to do it once and then those optimizations are set for every forward pass.