Comment by elchananHaas

This is correct. I will add that sampling from the distribution thereafter is equivalent to on policy learning.