Comment by tossandthrow

Comment by tossandthrow 3 days ago

> We have mathematically proven that transformers can solve any problem, provided they are allowed to generate as many intermediate reasoning tokens as needed.

This is also the case with plain and regular RNNs

baq 3 days ago

Now just need an autoregressive transformer <==> RNN isomorphism paper and we're golden

Reply View 2 replies

logicchains 3 days ago

Plain RNNs are theoretically weaker than transformers with COT: https://arxiv.org/abs/2402.18510 .

Reply View | 1 reply
- tossandthrow 3 days ago
  
  The paper says transformers perform better than RNNs, which is not surprising.
  However, they are both, theoretically, Turing complete computers. So they are equally expressive.
  
  Reply View | 0 replies

Intralexical 3 days ago

Isn't it also expected to be the case with RNGs?

Reply View 0 replies