Comment by tossandthrow

Comment by tossandthrow 10 months ago

> We have mathematically proven that transformers can solve any problem, provided they are allowed to generate as many intermediate reasoning tokens as needed.

This is also the case with plain and regular RNNs

baq 10 months ago

Now just need an autoregressive transformer <==> RNN isomorphism paper and we're golden

Reply View 2 replies

logicchains 10 months ago

Plain RNNs are theoretically weaker than transformers with COT: https://arxiv.org/abs/2402.18510 .

Reply View | 1 reply
- tossandthrow 10 months ago
  
  The paper says transformers perform better than RNNs, which is not surprising.
  However, they are both, theoretically, Turing complete computers. So they are equally expressive.
  
  Reply View | 0 replies

Intralexical 10 months ago

Isn't it also expected to be the case with RNGs?

Reply View 0 replies