Comment by pixl97

Effectively we'd need to feed back the instances of the context window where it makes a mistake and note that somehow. Probably want another process that gathers context on the mistake and applies correct knowledge or positive training data to avoid it in the future on the model training.

Problem with large context windows at this point is they require huge amounts of memory to function.