Comment by 4ad

Comment by 4ad 5 days ago

3 replies

This is the system prompt it uses:

    You are an expert AI assistant that explains your reasoning step by step. For each step, provide a title that describes what you're doing in that step, along with the content. Decide if you need another step or if you're ready to give the final answer. Respond in JSON format with 'title', 'content', and 'next_action' (either 'continue' or 'final_answer') keys. USE AS MANY REASONING STEPS AS POSSIBLE. AT LEAST 3. BE AWARE OF YOUR LIMITATIONS AS AN LLM AND WHAT YOU CAN AND CANNOT DO. IN YOUR REASONING, INCLUDE EXPLORATION OF ALTERNATIVE ANSWERS. CONSIDER YOU MAY BE WRONG, AND IF YOU ARE WRONG IN YOUR REASONING, WHERE IT WOULD BE. FULLY TEST ALL OTHER POSSIBILITIES. YOU CAN BE WRONG. WHEN YOU SAY YOU ARE RE-EXAMINING, ACTUALLY RE-EXAMINE, AND USE ANOTHER APPROACH TO DO SO. DO NOT JUST SAY YOU ARE RE-EXAMINING. USE AT LEAST 3 METHODS TO DERIVE THE ANSWER. USE BEST PRACTICES.
The Python crap around it is superfluous.

Does it work? Well not really:

https://lluminous.chat/?sl=Yjkxpu

https://lluminous.chat/?sl=jooz48

I have also been using this prompt, and while it fails on then problem above, it works better for me than OPs prompt:

    Write many chains of thought for how you’d approach solving the user's question. In this scenario, more is more. You need to type out as many thoughts as possible, placing all your thoughts inside <thinking> tags. 
    Your thoughts are only visible to yourself, the user does not see them and they should not be considered to be part of the final response.
    Consider every possible angle, recheck your work at every step, and backtrack if needed.
    Remember, there are no limits in terms of how long you can think - more thinking will always lead to a better solution.
    You should use your thoughts as a scratchpad, much like humans do when performing complicated math with paper and pen. Don't omit any calculation, write everything out explicitly.
    When counting or maths is involved, write down an enormously verbose scratchpad containing the full calculation, count, or proof, making sure to LABEL every step of the calculation, and writing down the solution step by step.
    Always remember that if you find yourself consistently getting stuck, taking a step back and reconsidering your approach is a good idea. If multiple solutions are plausible, explore each one individually, and provide multiple answers.
    Always provide mathematical proofs of mathematical answers. Be as formal as possible and use LaTeX.
    Don't be afraid to give obvious answers. At the very very end, after pages upon pages of deep thoughts, synthesize the final answer, inside <answer> tags.
In particular it solves this problem: https://lluminous.chat/?sl=LkIWyS
astrange 4 days ago

That second prompt is interesting. Not magic though. I tried it with every other model I know and they're still basically unable to do:

* give me three sentences that end in "is"

* tell me the line of Star Spangled Banner that comes before "gave proof through the night"

But they did some good thinking before failing at it…

  • anonzzzies 4 days ago

    > Not magic though

    It's just a pile on of trial and error instructions (maybe learned from previous 'projects', but). There is no magic or skill to prompt 'engineering' anywhere.

    • astrange 3 days ago

      Skill is just learning from trial and error.