Devil's Advocate Pattern

[kfsone]

One of the benefits of pair programming are those moments when they say "I'm going to call that function, , because I'm not worrying about performance" and you say "but this IS the core of the hot path, this method will happen inside the core loop for every iteration".

I see a lot of tokens wasted because a bad suggestion introduces a token and if its in an attention spot, some models will never let up (ask codellama to "implement std::forward. do not include utility. do not use remove_reference", and it will either say "this is my implementation not using remove_reference: .. forward ... remove_reference ... or it will give you one of the s/o alternatives to this question but with 'remove_pointer'.

What I want is a fan-out pattern: Agent A -> {Agent B + Agent C ...} (in parallel or serial) with a reduce return path. That is, Agent A will basically predict which response is better and return that.

Use cases: comparing alternatives from different models or different seeds or different specialists ("you care about perf, you care about idiomatic correctness") or different providers (gcp vs azure). Especially with access to memory this will be super useful for LMs with memory of a LARGE code base.

[kfsone]

Of course the natural progression would be arguing philosophers where Agent A bounces the responses back to each agent until it determines they've come to an agreement. But I'm more interested in the DAP at the moment.

For this one, imagine "which is better, rust or python?" and having your own personal carbon foot print dedicated to hosting the debate...

[sonichi]

Good idea. This is easy to implement by using register_reply to register a custom reply function for Agent A, in which it sends messages to Agent B, C... and collect their response and eventually return one.

[kfsone]

So would it be something like:
initial request -> Agent A
Agent A -> { agent b context + request -> agent b } -> capture Response 1
Agent A -> { agent c context + request -> agent c } -> capture Response 2
Agent A -> { response-handling header + response 1 + separator + response 2 }

Such that the LM supporting Agent A doesn't see the original request, it only sees the replies?

originator: "next implement a sieve of erasthenes(sp be damned)"
agent a: -> agent b direct (without going thru lm)
agent b -> agent b lm -> agent a
agent a -> agent c direct (without going thru lm)
agent c -> agent c lm -> agent a
agent a -> { "originator: ..., performance agent proposed: {response a}\n---\nmaintainability agent proposed: {response b}\n---\nyou must determine which is the more appropriate given the context, or ask the user for additional context" }

[sonichi]

Yes. Please let me know how it works if you have a chance to try.