Prompts as a function of state?

[jmandel]

I've been having a lot of fun getting started with langchainjs. I'm working on some chains + tools for validating coded terms in healthcare data.

In some situations, text-based prompt templates feel over-constrained, or hard to use idiomatically. Maybe it's from playing with web frameworks like React, but I've been thinking about "prompts as a function of state". I've got a simple demo of this working (using langchainjs BasePromptTemplate under the hood) where the prompt code is just the "function from state to chat history". Here's an example to show:

• Branching logic (e.g., conditionally include content)
• Composition (e.g., to embed a sub-prompt)
• Asynchronous calls to generate content for some slots

In the end you use it like any other template, e.g.

const c = new LLMChain({llm, prompt: templatize(jokePrompt)});

async function randomNumbers(min = 100, max = 1000) {
  let random = await fetch(
    "https://www.randomnumberapi.com/api/v1.0/random?min=${min}&max=${max}&count=5"
  );
  return random.text();
}

interface PersonState {
  firstName: string;
  familyName: string;
}

const ai = (s: string) => new AIChatMessage(s);
const human = (s: string) => new HumanChatMessage(s);
const system = (s: string) => new SystemChatMessage(s);

const lightBanter = async (state: PersonState) => [
  human("Do you know some random numbers?"),
  ai(`Sure, I know ${await randomNumbers(100, 100 * state.firstName.length)}`),
  human("Please invoke your famous conversational skills to count them, then come up with a witty reference using this count."),
];

export const jokePrompt = async (state: PersonState) => [
  system(`You are a droll and delightful conversationalist.`),
  human(`Hi there, I'm ${state.firstName}.`),
  ...await lightBanter(state),
  ai(
    `Hello ${state.firstName}, what a lovely name.
    I bet your last name is ${state.familyName}, which is ${
      state.familyName.length
    } letters long.
    ${
      state.firstName.startsWith("A")
        ? "My favorite names start wiht A."
        : state.firstName.startsWith("Z")
        ? "How bold, the Zed at the start of your name."
        : ""
    }
   `
  ),
  human(`So tell me a new joke, incorporating details from our banter.`),
];

Is this an approach worth pursing? Is it already covered by a core component that I'm missing, or could there be room for something like it langchain's core abstractions? Or am I just struggling because I haven't yet seen the light? :-) This can be hard to tell when first picking up a new framework... but at the same time, it's hard to look at a framework "fresh eyes" after you've been in it for a while.

[arnaugomez]

Hi @jmandel, as a React developer myself, I find your proposal interesting. In today's AI workflows, prompts are becoming increasingly more complex. And a prompt might have many variants and subsections, so you might want to approach buiding a prompt the same way you would build a piece of UI with React.

With the tooling that we currently have at Langchain, this is how I would build a complex prompt

1. Create a prompt template with some slots to fill. These will be the customizable parameters of the prompt
2. Generate the values of the parameters in a previous step of the chain.

Here is an example:

const chain = new RunnablePassthrough<{
  favorite: string;
  isPromptEngineer: boolean;
}>()
  .pipe({
    question1: () => "What is your name?",
    question2: PromptTemplate.fromTemplate("What is your favorite {favorite}"),
    question3: RunnableLambda.from(({ isPromptEngineer }) =>
      isPromptEngineer
        ? "What is your favorite LLM?"
        : "What is your favorite color?"
    ),
  })
  .pipe(
    PromptTemplate.fromTemplate(
      "Act as a developer. Answer the following questions: {question1}, {question2}, {question3}"
    )
  );

If you need to use a different prompt altogether based on some logic, you might want to use RunnableLambda.from(). This method lets you return a new Runnable. So you can use it to build prompt templates dynamically inside the RunnableLambda.from() method.

https://js.langchain.com/docs/how_to/routing#using-a-custom-function