[JS] Difference between 0.9.0 and 0.9.3 in Output Instruction Following for Gemini

[i2amsam]

[Forking off from #1360 to not have two different cases being disussed]

Logging a reproducible case of differences in output conformance between 0.9.0 and 0.9.3 in Gemini

I noticed this in the character generate example template and have isolated a repeatable example

Code

import { genkit, z } from 'genkit';
import { googleAI, gemini15Flash } from '@genkit-ai/googleai'

const ai = genkit({
    plugins: [googleAI()],
});

const prompt = ai.definePrompt({
    name: 'Character Prompt',
    model: gemini15Flash,
    config: {
        temperature: 0,
    },
    input: {
        schema: z.object({
            inspiration: z.string()
        })
    },
    output: {
        format: 'json',
        schema: z.object({
            name: z.string(),
            strength: z.number(),
            intelligence: z.number(),
            description: z.string(),
        })
    },
}, `
You're a expert DnD designer, create a new character.

Base the character on {{inspiration}} but don't make it
and exact match.`
);

(async () => {
    console.log((await prompt({ inspiration: "Yogi Berra" })).output);
})();

0.9.0 Buggy Version Output (extra fields hallucinated)

{
  name: 'Yogi Ursa',
  strength: 14,
  intelligence: 12,
  description: "A grizzled, bear-like half-orc ranger with a penchant for cryptic pronouncements and an uncanny knack for finding the best berry patches (and trouble).  He's seen it all in his years traversing the wilds, and while his wisdom might seem questionable at times, his instincts are sharp as a badger's claw.  He often mutters things like, 'It ain't over 'til it's over... or maybe it is, who knows?'  Don't let his folksy demeanor fool you; he's a formidable tracker and surprisingly adept at diplomacy, even if his words are often contradictory.  He carries a well-worn hunting bow and a surprisingly large knapsack always overflowing with 'essentials'.",
  dexterity: 16,
  constitution: 15,
  wisdom: 10,
  charisma: 8,
  race: 'Half-orc',
  class: 'Ranger (Hunter)',
  background: 'Outlander',
  alignment: 'Chaotic Good',
  skills: [ 'Survival', 'Perception', 'Stealth', 'Athletics', 'Persuasion' ],
  feats: [ 'Sharpshooter' ],
  equipment: [
    'Longbow +20 arrows',
    'Hunting Knife',
    "Backpack overflowing with berries, nuts, and various other 'useful' items",
    'A well-worn map covered in cryptic markings'
  ]
}

0.9.3 Good Output (follows format specification, just the fields requested)

{
  description: "A surprisingly insightful and strong halfling rogue, known for his malapropisms and uncanny ability to find himself in (and out of) trouble.  He's got a knack for picking pockets and a surprisingly deep understanding of strategy, though he might describe it a bit... unconventionally.  Always seems to be one step ahead, or maybe one step behind, it's hard to tell.",
  intelligence: 14,
  name: 'Yogi Bearington',
  strength: 10
}

In this case the changes from 0.9.0 -> 0.9.3 are good, but maybe indicate that Gemini is responding different based on those changes.