Skip to content

Context Caching does not work with any Gemini Model #3362

@BraveEvidence

Description

@BraveEvidence

Describe the bug
Context Caching does not work with any Gemini Model gemini-1.5-flash-001 and gemini-2.5-flash-001

To Reproduce
I have below code to read a large text file and then use it as context for a chatbot

import { googleAI } from "@genkit-ai/googleai";
import { genkit } from "genkit";
import fs from "fs/promises";

const ai = genkit({
  plugins: [
    googleAI({
      apiKey: "MY_API_KEY",
    }),
  ],
});

async function main() {
  const textContent = await fs.readFile("./myfile.txt", "utf-8");
  const { text } = await ai.generate({
    messages: [
      {
        role: "user",
        content: [{ text: textContent }], // Include the large text as context
      },
      {
        role: "model",
        content: [
          {
            text: "This analysis is based on the provided text from War and Peace.",
          },
        ],
        metadata: {
          cache: {
            ttlSeconds: 300, // Cache the response to avoid reloading the full text
          },
        },
      },
    ],
    model: googleAI.model("gemini-1.5-flash-001"),
    prompt: "Who is Coding with nobody?.",
  });
  console.log(text);
}

main().catch(console.error);

now i have tried using both gemini-1.5-flash-001 and gemini-2.5-flash-001

when i use gemini-1.5-flash-001 i get error saying

Failed to create cache with key e335a709c48e1e32f5d9c25055b1d955c831d8305c645f8840c99538923f0ee3: Error: [GoogleGenerativeAI Error]: Error fetching from https://generativelanguage.googleapis.com/v1beta/cachedContents: [404 Not Found] models/gemini-1.5-flash-001 is not found for API version v1beta, or is not supported for createCachedContent. Call ListModels to see the list of available models and their supported methods.
GenkitError: INTERNAL: Failed to create cache: Error: [GoogleGenerativeAI Error]: Error fetching from https://generativelanguage.googleapis.com/v1beta/cachedContents: [404 Not Found] models/gemini-1.5-flash-001 is not found for API version v1beta, or is not supported for createCachedContent. Call ListModels to see the list of available models and their supported methods.
    at handleContextCache (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/googleai/src/context-caching/index.ts:77:13)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async handleCacheIfNeeded (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/googleai/src/context-caching/index.ts:118:37)
    at async <anonymous> (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/googleai/src/gemini.ts:1308:9)
    at async <anonymous> (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/core/src/action.ts:531:14)
    at async <anonymous> (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/core/src/action.ts:404:30)
    at async <anonymous> (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/core/src/tracing/instrumentation.ts:73:16)
    at async <anonymous> (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/core/src/tracing/instrumentation.ts:115:24)
    at async runInNewSpan (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/core/src/tracing/instrumentation.ts:103:10)
    at async newTrace (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/core/src/tracing/instrumentation.ts:62:10) {
  source: undefined,
  status: 'INTERNAL',
  detail: undefined,
  code: 500,
  originalMessage: 'Failed to create cache: Error: [GoogleGenerativeAI Error]: Error fetching from https://generativelanguage.googleapis.com/v1beta/cachedContents: [404 Not Found] models/gemini-1.5-flash-001 is not found for API version v1beta, or is not supported for createCachedContent. Call ListModels to see the list of available models and their supported methods.',
  traceId: 'cc38b24c0e7ca3c86f6c335ad391db65',
  ignoreFailedSpan: true
}

when i use gemini-2.5-flash-001 i get error saying

GenkitError: INVALID_ARGUMENT: Model version is required for context caching, supported only in gemini-1.5-flash-001,gemini-1.5-pro-001 models.
    at validateContextCacheRequest (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/googleai/src/context-caching/utils.ts:181:11)
    at handleCacheIfNeeded (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/googleai/src/context-caching/index.ts:113:6)
    at <anonymous> (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/googleai/src/gemini.ts:1308:15)
    at <anonymous> (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/ai/src/model.ts:195:14)
    at <anonymous> (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/core/src/action.ts:531:62)
    at AsyncLocalStorage.run (node:async_hooks:346:14)
    at AsyncStore.run (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/core/src/registry.ts:474:27)
    at runInActionRuntimeContext (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/core/src/action.ts:632:30)
    at <anonymous> (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/core/src/action.ts:531:20)
    at async <anonymous> (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/core/src/action.ts:404:30) {
  source: undefined,
  status: 'INVALID_ARGUMENT',
  detail: undefined,
  code: 400,
  originalMessage: 'Model version is required for context caching, supported only in gemini-1.5-flash-001,gemini-1.5-pro-001 models.',
  traceId: '11a2417965c9d515269830a50301d080',
  ignoreFailedSpan: true
}

the doc says gemini-2.5-flash and gemini-1.5-pro is supported here https://genkit.dev/docs/plugins/google-genai/#how-to-use-context-caching

Expected behavior
Context caching should work with gemini-2.5-flash

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingtooling

    Type

    No type

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions