-
Notifications
You must be signed in to change notification settings - Fork 344
Open
Labels
Description
Describe the bug
Context Caching does not work with any Gemini Model gemini-1.5-flash-001
and gemini-2.5-flash-001
To Reproduce
I have below code to read a large text file and then use it as context for a chatbot
import { googleAI } from "@genkit-ai/googleai";
import { genkit } from "genkit";
import fs from "fs/promises";
const ai = genkit({
plugins: [
googleAI({
apiKey: "MY_API_KEY",
}),
],
});
async function main() {
const textContent = await fs.readFile("./myfile.txt", "utf-8");
const { text } = await ai.generate({
messages: [
{
role: "user",
content: [{ text: textContent }], // Include the large text as context
},
{
role: "model",
content: [
{
text: "This analysis is based on the provided text from War and Peace.",
},
],
metadata: {
cache: {
ttlSeconds: 300, // Cache the response to avoid reloading the full text
},
},
},
],
model: googleAI.model("gemini-1.5-flash-001"),
prompt: "Who is Coding with nobody?.",
});
console.log(text);
}
main().catch(console.error);
now i have tried using both gemini-1.5-flash-001
and gemini-2.5-flash-001
when i use gemini-1.5-flash-001
i get error saying
Failed to create cache with key e335a709c48e1e32f5d9c25055b1d955c831d8305c645f8840c99538923f0ee3: Error: [GoogleGenerativeAI Error]: Error fetching from https://generativelanguage.googleapis.com/v1beta/cachedContents: [404 Not Found] models/gemini-1.5-flash-001 is not found for API version v1beta, or is not supported for createCachedContent. Call ListModels to see the list of available models and their supported methods.
GenkitError: INTERNAL: Failed to create cache: Error: [GoogleGenerativeAI Error]: Error fetching from https://generativelanguage.googleapis.com/v1beta/cachedContents: [404 Not Found] models/gemini-1.5-flash-001 is not found for API version v1beta, or is not supported for createCachedContent. Call ListModels to see the list of available models and their supported methods.
at handleContextCache (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/googleai/src/context-caching/index.ts:77:13)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async handleCacheIfNeeded (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/googleai/src/context-caching/index.ts:118:37)
at async <anonymous> (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/googleai/src/gemini.ts:1308:9)
at async <anonymous> (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/core/src/action.ts:531:14)
at async <anonymous> (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/core/src/action.ts:404:30)
at async <anonymous> (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/core/src/tracing/instrumentation.ts:73:16)
at async <anonymous> (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/core/src/tracing/instrumentation.ts:115:24)
at async runInNewSpan (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/core/src/tracing/instrumentation.ts:103:10)
at async newTrace (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/core/src/tracing/instrumentation.ts:62:10) {
source: undefined,
status: 'INTERNAL',
detail: undefined,
code: 500,
originalMessage: 'Failed to create cache: Error: [GoogleGenerativeAI Error]: Error fetching from https://generativelanguage.googleapis.com/v1beta/cachedContents: [404 Not Found] models/gemini-1.5-flash-001 is not found for API version v1beta, or is not supported for createCachedContent. Call ListModels to see the list of available models and their supported methods.',
traceId: 'cc38b24c0e7ca3c86f6c335ad391db65',
ignoreFailedSpan: true
}
when i use gemini-2.5-flash-001
i get error saying
GenkitError: INVALID_ARGUMENT: Model version is required for context caching, supported only in gemini-1.5-flash-001,gemini-1.5-pro-001 models.
at validateContextCacheRequest (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/googleai/src/context-caching/utils.ts:181:11)
at handleCacheIfNeeded (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/googleai/src/context-caching/index.ts:113:6)
at <anonymous> (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/googleai/src/gemini.ts:1308:15)
at <anonymous> (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/ai/src/model.ts:195:14)
at <anonymous> (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/core/src/action.ts:531:62)
at AsyncLocalStorage.run (node:async_hooks:346:14)
at AsyncStore.run (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/core/src/registry.ts:474:27)
at runInActionRuntimeContext (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/core/src/action.ts:632:30)
at <anonymous> (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/core/src/action.ts:531:20)
at async <anonymous> (/Users/transformhub/Downloads/hub/samples/gen/second/node_modules/@genkit-ai/core/src/action.ts:404:30) {
source: undefined,
status: 'INVALID_ARGUMENT',
detail: undefined,
code: 400,
originalMessage: 'Model version is required for context caching, supported only in gemini-1.5-flash-001,gemini-1.5-pro-001 models.',
traceId: '11a2417965c9d515269830a50301d080',
ignoreFailedSpan: true
}
the doc says gemini-2.5-flash and gemini-1.5-pro is supported here https://genkit.dev/docs/plugins/google-genai/#how-to-use-context-caching
Expected behavior
Context caching should work with gemini-2.5-flash
Metadata
Metadata
Assignees
Labels
Type
Projects
Status
No status