This article is an update for the previous articles below:

  1. https://calvin.my/posts/on-device-generative-ai-with-gemini-nano
  2. https://calvin.my/posts/on-device-generative-ai-with-gemini-nano-part-2
  3. https://calvin.my/posts/api-updates-on-chrome-on-device-ai
  4. https://calvin.my/posts/api-updates-on-chrome-on-device-ai-nov-2024

Updates in April 2025

  1. From Chrome Canary 136+, the package name is renamed from self.ai.* to self.*

    await LanguageModel.availability();
await Summarizer.availability();
await Writer.availability();
await Rewriter.availability();
await LanguageDetector.availability();
await Translator.availability();

await LanguageModel.create();
await Summarizer.create();
await Writer.create();
await Rewriter.create();
await LanguageDetector.create();
await Translator.create();

  2. Streaming output changes. Previously, the chunk included everything from index 0 to the latest tokens. The updated behavior returns only the new tokens, which matches the behavior of the mainstream Generative AI streaming API.

    // New
for await (const chunk of stream) {
    this.outputTarget.textContent += chunk;
}

// Old
for await (const chunk of stream) {
    this.outputTarget.textContent = chunk;
}

  3. The optimization-guide-on-device-model flag in Google Chrome is no longer a necessary step in the setup. The prompt-api-for-gemini-nano is needed.

    chrome://flags/#prompt-api-for-gemini-nano

  4. Updated sample code:

    async askAi(prompt) {
        const availability = await LanguageModel.availability();

        if (availability === "available") {
            const capabilities = await LanguageModel.params();
            const systemPrompt = "You are a helpful assistant.";
            const session = await ai.languageModel.create({
                systemPrompt: systemPrompt,
                temperature: 0,
                topK: capabilities.defaultTopK
            });

            const stream = session.promptStreaming(prompt);
            for await (const chunk of stream) {
                this.outputTarget.textContent += chunk;
            }
            session.destroy();
        } else {
            this.outputTarget.textContent = "Error! On-device AI is not available on your browser."
        }
}