This article is an update for the previous articles below:

https://calvin.my/posts/on-device-generative-ai-with-gemini-nano

https://calvin.my/posts/on-device-generative-ai-with-gemini-nano-part-2

https://calvin.my/posts/api-updates-on-chrome-on-device-ai

Updates in November 2024

1. Package Name "ai.assistant" is changed to "ai.languageModel". This change avoids confusion with Google Assistant.
2. The API now supports more than 1024 input tokens.

Updated sample code

async askAi(prompt) {
        const {available, defaultTopK } = await ai.languageModel.capabilities();

        if (available !== "no") {
            const systemPrompt = "You are a helpful assistant.";
            const session = await ai.languageModel.create({
                systemPrompt: systemPrompt,
                temperature: 0,
                topK: defaultTopK
            });

            const stream = session.promptStreaming(prompt);
            for await (const chunk of stream) {
                this.outputTarget.textContent = chunk;
            }
            session.destroy();
        } else {
            this.outputTarget.textContent = "Error! On-device AI is not available on your browser."
        }
}