API updates on Chrome on-device AI

This article is an update for the previous articles below:

https://calvin.my/posts/on-device-generative-ai-with-gemini-nano

https://calvin.my/posts/on-device-generative-ai-with-gemini-nano-part-2


A few months after its initial release, the Chrome on-device AI has some breaking changes on the API.

1) Package

# Previous
window.ai

# New
ai.assistant

2) Checking availability of the API.

const {available, defaultTopK } = await ai.assistant.capabilities();

if (available !== "no") {
  // available
}

3) New create API

const session = await ai.assistant.create({
    temperature: 0,
    topK: defaultTopK
});

4) Sending System Prompt is now possible.

const systemPrompt = "You are a helpful assistant.";
const session = await ai.assistant.create({
    temperature: 0,
    topK: defaultTopK,
    systemPrompt: systemPrompt
});

The streaming API and streaming output remain unchanged.


AI Summary
gpt-4o-2024-05-13 2024-10-01 16:25:38
The blog post details recent updates to the Chrome on-device AI API, including changes to the package structure, how to check API availability, a new method for creating sessions, and the ability to send system prompts. The streaming API and output remain unchanged.
Chrome On-device AI 2024-10-04 23:04:05

Share Article