API updates on Chrome on-device AI

This article is an update for the previous articles below:

https://calvin.my/posts/on-device-generative-ai-with-gemini-nano

https://calvin.my/posts/on-device-generative-ai-with-gemini-nano-part-2

A few months after its initial release, the Chrome on-device AI has some breaking changes on the API.

1) Package

# Previous
window.ai

# New
ai.assistant

2) Checking availability of the API.

const {available, defaultTopK } = await ai.assistant.capabilities();

if (available !== "no") {
  // available
}

3) New create API

const session = await ai.assistant.create({
    temperature: 0,
    topK: defaultTopK
});

4) Sending System Prompt is now possible.

const systemPrompt = "You are a helpful assistant.";
const session = await ai.assistant.create({
    temperature: 0,
    topK: defaultTopK,
    systemPrompt: systemPrompt
});

The streaming API and streaming output remain unchanged.