API updates on Chrome on-device AI

This article is an update for the previous articles below:

1. https://calvin.my/posts/on-device-generative-ai-with-gemini-nano
2. https://calvin.my/posts/on-device-generative-ai-with-gemini-nano-part-2

A few months after its initial release, the Chrome on-device AI has some breaking changes on the API.

1. Package

   # Previous
   window.ai
   
   # New
   ai.assistant
   

2. Checking availability of the API.

   const {available, defaultTopK } = await ai.assistant.capabilities();
   
   if (available !== "no") {
     // available
   }
   

3. New create API

   const session = await ai.assistant.create({
       temperature: 0,
       topK: defaultTopK
   });
   

4. Sending System Prompt is now possible.

   const systemPrompt = "You are a helpful assistant.";
   const session = await ai.assistant.create({
       temperature: 0,
       topK: defaultTopK,
       systemPrompt: systemPrompt
   });
   

The streaming API and streaming output remain unchanged.