API updates on Chrome on-device AI
This article is an update for the previous articles below:
https://calvin.my/posts/on-device-generative-ai-with-gemini-nano
https://calvin.my/posts/on-device-generative-ai-with-gemini-nano-part-2
A few months after its initial release, the Chrome on-device AI has some breaking changes on the API.
1) Package
# Previous
window.ai
# New
ai.assistant
2) Checking availability of the API.
const {available, defaultTopK } = await ai.assistant.capabilities();
if (available !== "no") {
// available
}
3) New create API
const session = await ai.assistant.create({
temperature: 0,
topK: defaultTopK
});
4) Sending System Prompt is now possible.
const systemPrompt = "You are a helpful assistant.";
const session = await ai.assistant.create({
temperature: 0,
topK: defaultTopK,
systemPrompt: systemPrompt
});
The streaming API and streaming output remain unchanged.
AI Summary
gpt-4o-2024-05-13
2024-10-01 16:25:38
The blog post details recent updates to the Chrome on-device AI API, including changes to the package structure, how to check API availability, a new method for creating sessions, and the ability to send system prompts. The streaming API and output remain unchanged.
Chrome On-device AI
2024-10-04 23:04:05
Share Article