On-device generative AI with Gemini Nano (Part 2)
In this demo, we want to enhance our previous On-device generative AI demo with 2 UX improvements.
- Add a loading icon, to let the user know we are working on getting the result.
- Use the streaming API to render the result via a text stream instead of waiting for the full response to be ready.
First, we change our code from the prompt() API to the promptStreaming() API.
const stream = session.promptStreaming(prompt);
for await (const chunk of stream) {
this.outputTarget.textContent = chunk;
}
Next, we reset the output DOM with a loading icon every time the user triggers a generate.
const stream = session.promptStreaming(prompt);
this.outputTarget.classList.remove("hidden");
this.outputTarget.innerHTML = '<i class="fa-solid fa-pencil fa-fade"></i>';
for await (const chunk of stream) {
this.outputTarget.textContent = chunk;
}
The outcome:
Since this is a case of summarizing technical articles, we want the API to be "less creative" and "more precise". We can further improve our code to set the temperature to a lower value, e.g. 0.1
async createAiSession() {
const defaults = await window.ai.defaultTextSessionOptions();
return await window.ai.createTextSession(
{
temperature: 0.1,
topK: defaults.topK
}
);
}
Now we have a simple working AI feature.
AI Summary
gpt-4o-2024-05-13
2024-07-14 00:37:54
This blog post details enhancing an on-device generative AI demo by adding a loading icon and using a streaming API for incremental results. It includes code snippets showing the modifications and highlights setting a lower temperature value for more precise responses.
Chrome On-device AI
2024-12-06 18:20:49
Share Article