Vercel AI SDK Provider
Use WebLLM with the Vercel AI SDK for streaming, tool calling, and more
Installation
Basic Usage
Import the webllm provider and use it with any Vercel AI SDK function:
WebLLM automatically selects the best available model based on user configuration.
Task-Based Routing
Specify a task type and hints to guide model selection:
speed: fastest, fast, balanced, quality
quality: draft, standard, high, best
costPriority: free, budget, balanced, premium
capabilities: multilingual, codeGeneration, reasoning, longContext, math
Streaming
Stream responses in real-time using streamText:
Tool Calling
Define tools that the AI can use. Tools execute client-side in the browser:
Client-side tool execution means no server round-trips - AI can directly update the UI, play sounds, or manipulate the DOM.
React Example
Use WebLLM in a React component with the useChat hook:
How Model Selection Works
Unlike traditional AI SDK providers where you specify a model like openai('gpt-4'), WebLLM uses intelligent routing. When you call webllm():
Request sent to WebLLM extension
The extension receives your task type and hints
Router evaluates options
Scores available models by 16 criteria (speed, quality, cost, capabilities)
Best model executes request
Could be Claude, GPT-4, a local model, or any configured provider
This means users control which AI they use, not your application. Your code works with any model.
API Reference
Creates a WebLLM language model for use with Vercel AI SDK functions.
Parameters
task?: 'general' | 'coding' | 'creative' | 'summarization' | 'qa' | 'extraction' | 'translation'
hints?: { speed?, quality?, costPriority?, capabilities?, model?, provider? }
model?: string — Direct model specification (optional)
provider?: string — Direct provider specification (optional)
Returns
A LanguageModel compatible with all Vercel AI SDK functions.
Next Steps
Use the navigator.llm API directly
View Guide →Learn how WebLLM selects models
View Routing Docs →Configure AI providers
View Provider Docs →Understand how WebLLM works
View Architecture →