Gateway System
An open protocol for proxying LLM requests—self-host or use WebLLM's hosted gateway
Open Gateway Protocol
The gateway protocol is open and anyone can run a gateway. WebLLM Gateway is our hosted implementation, but you can self-host for full control over your data and infrastructure.
Gateways are optional—most users connect directly to providers via the browser extension or their own API keys. Gateways are useful when you want to provide AI access to website visitors without requiring them to configure anything.
Dev configures gateway with secret key + API keys
Dev generates access token locked to origin domain
User app sends requests with token for inference
Token-Origin Binding
Each access token is cryptographically bound to specific origins. When a request arrives:
wlm-abc123.eyJ...Origin header against token's domains[]Allowed
Token domains: ["myapp.com", "*.myapp.com"]Requests from myapp.com or staging.myapp.com
Rejected
Origin: evil-site.comToken stolen and used from different domain
Security Parameters
| Parameter | Options | Description |
|---|---|---|
| quota.type | requests | tokens | Count API requests or LLM tokens consumed |
| quota.limit | number | Maximum allowed per period (e.g., 1000) |
| quota.period | hour | day | month | lifetime | When quota resets |
| Parameter | Example | Description |
|---|---|---|
| domains | ["myapp.com"] | Exact domain match |
| domains | ["*.myapp.com"] | Wildcard: any subdomain |
| domains | [] (empty) | Allow all origins (not recommended) |
| Parameter | Type | Description |
|---|---|---|
| exp | Unix timestamp (ms) | Token expires after this time |
| expiresIn | milliseconds | Convenience: expires N ms from creation |
Usage Example
Self-Hosted Gateway
Run your own gateway server for full control over data and infrastructure.
Clone the repository
git clone https://github.com/webllm-org/webllmConfigure your server
Set up API keys and gateway settings in your environment
Deploy and register
Add your gateway URL to the available gateway services
Self-hosted benefits:
- - Full data sovereignty - requests never leave your infrastructure
- - Custom rate limiting and logging
- - Integration with internal auth systems
- - No dependency on external gateway services
Learn More
Understanding Public, Token-Gated, and API-Key access modes
Learn about access modesConnect gateways together for distributed inference
Explore federationDeploy your own gateway for full control
View deployment guideGet Started
Use the console to create and configure your gateway
Open ConsoleFull documentation for gateway token API
View API Docs