Welcome
predictor.sh is a CLI that turns local AI models into secure, internet-accessible API endpoints in seconds.
What is predictor.sh?
With predictor.sh, you can expose local ML inference to the internet with a single command. Get a stable HTTPS URL, authentication, and observability without any cloud deployment complexity.
# Local model
predictor up ./llama-7b.gguf
# Or a HuggingFace model
predictor up TheBloke/Llama-2-7B-GGUF
# Your model is now live at https://abc123.predictor.shKey Features
One Command
Go from local model to public API in seconds
Stable URLs
Your endpoint URL survives disconnects and restarts
OpenAI Compatible
Drop-in replacement for OpenAI APIs
Multi-Modal
Text, speech-to-text (Whisper), and text-to-speech
GPU Accelerated
Native Metal (macOS) and CUDA (NVIDIA) support
Built-in Auth
Bearer token authentication out of the box
Supported Modalities
Text (LLM)
GGUF, SafeTensors, ONNX
OpenAI-compatible
Speech-to-Text
Whisper (16 variants)
OpenAI Whisper API
Text-to-Speech
Parler-TTS, Kokoro
OpenAI + ElevenLabs API
Quick Links
Last updated