# Welcome

**predictor.sh** is a CLI that turns local AI models into secure, internet-accessible API endpoints in seconds.

## What is predictor.sh?

With predictor.sh, you can expose local ML inference to the internet with a single command. Get a stable HTTPS URL, authentication, and observability without any cloud deployment complexity.

```bash
# Local model
predictor up ./llama-7b.gguf

# Or a HuggingFace model
predictor up TheBloke/Llama-2-7B-GGUF

# Your model is now live at https://abc123.predictor.sh
```

## Key Features

| Feature               | Description                                         |
| --------------------- | --------------------------------------------------- |
| **One Command**       | Go from local model to public API in seconds        |
| **Stable URLs**       | Your endpoint URL survives disconnects and restarts |
| **OpenAI Compatible** | Drop-in replacement for OpenAI APIs                 |
| **Multi-Modal**       | Text, speech-to-text (Whisper), and text-to-speech  |
| **GPU Accelerated**   | Native Metal (macOS) and CUDA (NVIDIA) support      |
| **Built-in Auth**     | Bearer token authentication out of the box          |

## Supported Modalities

| Modality           | Formats                 | APIs                    |
| ------------------ | ----------------------- | ----------------------- |
| **Text (LLM)**     | GGUF, SafeTensors, ONNX | OpenAI-compatible       |
| **Speech-to-Text** | Whisper (16 variants)   | OpenAI Whisper API      |
| **Text-to-Speech** | Parler-TTS, Kokoro      | OpenAI + ElevenLabs API |

## Quick Links

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Installation</strong></td><td>Get predictor.sh installed</td><td><a href="/pages/Jftri57WKr5auJzlXr57">/pages/Jftri57WKr5auJzlXr57</a></td></tr><tr><td><strong>Quickstart</strong></td><td>Your first endpoint in 5 minutes</td><td><a href="/pages/7FvWQMF0kTK7HGhlQfmo">/pages/7FvWQMF0kTK7HGhlQfmo</a></td></tr><tr><td><strong>API Reference</strong></td><td>OpenAI-compatible endpoints</td><td><a href="/pages/PZh3TlSLLSfnQkin94KX">/pages/PZh3TlSLLSfnQkin94KX</a></td></tr></tbody></table>


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.predictor.sh/readme.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.