Maniac: Continually optimizing models from your LLM telemetry and evals.

Maniac is an enterprise AI platform that makes it easy to replace existing LLM API calls with fine-tuned, task-specific models. Drop in Maniac with one line of code to:

• Capture and structure production LLM traffic

• Automatically fine-tune and evaluate Small Language Models (SLMs) on your tasks

• Replace over-generalized LLM calls with higher performance, lower latency models built for just what you need

• Focus engineering time where it matters most: building and refining high-quality model evaluations—not managing infrastructure, hyperparameters, or bespoke fine-tuning pipelines

All with virtually no changes to your existing codebase.

Getting started

1

Sign up for Maniac

Head over to https://app.maniac.ai/auth/register

2

Create a new Organization

Organizations house multiple projects.

3

Add a Project

All your work — containers, evals, and deployments — live here.

4

Generate an API key

From your project settings

---

Dropping Maniac into your Codebase

For an agentic setup, copy this prompt and give it to your preferred coding agent:

Fetch and follow https://raw.githubusercontent.com/ManiacIncorporated/docs/main/agent-setup.md to instrument this repo for Maniac telemetry.

Install the library

Python

pip install maniac

TypeScript

Initialize client

Python

from maniac import Maniac

maniac = Maniac(api_key="your-maniac-api-key")

TypeScript

Create a container

Containers log inference and automatically build datasets for fine-tuning and evaluation. initial_model sets the model used in that container until a Maniac model is deployed in its place.

Python

container = maniac.containers.create(
  label = "my-container",
  default_system_prompt = "You are a helpful math tutor."
)

TypeScript

Log Completions

Now that you've made a container, let's add some data to it.

Python

response = maniac.chat.completions.register(
    container = "my-container",
    items = [{
        "input": {
            "messages": [{"role": "user", "content": "Hello!"}]
        },
        "output": {
            "choices": [{
                "message": [{"role": "assistant", "content": "Hello! How can I help?"}]
            }]
        }
    }]
)

TypeScript

---

Optimizing your model

The inference logs in your container now serve as training data for a new SLM—fully yours, lower latency, most cost effective, and optimized specifically for your task.

1

Create an Eval

Evaluations define the optimization target. They can be implemented as arbitrary code or defined using judge prompts.

From the Evals tab inside a container, Add Eval.

Judge Eval

Code Eval

2

Optimization happens automatically

Once your telemetry hooks and evals are in place, Maniac automatically optimizes a model for your task — no manual configuration required.

---

Deploy.

Optimized models can be be deployed into a container from the Models tab. Once deployed, you can chat with your generated models, and inference requests are now routed through the Maniac model instead of the initial_model.

Need help?

📧 Email us at [email protected]

We'll get back to you within a day.