# Create Chat Completion
Source: https://docs.subconscious.dev/api-reference/chat-completions
api-reference/openapi.json POST /chat/completions
Generates a model response for the given conversation. This endpoint is fully compatible with the OpenAI Chat Completions API.
# Create Message
Source: https://docs.subconscious.dev/api-reference/messages
api-reference/openapi.json POST /messages
Generates a model response for the given conversation. This endpoint is fully compatible with the Anthropic Messages API, so you can use the Anthropic SDK by pointing its base URL at Subconscious.
# Streaming
Source: https://docs.subconscious.dev/features/streaming
Stream responses token by token in real time
Streaming delivers tokens as they're generated, which enables responsive UIs and real-time output. Subconscious streams using whichever wire format you call: the OpenAI format emits Server-Sent Events (SSE) with `ChatCompletionChunk` objects, and the Anthropic Messages format emits the Anthropic event protocol (`message_start`, `content_block_delta`, `message_stop`).
## Basic Streaming
Set `stream=True` (or use `client.messages.stream(...)` with the Anthropic SDK) to receive a stream of chunks instead of waiting for the full response:
```python Python (OpenAI) theme={null}
from openai import OpenAI
client = OpenAI(
api_key="your-api-key",
base_url="https://api.subconscious.dev/v1",
)
stream = client.chat.completions.create(
model="subconscious/tim-qwen3.6-27b",
messages=[{"role": "user", "content": "Explain how neural networks learn."}],
stream=True,
)
for chunk in stream:
content = chunk.choices[0].delta.content
if content:
print(content, end="", flush=True)
print()
```
```typescript Node.js (OpenAI) theme={null}
import OpenAI from "openai";
const client = new OpenAI({
apiKey: "your-api-key",
baseURL: "https://api.subconscious.dev/v1",
});
const stream = await client.chat.completions.create({
model: "subconscious/tim-qwen3.6-27b",
messages: [{ role: "user", content: "Explain how neural networks learn." }],
stream: true,
});
for await (const chunk of stream) {
const content = chunk.choices[0]?.delta?.content;
if (content) process.stdout.write(content);
}
console.log();
```
```python Python (Anthropic) theme={null}
from anthropic import Anthropic
client = Anthropic(
auth_token="your-api-key",
base_url="https://api.subconscious.dev",
)
with client.messages.stream(
model="subconscious/tim-qwen3.6-27b",
max_tokens=1024,
messages=[{"role": "user", "content": "Explain how neural networks learn."}],
) as stream:
for text in stream.text_stream:
print(text, end="", flush=True)
print()
```
```typescript Node.js (Anthropic) theme={null}
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic({
authToken: "your-api-key",
baseURL: "https://api.subconscious.dev",
});
const stream = client.messages.stream({
model: "subconscious/tim-qwen3.6-27b",
max_tokens: 1024,
messages: [{ role: "user", content: "Explain how neural networks learn." }],
});
for await (const event of stream) {
if (event.type === "content_block_delta" && event.delta.type === "text_delta") {
process.stdout.write(event.delta.text);
}
}
console.log();
```
```bash cURL (Chat Completions) theme={null}
curl https://api.subconscious.dev/v1/chat/completions \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "subconscious/tim-qwen3.6-27b",
"messages": [{"role": "user", "content": "Explain how neural networks learn."}],
"stream": true
}'
```
```bash cURL (Messages) theme={null}
curl https://api.subconscious.dev/v1/messages \
-H "x-api-key: YOUR_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-H "content-type: application/json" \
-d '{
"model": "subconscious/tim-qwen3.6-27b",
"max_tokens": 1024,
"messages": [{"role": "user", "content": "Explain how neural networks learn."}],
"stream": true
}'
```
## SSE Format
### Chat Completions
Each event in the stream is a `data:` line containing a JSON `ChatCompletionChunk` object:
```
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","model":"subconscious/tim-qwen3.6-27b","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","model":"subconscious/tim-qwen3.6-27b","choices":[{"index":0,"delta":{"content":" world"},"finish_reason":null}]}
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","model":"subconscious/tim-qwen3.6-27b","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: [DONE]
```
The final chunk has `finish_reason: "stop"` and is followed by `data: [DONE]`.
### Messages
The Messages endpoint emits the Anthropic event protocol. Each SSE message has an `event:` type and a `data:` JSON payload, progressing through `message_start`, one or more content blocks (`content_block_start` → `content_block_delta` → `content_block_stop`), then `message_delta` and `message_stop`:
```
event: message_start
data: {"type":"message_start","message":{"id":"msg_abc123","type":"message","role":"assistant","content":[],"model":"subconscious/tim-qwen3.6-27b","stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":12,"output_tokens":0,"cache_read_input_tokens":0}}}
event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"Hello"}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" world"}}
event: content_block_stop
data: {"type":"content_block_stop","index":0}
event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn","stop_sequence":null},"usage":{"output_tokens":18}}
event: message_stop
data: {"type":"message_stop"}
```
Unlike the Chat Completions format, there is no `data: [DONE]` sentinel — the stream ends with `message_stop`.
## Error Handling
Errors during streaming are delivered as SSE events. Both SDKs raise exceptions automatically:
```python Python (OpenAI) theme={null}
from openai import APIError
try:
stream = client.chat.completions.create(
model="subconscious/tim-qwen3.6-27b",
messages=[{"role": "user", "content": "Hello"}],
stream=True,
)
for chunk in stream:
content = chunk.choices[0].delta.content
if content:
print(content, end="", flush=True)
except APIError as e:
print(f"API error: {e.status_code} - {e.message}")
```
```typescript Node.js (OpenAI) theme={null}
import OpenAI from "openai";
try {
const stream = await client.chat.completions.create({
model: "subconscious/tim-qwen3.6-27b",
messages: [{ role: "user", content: "Hello" }],
stream: true,
});
for await (const chunk of stream) {
const content = chunk.choices[0]?.delta?.content;
if (content) process.stdout.write(content);
}
} catch (error) {
if (error instanceof OpenAI.APIError) {
console.error(`API error: ${error.status} - ${error.message}`);
}
}
```
```python Python (Anthropic) theme={null}
from anthropic import APIError
try:
with client.messages.stream(
model="subconscious/tim-qwen3.6-27b",
max_tokens=1024,
messages=[{"role": "user", "content": "Hello"}],
) as stream:
for text in stream.text_stream:
print(text, end="", flush=True)
except APIError as e:
print(f"API error: {e.status_code} - {e.message}")
```
```typescript Node.js (Anthropic) theme={null}
import Anthropic from "@anthropic-ai/sdk";
try {
const stream = client.messages.stream({
model: "subconscious/tim-qwen3.6-27b",
max_tokens: 1024,
messages: [{ role: "user", content: "Hello" }],
});
for await (const event of stream) {
if (event.type === "content_block_delta" && event.delta.type === "text_delta") {
process.stdout.write(event.delta.text);
}
}
} catch (error) {
if (error instanceof Anthropic.APIError) {
console.error(`API error: ${error.status} - ${error.message}`);
}
}
```
## Usage Statistics
For the Chat Completions format, include `stream_options` to receive token usage in the final chunk. For the Messages format, usage is built in: `input_tokens` arrives on `message_start` and `output_tokens` on `message_delta`.
```python Python (OpenAI) theme={null}
stream = client.chat.completions.create(
model="subconscious/tim-qwen3.6-27b",
messages=[{"role": "user", "content": "Hello"}],
stream=True,
stream_options={"include_usage": True},
)
for chunk in stream:
if chunk.usage:
print(f"Input: {chunk.usage.prompt_tokens}, Output: {chunk.usage.completion_tokens}")
elif chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)
```
```python Python (Anthropic) theme={null}
with client.messages.stream(
model="subconscious/tim-qwen3.6-27b",
max_tokens=1024,
messages=[{"role": "user", "content": "Hello"}],
) as stream:
for text in stream.text_stream:
print(text, end="", flush=True)
final = stream.get_final_message()
print(f"\nInput: {final.usage.input_tokens}, Output: {final.usage.output_tokens}")
```
For Chat Completions, usage data is included in the final chunk of the stream.
# Structured Output
Source: https://docs.subconscious.dev/features/structured-output
Get typed JSON responses with schema validation
Structured output lets you define a JSON schema for the model's response, ensuring you get
consistently typed data back. The **OpenAI** format uses the standard `response_format`
parameter. The **Anthropic** Messages format has no `response_format`; instead you achieve the
same result with **forced tool use** — define a tool whose `input_schema` is your target
schema, require it via `tool_choice`, and read the typed `tool_use` block from the response.
## JSON Schema
Pass a JSON schema via `response_format` (OpenAI) or a forced tool (Anthropic) to constrain the
model's output:
```python Python (OpenAI) theme={null}
from openai import OpenAI
import json
client = OpenAI(
api_key="your-api-key",
base_url="https://api.subconscious.dev/v1",
)
response = client.chat.completions.create(
model="subconscious/tim-qwen3.6-27b",
messages=[{"role": "user", "content": "Extract the key facts from: 'Tesla reported $25.5B in Q3 2024 revenue, up 8% year-over-year.'"}],
response_format={
"type": "json_schema",
"json_schema": {
"name": "financial_extract",
"schema": {
"type": "object",
"properties": {
"company": {"type": "string"},
"revenue": {"type": "string"},
"period": {"type": "string"},
"growth": {"type": "string"},
},
"required": ["company", "revenue", "period", "growth"],
},
},
},
)
result = json.loads(response.choices[0].message.content)
print(result)
# {"company": "Tesla", "revenue": "$25.5B", "period": "Q3 2024", "growth": "8% YoY"}
```
```typescript Node.js (OpenAI) theme={null}
import OpenAI from "openai";
const client = new OpenAI({
apiKey: "your-api-key",
baseURL: "https://api.subconscious.dev/v1",
});
const response = await client.chat.completions.create({
model: "subconscious/tim-qwen3.6-27b",
messages: [
{
role: "user",
content:
"Extract the key facts from: 'Tesla reported $25.5B in Q3 2024 revenue, up 8% year-over-year.'",
},
],
response_format: {
type: "json_schema",
json_schema: {
name: "financial_extract",
schema: {
type: "object",
properties: {
company: { type: "string" },
revenue: { type: "string" },
period: { type: "string" },
growth: { type: "string" },
},
required: ["company", "revenue", "period", "growth"],
},
},
},
});
const result = JSON.parse(response.choices[0].message.content!);
console.log(result);
```
```python Python (Anthropic) theme={null}
from anthropic import Anthropic
client = Anthropic(
auth_token="your-api-key",
base_url="https://api.subconscious.dev",
)
schema = {
"type": "object",
"properties": {
"company": {"type": "string"},
"revenue": {"type": "string"},
"period": {"type": "string"},
"growth": {"type": "string"},
},
"required": ["company", "revenue", "period", "growth"],
}
message = client.messages.create(
model="subconscious/tim-qwen3.6-27b",
max_tokens=1024,
tools=[{
"name": "financial_extract",
"description": "Record the extracted financial facts.",
"input_schema": schema,
}],
tool_choice={"type": "tool", "name": "financial_extract"},
messages=[{"role": "user", "content": "Extract the key facts from: 'Tesla reported $25.5B in Q3 2024 revenue, up 8% year-over-year.'"}],
)
result = next(b.input for b in message.content if b.type == "tool_use")
print(result)
# {"company": "Tesla", "revenue": "$25.5B", "period": "Q3 2024", "growth": "8% YoY"}
```
```typescript Node.js (Anthropic) theme={null}
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic({
authToken: "your-api-key",
baseURL: "https://api.subconscious.dev",
});
const message = await client.messages.create({
model: "subconscious/tim-qwen3.6-27b",
max_tokens: 1024,
tools: [
{
name: "financial_extract",
description: "Record the extracted financial facts.",
input_schema: {
type: "object",
properties: {
company: { type: "string" },
revenue: { type: "string" },
period: { type: "string" },
growth: { type: "string" },
},
required: ["company", "revenue", "period", "growth"],
},
},
],
tool_choice: { type: "tool", name: "financial_extract" },
messages: [
{
role: "user",
content:
"Extract the key facts from: 'Tesla reported $25.5B in Q3 2024 revenue, up 8% year-over-year.'",
},
],
});
const block = message.content.find((b) => b.type === "tool_use");
console.log(block?.input);
```
## With Pydantic (Python)
Use Pydantic models to define your schema and parse the response. The same
`model_json_schema()` output works as the OpenAI `json_schema` or as an Anthropic tool's
`input_schema`:
```python Python (OpenAI) theme={null}
from openai import OpenAI
from pydantic import BaseModel
client = OpenAI(
api_key="your-api-key",
base_url="https://api.subconscious.dev/v1",
)
class SentimentAnalysis(BaseModel):
sentiment: str
confidence: float
keywords: list[str]
response = client.chat.completions.create(
model="subconscious/tim-qwen3.6-27b",
messages=[{"role": "user", "content": "Analyze: 'The new update is fantastic, everything runs so smoothly now!'"}],
response_format={
"type": "json_schema",
"json_schema": {
"name": "sentiment_analysis",
"schema": SentimentAnalysis.model_json_schema(),
},
},
)
result = SentimentAnalysis.model_validate_json(response.choices[0].message.content)
print(result.sentiment) # "positive"
print(result.confidence) # 0.95
print(result.keywords) # ["fantastic", "smoothly"]
```
```python Python (Anthropic) theme={null}
import json
from anthropic import Anthropic
from pydantic import BaseModel
client = Anthropic(
auth_token="your-api-key",
base_url="https://api.subconscious.dev",
)
class SentimentAnalysis(BaseModel):
sentiment: str
confidence: float
keywords: list[str]
message = client.messages.create(
model="subconscious/tim-qwen3.6-27b",
max_tokens=1024,
tools=[{
"name": "sentiment_analysis",
"description": "Record the sentiment analysis result.",
"input_schema": SentimentAnalysis.model_json_schema(),
}],
tool_choice={"type": "tool", "name": "sentiment_analysis"},
messages=[{"role": "user", "content": "Analyze: 'The new update is fantastic, everything runs so smoothly now!'"}],
)
raw = next(b.input for b in message.content if b.type == "tool_use")
result = SentimentAnalysis.model_validate(raw)
print(result.sentiment) # "positive"
print(result.confidence) # 0.95
print(result.keywords) # ["fantastic", "smoothly"]
```
## With Zod (TypeScript)
Use Zod schemas with `zodResponseFormat` (OpenAI), or convert the Zod schema to JSON Schema and
use it as an Anthropic tool's `input_schema`:
```typescript Node.js (OpenAI) theme={null}
import OpenAI from "openai";
import { zodResponseFormat } from "openai/helpers/zod";
import { z } from "zod";
const client = new OpenAI({
apiKey: "your-api-key",
baseURL: "https://api.subconscious.dev/v1",
});
const SentimentAnalysis = z.object({
sentiment: z.enum(["positive", "negative", "neutral"]),
confidence: z.number(),
keywords: z.array(z.string()),
});
const response = await client.chat.completions.create({
model: "subconscious/tim-qwen3.6-27b",
messages: [
{
role: "user",
content:
"Analyze: 'The new update is fantastic, everything runs so smoothly now!'",
},
],
response_format: zodResponseFormat(SentimentAnalysis, "sentiment_analysis"),
});
const result = SentimentAnalysis.parse(
JSON.parse(response.choices[0].message.content!)
);
console.log(result.sentiment); // "positive"
```
```typescript Node.js (Anthropic) theme={null}
import Anthropic from "@anthropic-ai/sdk";
import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";
const client = new Anthropic({
authToken: "your-api-key",
baseURL: "https://api.subconscious.dev",
});
const SentimentAnalysis = z.object({
sentiment: z.enum(["positive", "negative", "neutral"]),
confidence: z.number(),
keywords: z.array(z.string()),
});
const message = await client.messages.create({
model: "subconscious/tim-qwen3.6-27b",
max_tokens: 1024,
tools: [
{
name: "sentiment_analysis",
description: "Record the sentiment analysis result.",
input_schema: zodToJsonSchema(SentimentAnalysis) as Anthropic.Tool.InputSchema,
},
],
tool_choice: { type: "tool", name: "sentiment_analysis" },
messages: [
{
role: "user",
content:
"Analyze: 'The new update is fantastic, everything runs so smoothly now!'",
},
],
});
const block = message.content.find((b) => b.type === "tool_use");
const result = SentimentAnalysis.parse(block?.input);
console.log(result.sentiment); // "positive"
```
## JSON Mode
JSON mode is specific to the OpenAI format. For simpler cases where you just need valid JSON
without a specific schema, use JSON mode. (The Anthropic Messages format has no JSON mode — use
the forced tool-use pattern shown above when you need structured JSON.)
```python Python theme={null}
response = client.chat.completions.create(
model="subconscious/tim-qwen3.6-27b",
messages=[
{"role": "system", "content": "Respond only in JSON."},
{"role": "user", "content": "List three programming languages and their main use cases."},
],
response_format={"type": "json_object"},
)
```
When using JSON mode without a schema, include "respond in JSON" or similar instructions in your prompt. The model needs to know you expect JSON output.
# Subconscious Cache
Source: https://docs.subconscious.dev/features/subconscious-cache
(Advanced) Disable auto compaction and trigger the subconscious cache manually.
Normally, our TIMRUN inference runtime has two features enabled by default to improve agent performance and ability.
1. Auto Compaction -> compact the message list at runtime.
2. Subconscious Cache -> Maintain both the prefix and the suffix around the pruned messages.
This guide will walk you through how to disable auto compaction, so that you can use the subconscious cache explicitly.
## How It Works
To hit the subconscious cache, the cached tokens and new inputs need to satisfy two criteria:
1. The cached chain can be precisely split into three sections `A, B, C`
2. Section `B` is pruned.
3. The new input chain can be precisely split into three sections `A, C, D`, such that `A` and `C` match the prefix `A` and suffix `C` in the cache and `len(C) > threshold`. We usually set `threshold = 8` tokens to avoid matching the suffix of chat templates.
## Manually Triggering Subconscious Cache
Subconscious API enables auto-compaction by default. Under the auto-compaction mode, developers can send any message list to the LLM API and the inference system will detect prunable messages. Message pruning in the auto compaction mode will automatically hit the subconscious cache.
If you want to manually hit subconscious by controlling the context by yourself instead of auto-compaction, simply disable auto compaction in the chat kwards. The `chat_template_kwargs` extension is accepted on both the Completions and Messages endpoints.
```python Python (OpenAI) theme={null}
from openai import OpenAI
client = OpenAI(
api_key="your-api-key",
base_url="https://api.subconscious.dev/v1",
)
response = client.chat.completions.create(
model="subconscious/tim-qwen3.6-27b",
messages=[{"role": "user", "content": "What is 127 * 849 + 3621?"}],
extra_body={
"chat_template_kwargs": {"enable_auto_compaction": False},
},
)
print(response.choices[0].message.content)
```
```typescript Node.js (OpenAI) theme={null}
import OpenAI from "openai";
const client = new OpenAI({
apiKey: "your-api-key",
baseURL: "https://api.subconscious.dev/v1",
});
const response = await client.chat.completions.create({
model: "subconscious/tim-qwen3.6-27b",
messages: [{ role: "user", content: "What is 127 * 849 + 3621?" }],
// @ts-expect-error Subconscious extension
chat_template_kwargs: { enable_auto_compaction: false },
});
console.log(response.choices[0].message.content);
```
```python Python (Anthropic) theme={null}
from anthropic import Anthropic
client = Anthropic(
auth_token="your-api-key",
base_url="https://api.subconscious.dev",
)
message = client.messages.create(
model="subconscious/tim-qwen3.6-27b",
max_tokens=1024,
messages=[{"role": "user", "content": "What is 127 * 849 + 3621?"}],
extra_body={
"chat_template_kwargs": {"enable_auto_compaction": False},
},
)
print(message.content[0].text)
```
```typescript Node.js (Anthropic) theme={null}
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic({
authToken: "your-api-key",
baseURL: "https://api.subconscious.dev",
});
const message = await client.messages.create({
model: "subconscious/tim-qwen3.6-27b",
max_tokens: 1024,
messages: [{ role: "user", content: "What is 127 * 849 + 3621?" }],
// @ts-expect-error Subconscious extension
chat_template_kwargs: { enable_auto_compaction: false },
});
console.log(message.content[0].text);
```
```bash cURL (Chat Completions) theme={null}
curl https://api.subconscious.dev/v1/chat/completions \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "subconscious/tim-qwen3.6-27b",
"messages": [{"role": "user", "content": "What is 127 * 849 + 3621?"}],
"chat_template_kwargs": {"enable_auto_compaction": false}
}'
```
```bash cURL (Messages) theme={null}
curl https://api.subconscious.dev/v1/messages \
-H "x-api-key: YOUR_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-H "content-type: application/json" \
-d '{
"model": "subconscious/tim-qwen3.6-27b",
"max_tokens": 1024,
"messages": [{"role": "user", "content": "What is 127 * 849 + 3621?"}],
"chat_template_kwargs": {"enable_auto_compaction": false}
}'
```
## When to Turn Off Auto Compaction
If you turn off auto compaction, you need to manually construct inputs that can hit the subconscious cache. Just make sure you only prune **one** continuous token sequence from the message list. If there is no context pruning, the new input will simply hit prefix cache. If more than one chunks are pruned, we cannot find suffix tokens satisfying the subconscious rules.
**Use Auto Compaction for:**
* Programming tasks, where assistant-tool-user messages keeps growing in a message list
* Browser automation, where dead end exploration is easily pruned
* Workflow automation, where stale tool calls pile up quickly
* Multi-turn conversation, where rigid context pruning rule cannot handle arbitrary user inputs
**Skip auto compaction for:**
* ReACT multi-modal reasoning: Subconscious cache works perfectly when you only keep latest turns / images in the message list
* Other applications where you need to carefully control context engineering.
# Thinking Mode
Source: https://docs.subconscious.dev/features/thinking
Enable step-by-step reasoning for complex tasks
Thinking mode enables the model to reason step by step before producing its final answer. This provides higher quality outputs for complex tasks like math, logic, code generation, and multi-step analysis.
Closed models usually **hide** their reasoning, returning only a summary or nothing at all. Because Subconscious serves **open models**, the model's reasoning is **completely visible**. You get the full, unaltered chain of thought, giving you total transparency for debugging, auditing, and trust.
## How It Works
When thinking mode is enabled, the model generates internal reasoning tokens (wrapped in `` tags) before the final response. These reasoning tokens help the model work through complex problems but are included in your output token usage.
## Enabling Thinking Mode
Each wire format controls thinking with its own syntax.
With the **OpenAI** format,
pass the Subconscious extension `chat_template_kwargs` with `enable_thinking: true` via the
`extra_body` parameter.
With the **Anthropic** format, use the native `thinking` parameter
(`{"type": "enabled", "budget_tokens": ...}`):
```python Python (OpenAI) theme={null}
from openai import OpenAI
client = OpenAI(
api_key="your-api-key",
base_url="https://api.subconscious.dev/v1",
)
response = client.chat.completions.create(
model="subconscious/tim-qwen3.6-27b",
messages=[{"role": "user", "content": "What is 127 * 849 + 3621?"}],
extra_body={
"chat_template_kwargs": {"enable_thinking": True},
},
)
print(response.choices[0].message.content)
```
```typescript Node.js (OpenAI) theme={null}
import OpenAI from "openai";
const client = new OpenAI({
apiKey: "your-api-key",
baseURL: "https://api.subconscious.dev/v1",
});
const response = await client.chat.completions.create({
model: "subconscious/tim-qwen3.6-27b",
messages: [{ role: "user", content: "What is 127 * 849 + 3621?" }],
// @ts-expect-error Subconscious extension
chat_template_kwargs: { enable_thinking: true },
});
console.log(response.choices[0].message.content);
```
```python Python (Anthropic) theme={null}
from anthropic import Anthropic
client = Anthropic(
auth_token="your-api-key",
base_url="https://api.subconscious.dev",
)
message = client.messages.create(
model="subconscious/tim-qwen3.6-27b",
max_tokens=2048,
thinking={"type": "enabled", "budget_tokens": 2000},
messages=[{"role": "user", "content": "What is 127 * 849 + 3621?"}],
)
for block in message.content:
if block.type == "thinking":
print("[thinking]", block.thinking)
elif block.type == "text":
print(block.text)
```
```typescript Node.js (Anthropic) theme={null}
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic({
authToken: "your-api-key",
baseURL: "https://api.subconscious.dev",
});
const message = await client.messages.create({
model: "subconscious/tim-qwen3.6-27b",
max_tokens: 2048,
thinking: { type: "enabled", budget_tokens: 2000 },
messages: [{ role: "user", content: "What is 127 * 849 + 3621?" }],
});
for (const block of message.content) {
if (block.type === "thinking") console.log("[thinking]", block.thinking);
else if (block.type === "text") console.log(block.text);
}
```
```bash cURL (Chat Completions) theme={null}
curl https://api.subconscious.dev/v1/chat/completions \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "subconscious/tim-qwen3.6-27b",
"messages": [{"role": "user", "content": "What is 127 * 849 + 3621?"}],
"chat_template_kwargs": {"enable_thinking": true}
}'
```
```bash cURL (Messages) theme={null}
curl https://api.subconscious.dev/v1/messages \
-H "x-api-key: YOUR_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-H "content-type: application/json" \
-d '{
"model": "subconscious/tim-qwen3.6-27b",
"max_tokens": 2048,
"thinking": {"type": "enabled", "budget_tokens": 2000},
"messages": [{"role": "user", "content": "What is 127 * 849 + 3621?"}]
}'
```
Both controls enable the same underlying feature. The OpenAI format toggles it with the
`enable_thinking` extension; the Anthropic format uses the native `thinking` config and also
lets you cap reasoning with `budget_tokens`.
## Response Format
With the **OpenAI** format, the model's response includes reasoning wrapped in `` tags
followed by the final answer:
```
Let me calculate this step by step.
127 * 849 = 127 * 800 + 127 * 49
127 * 800 = 101,600
127 * 49 = 6,223
101,600 + 6,223 = 107,823
107,823 + 3,621 = 111,444
The answer is **111,444**.
```
With the **Anthropic** format, the reasoning is returned as a separate `thinking` content block
before the `text` block, rather than inline tags:
```json theme={null}
{
"content": [
{"type": "thinking", "thinking": "Let me calculate this step by step...", "signature": ""},
{"type": "text", "text": "The answer is **111,444**."}
]
}
```
## Streaming with Thinking
Thinking mode works with streaming. The reasoning tokens stream first, followed by the final answer:
```python Python (OpenAI) theme={null}
stream = client.chat.completions.create(
model="subconscious/tim-qwen3.6-27b",
messages=[{"role": "user", "content": "Solve: If 3x + 7 = 22, what is x?"}],
stream=True,
extra_body={
"chat_template_kwargs": {"enable_thinking": True},
},
)
for chunk in stream:
content = chunk.choices[0].delta.content
if content:
print(content, end="", flush=True)
```
```python Python (Anthropic) theme={null}
with client.messages.stream(
model="subconscious/tim-qwen3.6-27b",
max_tokens=2048,
thinking={"type": "enabled", "budget_tokens": 2000},
messages=[{"role": "user", "content": "Solve: If 3x + 7 = 22, what is x?"}],
) as stream:
for event in stream:
if event.type == "content_block_delta":
if event.delta.type == "thinking_delta":
print(event.delta.thinking, end="", flush=True)
elif event.delta.type == "text_delta":
print(event.delta.text, end="", flush=True)
```
## When to Use Thinking Mode
**Use thinking mode for:**
* Math and arithmetic problems
* Logic puzzles and reasoning tasks
* Complex code generation
* Multi-step analysis
* Tasks requiring planning or strategy
**Skip thinking mode for:**
* Simple Q\&A
* Creative writing
* Translation
* Summarization
* Tasks where speed matters more than accuracy
Thinking tokens count toward your output token usage. For simple tasks, leaving thinking mode off will be faster and more cost-effective.
# Overview
Source: https://docs.subconscious.dev/on-prem/api-gateway/overview
The customer-facing control point for coding-agent traffic
The **API Gateway** is the customer-facing entry point for coding-agent traffic. Coding agents call the API Gateway as their customer-hosted endpoint. The gateway authenticates requests, applies access policy and limits, records usage, and routes the request to a GPU. It runs in the customer-controlled environment and gives administrators a single place to manage access, routing, usage, and day-two operations.
It handles things like:
* End-user API requests from coding agents and SDK clients.
* Load balancing.
* OpenAI- and Anthropic-compatible gateway endpoints.
* Model routing to the Inference Runtime or external model endpoints.
* User and API key management.
* Access controls, limits, and usage tracking.
* Runtime observability, readiness, and operational dashboards.
```mermaid theme={null}
flowchart LR
agents["Coding agents"]
subgraph customer["Customer Cloud"]
direction LR
gateway["API Gateway
(auth, limits, model routing)"]
gpus["Customer GPUs"]
gateway --> gpus
end
agents --> gateway
style customer fill:#eef7ff,stroke:#4ea3ff,stroke-width:1px,color:#17324d
```
# Setup
Source: https://docs.subconscious.dev/on-prem/api-gateway/setup
Admin setup for dashboard access, routes, credentials, and pilot readiness
After the Subconscious Inference System is deployed, your Subconscious FDE will walk through API Gateway admin setup with your customer admin.
## Admin setup
Your FDE and customer admin work through this flow together:
1. **Confirm dashboard access**: sign in, verify the customer organization, and confirm the approved URL, DNS, TLS, and network path are working.
2. **Confirm prerequisites**: make sure the Subconscious Inference System is deployed, the Inference Runtime or external model endpoints are reachable, required credentials are available, and first admins and pilot users are identified.
3. **Configure credentials and routes**: add only the provider credentials needed for routes, then create the initial model route to the Inference Runtime or external model endpoint.
4. **Send a test request**: use an admin test key and a small prompt to confirm authentication, model access, routing, response behavior, and usage recording.
5. **Configure pilot users**: create or share the pilot API key pattern, confirm model slugs, and point the first coding agents at the customer API Gateway endpoint.
6. **Confirm day-two ownership**: align on user management, usage metering, logging, monitoring, support escalation, and handoff.
After the initial setup, administrators usually operate a few recurring workflows.
### User management
* Invite administrators and pilot users.
* Reset passwords and manage roles.
* Grant or revoke model access.
* Keep user and team ownership clear during rollout.
### Usage metering
* Create and revoke API keys.
* Configure organization-level and API-key-level limits.
* Review usage by date range, model, key, or organization.
* Adjust limits after pilot feedback or capacity changes.
### Logging
* Review gateway service logs, router logs, route health logs, and audit logs for admin actions.
* Keep sensitive prompts, completions, secrets, and API keys out of shared logs.
* Prepare customer-approved support bundles or diagnostic summaries when needed.
### Monitoring
* Monitor gateway availability, request rate, latency, error rate, and route health.
* Review authentication failures, authorization failures, and rate-limit events.
* Pair gateway metrics with Inference Runtime metrics such as GPU utilization, queue depth, and token throughput.
## Admin handoff checklist
* Customer admins can sign in.
* Admin users and pilot users are configured.
* Initial API keys are created and stored.
* Initial Inference Runtime routes are active and healthy.
* Model access is granted.
* Limits are configured.
* Test request succeeds.
* Coding-agent setup is validated for at least one pilot user.
* Usage is visible.
* Monitoring and logging ownership is confirmed.
* Support and escalation process is understood.
# Configurations
Source: https://docs.subconscious.dev/on-prem/deployments/configurations
Cloud, GPU placement, Helm overrides, secrets, observability, and upgrade choices
Configuration depends on your deployment method, cloud, Kubernetes environment, API Gateway placement, Inference Runtime GPU placement, networking model, observability stack, and security requirements. This page collects the main permutations customers usually decide during onboarding.
## API Gateway and Inference Runtime placement
After the deployment method is selected, the next major configuration choice is where the API Gateway runs and where the Inference Runtime's GPU capacity runs.
The API Gateway usually runs close to your engineering stack, identity systems, networking controls, and observability tools. The Inference Runtime may run on GPUs colocated in the same cloud environment or on GPU capacity provided by a separate environment such as a NeoCloud, specialized inference provider, or local GPU cluster.
This choice drives Helm values for networking, model routes, credentials, timeouts, observability, scaling, and failure handling. The API Gateway owns routing and load balancing; the Inference Runtime owns efficient GPU-backed model execution.
## Placement option 1: API Gateway and Inference Runtime in the same cloud
In this model, the API Gateway and Inference Runtime run in the same customer cloud environment.
Examples (coming soon):
* API Gateway on AWS EKS with Inference Runtime GPU nodes on AWS.
* API Gateway on GCP GKE with Inference Runtime GPU nodes on GCP.
* API Gateway on Azure AKS with Inference Runtime GPU nodes on Azure.
This mode is best when:
* Your cloud already has approved GPU capacity.
* You want simpler private networking.
* You want inference traffic to remain inside one cloud boundary.
* You want standard cloud observability and IAM patterns.
## Placement option 2: API Gateway in your cloud, Inference Runtime elsewhere
In this model, the API Gateway runs in your cloud environment and routes model traffic to Inference Runtime GPU capacity hosted elsewhere.
Examples (coming soon):
* API Gateway on AWS with Inference Runtime capacity on a Baseten.
* API Gateway on GCP with Inference Runtime capacity on Baseten.
* API Gateway on Azure with Inference Runtime capacity on Together AI or another provider.
* API Gateway in customer Kubernetes with Inference Runtime workers in a separate private cluster.
This mode is best when:
* Your main cloud does not have enough GPU capacity for the Inference Runtime.
* You want to use a specialized GPU provider.
* You want to colocate the API Gateway with the rest of your stack while routing inference to external capacity.
* You want to compare GPU providers without moving the API Gateway.
## Placement option 3: API Gateway and Inference Runtime in separate customer-controlled environments
In this model, both the API Gateway and Inference Runtime GPUs are customer-controlled, but they are not colocated.
Examples (coming soon):
* API Gateway in the corporate cloud, Inference Runtime GPUs in a private research cluster.
* API Gateway in Kubernetes, Inference Runtime workers in an on-prem environment.
* API Gateway in one region, Inference Runtime workers in another region.
This mode is best when GPU capacity already exists in a separate customer environment, network isolation requires an API Gateway boundary, or a central endpoint must route to multiple Inference Runtime backends.
## Cloud-specific notes
AWS deployments usually need EKS compatibility, ALB or NLB guidance, IAM role requirements, Secrets Manager integration, CloudWatch logging and metrics, GPU node pool configuration, and PrivateLink or VPC peering options for external GPU providers.
GCP deployments usually need GKE compatibility, internal load balancer guidance, Workload Identity requirements, Secret Manager integration, Cloud Logging and Cloud Monitoring, GPU node pool configuration, and Private Service Connect or VPC connectivity for external GPU providers.
Azure deployments usually need AKS compatibility, internal load balancer guidance, managed identity requirements, Key Vault integration, Azure Monitor and Log Analytics, GPU node pool configuration, and Private Link or VNet peering options for external GPU providers.
NeoCloud and external GPU deployments usually need endpoint format, authentication method, network allowlisting, private connectivity options, TLS requirements, health check path, rate limits, concurrency limits, model naming, route mapping, fallback behavior, and cost or capacity assumptions.
## Secrets and credentials
Secrets are configured during onboarding based on deployment method and customer policy.
Common secrets include:
* API Gateway admin credentials.
* User and API key management secrets.
* Model provider credentials.
* Database credentials.
* Cache credentials.
* Observability export credentials.
* Registry access tokens.
* TLS certificate references.
In assisted self-managed deployments, some deployment secrets may be managed through the customer portal when enabled. In traditional self-hosted deployments, customers may prefer to store secrets entirely in their own secret manager and reference them from Helm values.
## Monitoring configuration
Monitoring configuration depends on the customer's existing observability stack.
Common goals include:
* API Gateway availability.
* Request rate.
* Latency.
* Error rate.
* Model route health.
* GPU utilization.
* Queue depth.
* Token throughput.
* Cost and capacity signals.
* Agent deployment status in assisted self-managed mode.
Subconscious can help map API Gateway and Inference Runtime metrics into the customer's monitoring stack during onboarding.
## Logging configuration
Logging should support operations without exposing customer source code, prompts, completions, secrets, or other sensitive data. We can help configure sending logs to external sources like Datadog.
Common log streams include:
* API Gateway service logs.
* Router logs.
* Deployment logs.
* Agent logs in assisted self-managed mode.
* Model worker logs.
* Audit logs for admin actions.
* Upgrade and rollback logs.
Customers should define log retention, redaction, support bundle approval, and approved sharing channels before production use.
## Upgrade and rollback process
Upgrade flow depends on the deployment method.
In traditional self-hosted deployments:
1. Subconscious publishes a release.
2. The customer reviews release evidence.
3. The customer pulls or mirrors artifacts.
4. The customer applies the Helm upgrade through its own process.
5. The customer validates health and rolls back if needed.
In assisted self-managed deployments:
1. Subconscious publishes a release.
2. The customer reviews release evidence and update policy.
3. The Distr agent applies the update locally when approved or when automatic updates are enabled.
4. The customer and Subconscious monitor deployment health.
5. Rollback is coordinated according to the customer's policy.
Upgrade planning should define maintenance windows, release notes, rollback commands, compatibility policy, migration handling, expected downtime, and emergency patch process.
# Methods
Source: https://docs.subconscious.dev/on-prem/deployments/methods
Choose assisted self-managed or traditional self-hosted deployment
Subconscious supports two primary deployment methods for customer-hosted production deployments, powered by [Distr](https://distr.sh/docs/): [traditional self-hosted](https://distr.sh/docs/use-cases/fully-self-managed/) and [assisted self-managed](https://distr.sh/docs/use-cases/assisted-self-managed/). Both keep the Subconscious Inference System in your environment. The difference is how installation, updates, and operational actions are performed.
In both methods:
* The Subconscious Inference System runs in the customer's cloud or customer-controlled environment.
* The customer controls infrastructure, networking, access, data, monitoring, and production change management.
* Subconscious provides licensed artifacts, deployment guidance, release notes, support, and upgrade coordination.
The deployment method should match your internal platform practices and security requirements.
## Traditional self-hosted
Traditional self-hosted deployment gives your platform team the most direct control over release promotion and Subconscious Inference System operations.
In this mode:
* Subconscious distributes Helm charts, container images, checksums, release notes, and configuration guidance.
* Your team pulls artifacts from the Distr registry or mirrors them into an internal registry.
* Your team reviews the release and applies it with your own deployment tooling.
* Your team owns deployment, upgrades, rollback, monitoring, alerting, and vulnerability scanning.
* Subconscious supports your team through onboarding and ongoing FDE support, but does not run a long-lived deployment agent in your cluster.
This mode is best when:
* Your platform team wants complete control over release promotion.
* You require all changes to pass through internal CI/CD, security scanning, and approval.
* You prefer no persistent vendor-managed deployment process in the cluster.
Customers use the artifact registry and their own deployment process without a long-lived deployment agent. Follow Distr's [fully self-managed deployment](https://distr.sh/docs/use-cases/fully-self-managed/) docs for more info.
### Traditional self-hosted walkthrough
1. Your FDE provisions your Subconscious customer portal.
2. Your team receives access to licensed Subconscious artifacts.
3. Your team generates or receives registry access according to policy.
4. Your team reviews available Helm chart versions, container images, release notes, and vulnerability reports.
5. Your team mirrors artifacts into an internal registry if required.
6. Your team creates a values override file for your environment.
7. Your team deploys with Helm or your internal GitOps process.
8. Your team validates API Gateway readiness, model routes, observability, and coding agent connectivity.
9. Subconscious and your team complete handoff.
## Assisted self-managed
Assisted self-managed deployment keeps the Subconscious Inference System in your environment while allowing a Distr deployment agent to help manage application lifecycle operations.
This maps to Distr's [assisted self-managed deployment](https://distr.sh/docs/use-cases/assisted-self-managed/) pattern: a customer-installed agent applies updates according to customer-defined policies and reports deployment status back to the portal.
In this mode:
* The Subconscious Inference System still runs in your cloud or customer-controlled Kubernetes environment.
* Your team retains control over environment access.
* A Distr Kubernetes agent runs in your cluster with an agreed scope.
* The agent manages Helm deployment operations locally.
* Subconscious can publish updates through Distr.
* Updates can be customer-approved, scheduled, or automatic depending on policy.
* The portal provides visibility into deployment status, versions, health, and logs.
This mode is best when:
* You want faster onboarding with fewer manual Helm steps.
* You want Subconscious and your FDE to help operate upgrades and maintenance.
* You want a more SaaS-like experience.
* You want shared visibility into deployment state, versions, and operational status.
### Assisted self-managed walkthrough
1. Your FDE provisions your Subconscious customer portal.
2. Your team confirms the target Kubernetes cluster, namespace, network policy, and RBAC scope.
3. Your team configures environment variables and secrets in your customer portal (Subconscious never sees them).
4. Your team installs the Distr Kubernetes agent using a customer-approved command or manifest.
5. The agent connects to Distr Hub and appears in the customer portal.
6. Your FDE configures the Subconscious application deployment for your target environment.
7. Your team enters or approves required deployment secrets.
8. Your team reviews the initial values override configuration.
9. The Distr agent deploys the Subconscious Helm chart into your cluster.
10. Your team and Subconscious validate readiness, ingress, routing, model connectivity, and coding agent setup.
11. Future updates follow the customer's selected policy and are managed in the customer portal.
## Choosing between methods
Choose traditional self-hosted when your platform team wants every deployment action to run through internal tooling and approval.
Choose assisted self-managed when you want Subconscious to help manage initial installation, updates, logging visibility, and ongoing maintenance while the Subconscious Inference System remains in your environment.
Both methods can support customer-controlled approval, vulnerability scanning, private registries, restricted egress, and strict change-management processes.
## Related pages
* [Distribution Platform](/on-prem/distribution-platform/overview): How Distr supports artifact delivery, agents, release evidence, and secrets.
* [Configurations](/on-prem/deployments/configurations): Cloud, GPU placement, Helm overrides, secrets, monitoring, logging, upgrades, and rollback.
* [API Gateway setup](/on-prem/api-gateway/setup): API Gateway dashboard configuration after deployment.
# Overview
Source: https://docs.subconscious.dev/on-prem/distribution-platform/overview
Software delivery, versioning, updates, and assisted self-managed deployment
The **Distribution Platform** manages software delivery, versioning, updates, release metadata, and assisted self-managed deployment for the Subconscious Inference System.
It is powered by [Distr](https://distr.sh/docs/), a platform built for delivering Docker, Helm, and OCI artifacts into self-managed, assisted self-managed, bring-your-own-cloud, air-gapped, and edge environments.
## What it is used for
The Distribution Platform delivers the software that runs in the customer's environment. It is separate from the API Gateway and Inference Runtime that serve production inference traffic.
Subconscious uses it to deliver:
* Helm charts and container images.
* Customer-specific deployment instructions.
* Registry access and artifact metadata.
* Deployment status for assisted self-managed deployments.
* Release notes, version metadata, and upgrade guidance.
* Vulnerability reports and release evidence where available.
* Customer-managed deployment secrets when enabled for the deployment.
The API Gateway and Inference Runtime are the components your engineers use during production inference. The Distribution Platform is the mechanism for delivering, updating, and supporting those components.
## Key workflows
During onboarding, your Subconscious FDE provisions a customer portal for your organization. The portal gives your team a place to view deployment instructions, inspect available artifacts, manage deployments, generate access tokens, and review release resources.
Only the admins of your team will have access to the customer portal. We recommend exploring the [Distr customer portal docs](https://distr.sh/docs/platform/customer-portal/) for more information.
### Customer portal
The customer portal is the customer-facing workspace for deployment and distribution workflows. Customers use it to view deployment instructions, inspect artifacts, generate registry access tokens, manage secrets when enabled, and view deployment status in assisted self-managed mode.
For deeper Distr details, see [Customer Portal](https://distr.sh/docs/platform/customer-portal/), [Customer Management](https://distr.sh/docs/platform/customer-management/), and [Role-Based Access Control (RBAC)](https://distr.sh/docs/platform/rbac/).
### Artifact registry
The artifact registry is where licensed Subconscious artifacts are made available to your organization. It can include Docker images, Helm charts, and other OCI artifacts.
Depending on your policy, your team may pull artifacts directly from the Distr registry, mirror them into an internal registry, scan artifacts before promotion, or promote approved artifacts through an internal GitOps process.
For deeper Distr details, see [Customer Portal Artifact Registry](https://distr.sh/docs/platform/customer-portal/registry/) and Distr's [fully self-managed deployment](https://distr.sh/docs/use-cases/fully-self-managed/) pattern.
### Deployment agents
In assisted self-managed deployments, a Distr Kubernetes agent can run in the target environment. The agent manages Helm lifecycle operations locally and reports deployment status back to the portal.
The customer still defines the namespace, RBAC scope, network policy, secrets policy, and approval process for agent installation.
For deeper Distr details, see [Customer Portal Deployments](https://distr.sh/docs/platform/customer-portal/deployments/) and Distr's [assisted self-managed deployment](https://distr.sh/docs/use-cases/assisted-self-managed/) pattern.
### Release evidence and support
Version resources are customer-visible materials attached to a release. They help the customer review, approve, and audit deployment changes.
Resources may include release notes, vulnerability scan reports, SBOMs, image digests, Helm chart versions, checksums, compatibility notes, and rollback instructions.
For deeper Distr details, see [Vulnerability Scanning](https://distr.sh/docs/platform/vulnerability-scanning/), [Kubernetes Compatibility Matrix](https://distr.sh/docs/platform/kubernetes-compatibility-matrix/), [Customer Portal Support Bundles](https://distr.sh/docs/platform/customer-portal/support/), and [Support Bundles](https://distr.sh/docs/platform/support-bundles/).
### Secrets
Secrets are configured based on the deployment method and customer policy. In assisted self-managed mode, deployment secrets may be managed through the customer portal when enabled for the organization. In traditional self-hosted mode, customers may prefer to store secrets entirely in their own secret manager and reference them from Helm values, Kubernetes Secrets, or an external secrets operator.
Common secrets include registry access tokens, API Gateway admin credentials, model provider credentials, database and cache credentials, observability export credentials, and TLS certificate references.
For deeper Distr details, see [Customer Portal Secrets](https://distr.sh/docs/platform/customer-portal/secrets/). API Gateway route credentials and API keys are covered in [API Gateway setup](/on-prem/api-gateway/setup).
For deployment method details, see [Methods](/on-prem/deployments/methods).
# FAQ
Source: https://docs.subconscious.dev/on-prem/faq
Common questions about the Subconscious Inference System
## Why use this over Claude Code or Codex?
Closed hosted tools can send prompts, code, and engineering context outside your trust boundary. The Subconscious Inference System gives your team frontier-level coding-agent intelligence while keeping API Gateway traffic, Inference Runtime execution, and deployment policy inside infrastructure you control.
See [Compliance](/on-prem/trust-center/compliance) for the data and IP boundary.
## Why use Subconscious instead of hosting the models and managing GPUs ourselves?
You can rent GPUs and host open models yourself, but the hard part is serving coding-agent workloads efficiently and reliably. Subconscious helps manage the API Gateway, Inference Runtime, Distribution Platform, admin surfaces, upgrades, and support.
The Subconscious Inference System is designed to serve teams of engineers with roughly half the GPUs compared with off-the-shelf inference runtimes like vLLM.
See [How it works](/on-prem/how-it-works) and [Configurations](/on-prem/deployments/configurations).
## How does this fit into our security and compliance review?
The customer-hosted deployment is designed to fit into customer-controlled security, compliance, and change-management processes. The API Gateway and Inference Runtime run in your cloud account or controlled environment, and you retain control over deployment, access, monitoring, networking, data, and change management.
See [Compliance](/on-prem/trust-center/compliance).
## Where does the Subconscious Inference System run?
The API Gateway and Inference Runtime run in the customer's cloud or controlled infrastructure. Depending on your configuration, Inference Runtime GPU workers may run in the same cloud, another customer-controlled environment, or a specialized GPU provider behind the API Gateway.
See [How it works](/on-prem/how-it-works) and [Configurations](/on-prem/deployments/configurations).
## What data does Subconscious access?
By default, production prompts, completions, source code, API keys, API Gateway logs, Inference Runtime logs, and operational data should remain in the customer environment. Customers may choose to share selected logs, screenshots, metrics, or traces for support.
See [Compliance](/on-prem/trust-center/compliance).
## How are upgrades and patches delivered?
The Distribution Platform delivers releases, patches, and release metadata. Customers can approve updates through their own change-management process or choose an assisted update workflow.
See [Distribution Platform](/on-prem/distribution-platform/overview) and [Customer success](/on-prem/integration-journey/customer-success).
## Can we control when updates are deployed?
Yes. Production updates are intended to be customer-approved and deployed according to your policy, maintenance windows, and change-management requirements.
See [Methods](/on-prem/deployments/methods) and [Compliance](/on-prem/trust-center/compliance).
## Do you support vulnerability scanning?
Subconscious can provide or work toward providing release evidence such as vulnerability scan results, SBOMs, image digests, checksums, and release notes. Customers can also scan release artifacts and deployed components with their own tools.
See [Compliance](/on-prem/trust-center/compliance) and [Distribution Platform](/on-prem/distribution-platform/overview).
## What clouds and Kubernetes environments are supported?
The deployment can be planned for AWS, GCP, Azure, NeoCloud or dedicated GPU providers, and customer-controlled Kubernetes environments. Exact support depends on your cloud, networking, GPU placement, registry, and security requirements.
See [Configurations](/on-prem/deployments/configurations).
## Do we need to bring our own GPUs?
Not always. Customers can provide approved GPU capacity, use cloud GPU resources, or route through specialized GPU providers depending on the deployment design. We can also help you source GPU capacity through our network of partnerships.
See [Configurations](/on-prem/deployments/configurations).
## If a new model comes out, can we deploy it?
Yes, as long as the model architecture is supported by the Inference Runtime and enough GPU capacity is available. If existing capacity is already allocated, we may need to add GPU resources or replace an existing route before deploying the new model.
See [Customer success](/on-prem/integration-journey/customer-success).
## Which coding agents are supported?
The API Gateway exposes OpenAI- and Anthropic-compatible endpoints, so teams can connect common coding agents (Claude Code, Cursor, Codex, OpenCode, Pi, etc.) and internal tools that support those API shapes.
See [API Gateway setup](/on-prem/api-gateway/setup).
## What happens if the deployment has an incident?
The customer owns incident response for the API Gateway and Inference Runtime in its environment. Subconscious supports investigation and remediation when requested by the customer, using customer-approved access or diagnostic sharing.
See [Customer success](/on-prem/integration-journey/customer-success) and [Compliance](/on-prem/trust-center/compliance).
# How It Works
Source: https://docs.subconscious.dev/on-prem/how-it-works
The API Gateway, Inference Runtime, and Distribution Platform behind the Subconscious Inference System
The Subconscious Inference System consists of three core components:
* The **API Gateway**, which handles agent traffic, authentication, routing, API keys, and usage controls.
* The **Inference Runtime**, which drives Subconscious's agent-native, compute-efficiency advantages.
* The **Distribution Platform**, which manages software delivery, versioning, updates, and assisted self-managed deployment.
```mermaid theme={null}
flowchart TD
subgraph distribution["Distribution Platform"]
direction TB
releases["Subconscious releases"]
distr["Customer deployment portal"]
releases -->|"versions, signs, scans, packages"| distr
end
codingAgents["Coding agents"]
subgraph customer["Customer Cloud"]
direction TB
gateway["API Gateway"]
runtime["Inference Runtime"]
gpus["Customer GPUs"]
gateway --> runtime
runtime --> gpus
end
codingAgents --> gateway
distr -->|"opt-in or auto-deploy"| gateway
distr -->|"opt-in or auto-deploy"| runtime
style distribution fill:#fff3e8,stroke:#ff5c28,stroke-width:1px,color:#5f2411
style customer fill:#eef7ff,stroke:#4ea3ff,stroke-width:1px,color:#17324d
```
## API Gateway
The **API Gateway** is the customer-facing entry point for coding-agent traffic. It runs in the customer-controlled environment and it handles things like:
* End-user API requests from coding agents and SDK clients.
* Load balancing.
* OpenAI- and Anthropic-compatible gateway endpoints.
* Model routing to the Inference Runtime or external model endpoints.
* User and API key management.
* Access controls, limits, and usage tracking.
* Runtime observability, readiness, and operational dashboards.
See [API Gateway](/on-prem/api-gateway/overview).
## Inference Runtime
The **Inference Runtime** is the GPU-backed execution layer that gives the Subconscious Inference System its efficiency advantage. It runs on customer-provided GPUs and is tuned for coding-agent workloads.
It handles things like:
* Model execution on customer GPU resources.
* Cache behavior for agent workloads.
* Batching and scheduling.
* GPU utilization improvements.
* Serving more coding-agent workload per GPU.
See [Inference Runtime](/on-prem/inference-runtime/overview).
## Distribution Platform
The **Distribution Platform** is powered by [Distr](https://distr.sh/), a purpose-built software distribution platform for companies shipping software into customer-controlled environments.
It helps deliver:
* Licensed artifacts.
* Deployment instructions.
* Updates and patches.
* Release metadata.
* Vulnerability reports.
* Customer-specific deployment workflows.
See [Distribution Platform](/on-prem/distribution-platform/overview).
# Overview
Source: https://docs.subconscious.dev/on-prem/inference-runtime/overview
The GPU-backed execution layer for coding-agent workloads
The **Inference Runtime** is the GPU-backed execution layer that gives the Subconscious Inference System its efficiency advantage. It runs on customer-provided GPUs and is tuned for coding-agent workloads.
It handles things like:
* Model execution on customer GPU resources.
* Cache behavior for agent workloads.
* Batching and scheduling.
* GPU utilization improvements.
* Serving more coding-agent workload per GPU.
Depending on the deployment design, the Inference Runtime may be colocated with the API Gateway in the same cloud, run in a separate customer-controlled GPU environment, or use specialized GPU capacity behind the customer endpoint.
## Placement options
Inference Runtime placement is planned during deployment configuration. Common patterns include:
* **API Gateway and Inference Runtime in the same cloud**: simplest private networking when the customer's main cloud has approved GPU capacity.
* **API Gateway in your cloud, Inference Runtime elsewhere**: useful when the gateway should stay near the customer's stack but GPU capacity comes from a NeoCloud or specialized GPU provider.
* **API Gateway and Inference Runtime in separate customer-controlled environments**: useful when GPU capacity already exists in another customer-controlled cluster, region, or on-prem environment.
See [Configurations](/on-prem/deployments/configurations).
# Customer Success
Source: https://docs.subconscious.dev/on-prem/integration-journey/customer-success
Support, upgrades, reviews, and expansion after onboarding
Customer success starts after the initial deployment is working and pilot users are connected. The goal is to keep the Subconscious Inference System healthy, make upgrades predictable, improve cost-performance over time, and help adoption grow across the engineering organization.
## Support model
After onboarding, the customer support channel remains open and the assigned Subconscious FDE remains the primary point of contact.
Support commonly includes:
* Deployment and configuration questions.
* Operational troubleshooting.
* Upgrade and patch coordination.
* Performance reviews.
* New model or route configuration.
* Guidance for expanding coding-agent adoption.
* Help interpreting logs, metrics, deployment status, and usage patterns.
Support access to the customer environment is not required by default. When access is needed, it should be customer-approved, scoped, logged, time-bound, and revoked when the support task is complete. See [Compliance](/on-prem/trust-center/compliance) for the security framing.
## Upgrades and patches
Subconscious delivers upgrades and patches through the software distribution process. The customer controls approval and deployment cadence unless they explicitly choose an assisted policy.
Common options include:
* Opt-in upgrades approved by the customer.
* Scheduled upgrades during customer-approved maintenance windows.
* Automatic or FDE-assisted update pushes in assisted self-managed deployments.
* Emergency security patch coordination for critical vulnerabilities.
Upgrade planning usually covers release notes, vulnerability evidence, compatibility, rollback process, maintenance windows, and customer communication. See [Distribution Platform](/on-prem/distribution-platform/overview) and [Configurations](/on-prem/deployments/configurations) for more detail.
## Operational reviews
Operational reviews help confirm that the deployment continues to meet customer goals as adoption grows.
Review topics may include:
* Usage trends by model, team, key, or time period.
* Cost and GPU utilization.
* Latency, throughput, and error rates.
* Route health and model performance.
* User adoption and pilot feedback.
* Support history and recurring operational issues.
* Expansion opportunities and new model needs.
These reviews can be lightweight monthly digests, quarterly meetings, or customer-specific check-ins depending on the deployment size and rollout stage.
## Expanding adoption
Most customers start with a small pilot group before broad rollout. Expansion usually happens after the team validates model quality, API Gateway reliability, limits, observability, and support workflows.
Expansion planning usually includes:
* Adding more pilot teams.
* Creating new API keys or key ownership patterns.
* Adjusting limits based on real usage.
* Adding or tuning model routes.
* Scaling the Inference Runtime with more GPU resources behind the API Gateway.
* Updating internal setup instructions for coding agents.
* Sharing training material with additional engineering teams.
## Incident support
The customer owns incident response for the Subconscious Inference System in its environment. Subconscious supports investigation and remediation when requested.
During an incident, support may include:
* Reviewing deployment status, route health, logs, metrics, and recent changes.
* Helping identify whether the issue is API Gateway, route, Inference Runtime, worker, network, capacity, credential, or client related.
* Coordinating patch or rollback guidance.
* Helping prepare a support bundle or diagnostic summary.
Any logs, screenshots, traces, or access shared with Subconscious should follow the customer's approval and redaction process.
# Evaluation
Source: https://docs.subconscious.dev/on-prem/integration-journey/evaluation
Discovery, ROI, security review, and optional trials
Evaluation helps your team decide whether the Subconscious Inference System is the right customer-hosted path before implementation starts.
The goal is to align on:
* **Business value**: cost, productivity, and expected adoption.
* **Technical fit**: cloud, Kubernetes, GPU, networking, and coding-agent requirements.
* **Security review**: data boundary, vendor assessment, and compliance fit.
* **Proof points**: what must be validated before deployment.
## Discovery
Discovery starts with your engineering workflows and coding-agent usage.
Key topics:
* Engineering team size and expected adoption.
* Current coding-agent usage and preferred tools.
* Target model quality, latency, and throughput expectations.
* Workload patterns and anticipated token volume.
* Existing cloud, Kubernetes, GPU, and networking constraints.
* Security, compliance, data residency, and vendor review requirements.
* Success criteria and evaluation timeline.
**Output**: a clear evaluation path: what must be validated, who needs to participate, and which deployment model is most likely to fit.
## ROI exercise
The ROI exercise compares customer-hosted Subconscious against frontier hosted APIs, generic self-hosted GPU options, and unmanaged open-model serving.
Useful inputs:
* Expected number of engineers and pilot users.
* Agent usage patterns and expected daily or monthly token volume.
* Target model families and quality expectations.
* Current hosted API spend or internal GPU cost assumptions.
* Latency and throughput targets for interactive coding workflows.
* Required control over data, networking, infrastructure, and deployment cadence.
**Output**: a shared view of expected savings, GPU needs, reliability goals, and what the pilot or deployment must prove.
## Security review
Security review usually runs alongside commercial and technical evaluation.
**Key question**: Does the customer-hosted model fit your data, IP, security, and compliance requirements?
Subconscious can support review with:
* Security architecture overview.
* Data-flow and data-retention summary.
* API Gateway, Inference Runtime, and Distribution Platform boundary.
* Access control and support access model.
* Vulnerability management and patching process.
* Release evidence, SBOMs, vulnerability reports, or related supply-chain materials where available.
* Shared responsibility guidance.
For the detailed security position, see [Compliance](/on-prem/trust-center/compliance). For deployment mechanics, see [Methods](/on-prem/deployments/methods) and [Distribution Platform](/on-prem/distribution-platform/overview).
## Optional model comparison trial
Some customers evaluate model quality before committing to a full customer-hosted deployment.
Typical shape:
1. Select 1-4 engineers or a small pilot group.
2. Choose representative coding-agent workflows.
3. Point local coding agents at open models hosted on-demand.
* [OpenRouter](https://openrouter.ai/)
* [Subconscious Cloud API](/ways-to-use/cloud-api)
* NeoCloud or dedicated GPU providers such as [Baseten](https://www.baseten.co/) or [Together AI](https://www.together.ai/)
4. Compare model quality, compatibility, latency, and workflow fit.
5. Capture gaps or configuration requirements before deployment planning.
**Security note**: Cloud hosted trials are separate from customer-hosted production deployments and should be evaluated under their own data-handling assumptions.
## Optional load-test trial
A load-test trial validates whether Subconscious can serve the target engineering capacity with acceptable latency, throughput, and reliability.
Typical shape:
1. Agree on traffic assumptions or benchmark tasks.
2. Provision an appropriate GPU environment.
3. Run throughput and latency tests on representative workloads.
4. Compare results against success criteria from discovery.
5. Decide whether to proceed to deployment planning.
**Success criteria**: against target engineering capacity, there is acceptable latency for coding-agent workflows and stable throughput under expected load.
## Contracts & onboarding planning
If we agree there is a fit, we move forward with contract negotiation and onboarding planning. After that, customers move into [Onboarding](/on-prem/integration-journey/onboarding).
# Onboarding
Source: https://docs.subconscious.dev/on-prem/integration-journey/onboarding
From kickoff to pilot users on the Subconscious Inference System
Onboarding turns the evaluation plan into a working customer-hosted deployment. A Subconscious FDE works with your technical champion to deploy the Subconscious Inference System, configure the API Gateway, connect pilot users, and hand off day-two operations.
This page gives an overview. For implementation details, use [Methods](/on-prem/deployments/methods), [Configurations](/on-prem/deployments/configurations), and [API Gateway setup](/on-prem/api-gateway/setup).
## Summary
* **Kickoff to first pilot use**: usually about **1-2 weeks**, assuming customer technical resources are available.
* **Technical champion effort**: usually less than **8 hours** of focused configuration time when the environment is ready.
* **Admin training**: usually less than **1 hour**.
* **Broader team training**: usually less than **1 hour**.
* **Communication channel**: Dedicated Slack channel.
* **Onsite support**: available when it helps accelerate setup.
Strict change management, private registries, custom networking, air-gapped requirements, or new GPU procurement can extend the timeline.
## Stakeholders
* **Subconscious**: CEO and assigned FDE.
* **Customer executive sponsor**: Owns business value, rollout priority, and commercial alignment.
* **Customer technical champion**: Drives deployment planning, environment readiness, and technical coordination.
* **Core admin team**: Manages API Gateway access, API keys, routes, limits, monitoring, and day-two operations.
## 1. Kickoff
Kickoff aligns the people, communication channels, and success criteria before deployment work begins.
* **Ceremony**: 30 min meeting or async.
* **Purpose**:
* Establish communication channels, expectations, and executive sponsorship.
* Confirm implementation timeline and success criteria from evaluation.
* **What happens**:
* A dedicated Slack channel or customer-approved support channel is created.
* A Subconscious FDE becomes the primary onboarding point of contact.
* The customer identifies the technical champion, admin team, security reviewers, and pilot sponsor.
* The team confirms success criteria, timeline, and responsibilities.
## 2. Deployment planning
Deployment planning defines how the Subconscious Inference System will run in the customer's environment.
* **Ceremony**: 60 min meeting plus async follow-up.
* **Purpose**:
* Confirm the target deployment shape.
* Resolve infrastructure, cloud, networking, GPU, and Distr requirements.
* **Topics to define**:
* Cloud provider, region, Kubernetes environment, and namespace.
* GPU sourcing and placement.
* Endpoint strategy, DNS, TLS, ingress, and network policy.
* Deployment method: assisted self-managed or traditional self-hosted.
* Distr customer portal, artifacts, deployment agent, registry access, and secrets.
* Observability, logging, alerting, and support access expectations.
For the detailed decision points, see [Methods](/on-prem/deployments/methods) and [Distribution Platform](/on-prem/distribution-platform/overview).
## 3. Deploy the Subconscious Inference System
This step installs the API Gateway, Inference Runtime, and supporting services your engineers will use.
* **Ceremony**: 30-60 min meeting plus async follow-up.
* **Purpose**: Deploy the Subconscious Inference System into the customer's cloud environment.
* **What gets deployed**:
* API Gateway for agent traffic, authentication, routing, and usage controls.
* Inference Runtime for GPU-backed model execution.
* Routing, load balancing, authentication, and usage control configuration.
* Runtime admin dashboard.
* Baseline observability and operational dashboards.
* **Customer responsibilities**:
* Provision or approve required infrastructure and GPU resources.
* Approve deployment commands, agent installation, or internal GitOps changes.
* **Subconscious responsibilities**:
* Guide deployment and configuration.
* Validate API Gateway and Inference Runtime health.
* Prepare the API Gateway for setup.
See [Configurations](/on-prem/deployments/configurations) for cloud, GPU placement, Helm overrides, secrets, monitoring, logging, and upgrade details.
## 4. API Gateway setup and admin training
API Gateway setup turns the installed system into a usable service for pilot users.
* **Ceremony**: 30-60 min meeting plus async follow-up.
* **Purpose**: Configure the API Gateway and train the customer's project leads to operate it.
* **What gets configured and taught**:
* Admin users and pilot users.
* User roles, invites, and password reset workflows.
* API Gateway API keys.
* Model routes to the Inference Runtime or external model endpoints.
* Model access grants.
* Organization-level and API-key-level limits.
* Test requests through the API Gateway.
* Usage, monitoring, logging, and operational workflows.
* General debugging, troubleshooting, support cases, upgrades, and maintenance workflows.
See [API Gateway setup](/on-prem/api-gateway/setup) for the detailed setup guide.
## 5. Coding agent integration and team training
Coding agent integration helps pilot users connect their preferred tools to the deployed customer endpoint.
* **Ceremony**: One 30 min live training session, with optional recording and written setup material.
* **Purpose**: Help the broader engineering team point their preferred coding agents at the deployed API Gateway endpoint.
* **Topics to cover**:
* What the system does and how it fits into the customer environment.
* API Gateway base URL, API keys, and model slugs.
* OpenAI-compatible configuration for coding agents and internal tools.
* Expectations for limits, acceptable use, support, and feedback.
Rollout usually starts with a small pilot group, then expands after usage, reliability, and route behavior are validated.
## 6. Handoff
Handoff confirms the deployment is ready for pilot use and day-two ownership.
* **Ceremony**: 30 min meeting with all stakeholders for sign-off.
* **Purpose**:
* Confirm the deployment is ready for pilot users.
* Confirm customer owners understand how to operate and escalate issues.
* **Sign-off checklist**:
* Customer admins can sign in and operate the API Gateway.
* API keys, routes, access, limits, and test requests are working.
* Pilot users can connect coding agents.
* Monitoring and logging ownership is clear.
* Support and escalation paths are understood.
* Upgrade, rollback, and patch expectations are documented.
After handoff, the relationship continues through [Customer success](/on-prem/integration-journey/customer-success).
# Overview
Source: https://docs.subconscious.dev/on-prem/integration-journey/overview
How evaluation, onboarding, and long-term support work
This guide explains the path from first evaluation to production use of the Subconscious Inference System.
## Typical phases
* **[Evaluation](/on-prem/integration-journey/evaluation)**: Align on use cases, security posture, ROI, expected usage, and proof points.
* **Timeline**: 2-4 weeks.
* **Ceremonies**: 2-4 meetings.
* **[Onboarding](/on-prem/integration-journey/onboarding)**: Plan deployment, stand up the Subconscious Inference System, configure the API Gateway, and connect pilot users.
* **Timeline**: 1-2 weeks.
* **Ceremonies**: kickoff, deployment planning, deployment/setup, team training, and handoff.
* **[Customer success](/on-prem/integration-journey/customer-success)**: Keep the support channel open for upgrades, patches, performance reviews, and rollout guidance.
## Stakeholders
### Your side
* **Executive sponsor**: Confirms business value, pilot success criteria, rollout priority, and expansion goals.
* **Technical lead**: Owns deployment coordination across cloud, Kubernetes, networking, GPU capacity, API Gateway setup, and day-two operations.
* **Security or compliance reviewer**: Reviews the data boundary, support access model, software supply-chain evidence, and vendor risk and compliance posture.
* **Core evaluation team**: Tests coding-agent workflows, validates model quality, confirms tool compatibility, and provides pilot feedback.
### Our side
* **Subconscious executive sponsor**: Our CEO, responsible for executive alignment, commercial context, and escalation when needed.
* **Subconscious FDE**: Leads evaluation, deployment planning, setup, training, and handoff.
* **Subconscious engineering**: Supports architecture review, performance tuning, troubleshooting, upgrades, and release planning.
# Overview
Source: https://docs.subconscious.dev/on-prem/overview
What is the Subconscious Inference System, what are the benefits, and who is it for
The **Subconscious Inference System** is a GPU orchestration platform designed to serve coding agents with industry-leading efficiency.
The system is deployed into the customer's cloud and runs on customer-provided GPUs, giving enterprises control over their infrastructure while Subconscious provides the software layer for reliable, efficient agent inference.
Prompts, completions, keys, logs, and operational data stay in your environment unless you choose to share them.
Engineers use OpenAI- and Anthropic-compatible endpoints from tools such as Cursor, Claude Code, OpenCode, and internal agents.
Subconscious FDEs help plan, deploy, configure, and support the system with your platform and security teams.
## Benefits
* **Protect data and IP**: Tools like Claude Code or Codex can ship prompts, code, and engineering context outside your trust boundary. The Subconscious Inference System keeps coding-agent inference in infrastructure you control.
* **Own your intelligence economics**: Rent GPUs, choose the model, and serve your own coding intelligence instead of depending on closed APIs with high or unpredictable costs.
* **Cut GPU spend for agent workloads**: The Inference Runtime is designed to serve teams of engineers with roughly half the GPUs compared with off-the-shelf inference runtimes like vLLM.
* **Frontier-level intelligence**: Open models like GLM 5.2 now match leading closed models on coding-agent performance, making owned deployment a practical option.
* **Reduce platform engineering burden**: Subconscious helps manage the API Gateway, Inference Runtime, Distribution Platform, upgrades, and support so your platform team can focus on higher-value work.
See the [Subconscious pricing page](https://www.subconscious.dev/pricing) for the ROI calculator and savings model.
## Who it is for
The Subconscious Inference System is for enterprises that want frontier-level coding agents without giving up control over data, IP, cost, or reliability.
It is usually a fit when:
* You have 50+ software engineers using or evaluating coding agents.
* Security or IP requirements make hosted tools hard to approve.
* You want predictable costs and owned GPU capacity.
* You do not want to build and maintain a full self-hosting stack yourself.
# Compliance
Source: https://docs.subconscious.dev/on-prem/trust-center/compliance
Security review, data handling, support access, and supply-chain assurance
The customer-hosted Subconscious Inference System is designed for enterprises that want high-performance coding-agent inference while retaining control over cloud environment, data, networking, system operations, and software deployment cadence.
This page explains how the deployment model supports security review, vendor assessment, and long-term operational trust. It is written for security, compliance, platform, and engineering teams evaluating Subconscious.
## Short answer
The customer-hosted Subconscious Inference System is designed to support a customer's existing security and compliance review process.
For production customer-hosted deployments:
* The API Gateway and Inference Runtime run in the customer's cloud account or customer-controlled environment.
* Customer requests, prompts, completions, logs, keys, and operational data remain in the customer environment unless the customer explicitly chooses otherwise.
* Subconscious does not host the customer's production inference system.
* Subconscious does not require persistent access to the customer's cloud account.
* Subconscious does not need ingress into the customer's cloud account.
* Subconscious does not control production availability.
* The customer controls versioning, deployment cadence, networking, access, monitoring, and Subconscious Inference System operations.
* Updates are customer-approved and installable through the customer's own change-management process unless the customer explicitly chooses an assisted update workflow.
This model helps customers evaluate Subconscious as customer-hosted software and software supply-chain risk, rather than as a hosted production service operating the customer's inference environment. Final vendor and audit treatment is determined by the customer and its auditor.
## Scope note
For customer-hosted production deployments, customers commonly review Subconscious as a software vendor and software supply-chain vendor. The API Gateway and Inference Runtime run under the customer's deployment, access, monitoring, networking, and change-management controls.
Subconscious can provide architecture documentation, data-flow evidence, release integrity materials, and shared responsibility documentation to support that review.
## Deployment model
The Subconscious Inference System has three core components:
* **API Gateway**: The customer-facing entry point for agent traffic, authentication, API key management, routing, usage controls, load balancing, and customer-facing admin workflows.
* **Inference Runtime**: The GPU-backed execution layer that runs on customer-provided GPUs and drives Subconscious's compute-efficiency advantage.
* **Distribution Platform**: The software delivery and update workflow used to distribute new versions, patches, and release metadata to customer-controlled environments.
The API Gateway and Inference Runtime serve the customer's engineering users. They run inside the customer's environment and are configured according to the customer's cloud, Kubernetes, networking, IAM, and security requirements.
The Distribution Platform is used for delivering software. It is not required to receive customer code, prompts, completions, application logs, API Gateway logs, Inference Runtime logs, or production inference data.
## Security review position
Subconscious is designed to support the following review position for customer-hosted deployments:
> The product runs in the customer's cloud account. In the standard customer-hosted configuration, Subconscious does not host the production system, does not require persistent access, and is not intended to receive production prompts, completions, source code, or production inference data. Updates are optional, customer-approved, signed where available, and installable through the customer's own change-management process.
This position depends on four operating principles:
* **Customer-controlled data handling by default**: Production inference traffic stays in the customer environment.
* **Production operation under customer control**: The API Gateway and Inference Runtime operate under the customer's operational control.
* **Customer-controlled deployment cadence**: The customer chooses when to approve and deploy new versions.
* **Customer-controlled access**: Subconscious support access is optional, customer-approved, time-bound, and logged.
## Shared responsibility model
Subconscious and the customer share responsibility for a secure deployment.
### Subconscious responsibilities
Subconscious is responsible for:
* Secure development of Subconscious software.
* Release integrity for distributed software artifacts.
* Release notes, version metadata, and update communication.
* Security patch development and disclosure.
* Documented deployment guidance.
* Support for installation, configuration, troubleshooting, and upgrades.
* Clear documentation of optional telemetry or support access.
### Customer responsibilities
The customer is responsible for:
* Cloud account security.
* Ownership and administration of the Kubernetes cluster.
* IAM, access control, and identity integration.
* Network configuration, ingress, egress, and firewall rules.
* GPU provisioning and capacity planning.
* API Gateway and Inference Runtime monitoring, alerting, and incident response.
* Backups and retention for customer-managed data stores.
* Final ownership of vulnerability scanning and continuous monitoring in the customer environment.
* Change management and approval of new releases.
* Internal rollout to engineering users.
### Joint responsibilities
Subconscious and the customer work together on:
* Deployment planning.
* Security review.
* Kubernetes cluster configuration for the Subconscious Helm chart.
* API Gateway and Inference Runtime configuration.
* Initial route and endpoint setup.
* Performance validation.
* Vulnerability scanning setup, scan result review, and remediation planning.
* Continuous monitoring setup and operational review.
* Upgrade planning.
* Automatic update pushes when the customer chooses that assisted self-managed mode.
* Incident investigation when customer-approved support is needed.
## Data handling
For customer-hosted production deployments, Subconscious is designed so customer data remains inside the customer environment.
The following should remain in the customer's cloud unless the customer explicitly chooses to share it:
* Source code.
* Prompts and completions.
* Inference requests and responses.
* API keys and user records.
* Runtime logs.
* Application logs.
* Operational metrics.
* Customer-specific route configuration.
Subconscious does not need default access to:
* Customer repositories.
* Customer source code.
* Customer prompts or completions.
* Customer production logs.
* Customer cloud credentials.
* Customer databases.
* Customer identity provider data.
Customers may choose to share selected logs, metrics, screenshots, traces, or configuration details during support or troubleshooting. Any such sharing should be customer-initiated or customer-approved, limited to the issue being investigated, shared through an approved support channel, redacted where appropriate, and time-bound.
## Telemetry
Customer-hosted deployments should be configured with no customer data telemetry by default.
If telemetry is enabled, it should be:
* Explicitly opt-in.
* Documented before activation.
* Limited to the fields required for the stated operational purpose.
* Configurable by the customer.
* Disabled through the customer's own deployment controls.
Telemetry should not include customer source code, prompts, completions, secrets, API keys, or other sensitive customer data.
## Support access
Subconscious does not require persistent access to the customer's cloud environment.
Support access, when needed, should follow the customer's access policy. Common support models include:
* Live screen-share sessions where the customer drives.
* Customer-approved temporary access.
* Break-glass access for urgent issues.
* Time-bound cloud console access.
* One-time credentials issued by the customer.
* Customer-provided logs or diagnostics exported from the customer's own tools.
Any support access should be approved by the customer, scoped to the support need, logged by the customer, time-bound, and revoked when the support task is complete.
## Release and update controls
Subconscious releases are intended to fit into the customer's change-management process.
Customers should be able to:
* Review release notes before deployment.
* Review version metadata and artifacts.
* Approve updates before they are installed.
* Choose automatic or opt-in update policies.
* Schedule maintenance windows.
* Roll back when needed.
* Maintain control over production deployment cadence.
Subconscious should not force production upgrades into a customer environment without customer approval.
## Software supply-chain assurance
Because Subconscious software runs in the customer environment, the primary security review focus is software supply-chain assurance.
During evaluation, Subconscious can provide or work toward providing a security assurance packet that includes:
* Architecture overview.
* Data-flow diagram.
* Shared responsibility matrix.
* Release notes.
* Signed containers, Helm charts, binaries, or checksums where available.
* Software bill of materials for releases where available.
* Vulnerability scan results for images and packages.
* Dependency management process.
* Secure development lifecycle overview.
* Release approval process.
* Incident disclosure policy.
* Support access policy.
* Optional offline or air-gapped installation guidance.
* Customer-controlled update process.
## Vulnerability management
Subconscious is responsible for addressing vulnerabilities in Subconscious-maintained software artifacts.
The expected vulnerability management process includes:
* Vulnerability scanning of release artifacts.
* Dependency monitoring.
* Prioritization based on severity, exploitability, and customer exposure.
* Security patches for affected supported versions.
* Customer notification for material security issues.
* Release notes or advisories describing the remediation path.
The customer remains responsible for scanning and monitoring the deployed environment according to its own security program. Subconscious can assist with interpretation, remediation planning, and patch coordination.
## Offline and restricted environments
Some customers may require restricted network access, private registries, or offline installation paths.
Subconscious can support deployment planning for environments that require:
* Private container registries.
* Customer-managed artifact mirrors.
* Restricted egress.
* Customer-controlled update promotion.
* Air-gapped or semi-air-gapped workflows.
* Internal vulnerability scanning before deployment.
The exact deployment path depends on the customer's cloud, Kubernetes environment, registry model, and security requirements.
## Hosted trials and production deployments
Subconscious may offer hosted trials or model comparison trials before a customer-hosted production deployment.
Hosted trials are separate from customer-hosted production deployments. A hosted trial may be useful for evaluating model quality, agent compatibility, or workflow fit, but it should be assessed under its own security and data-handling terms.
For production customer-hosted deployments, the API Gateway and Inference Runtime are deployed in the customer's cloud and governed by the customer-controlled model described in this document.
## Security review materials
During evaluation, Subconscious can provide the following materials to support customer security review:
* Customer-hosted architecture memo.
* Data-flow diagram.
* Shared responsibility matrix.
* Software supply-chain controls summary.
* Release and update process.
* Vulnerability management summary.
* Support access policy.
* Telemetry statement.
* Deployment guide.
* Security and vendor risk FAQ.
## FAQ
### How should we think about SOC 2 and vendor risk review?
For customer-hosted deployments, the key review point is that the API Gateway and Inference Runtime run in the customer's cloud account, under the customer's access, monitoring, networking, and change-management controls.
Subconscious can provide security assurance materials to support vendor risk review and SOC 2 audit discussions. Final audit treatment is determined by the customer and its auditor.
### How does this affect our SOC 2 review?
The deployment is designed to fit into the customer's existing security, compliance, and change-management process.
The customer retains control over production deployment, API Gateway and Inference Runtime operation, monitoring, scanning, access, data, and version cadence. Subconscious does not require persistent production access and does not need customer data to operate the Subconscious Inference System.
Final audit treatment is determined by the customer and its auditor.
### Does Subconscious process or store our customer data?
For customer-hosted production deployments, Subconscious is not intended to process or store customer data. Inference requests, prompts, completions, logs, keys, and operational data remain in the customer's environment unless the customer explicitly chooses to share specific data for support or troubleshooting.
### Where do source code, prompts, and completions live?
In the standard customer-hosted production deployment, source code, prompts, and completions should remain in the customer's environment unless the customer explicitly configures or shares them.
### Does Subconscious require access to our cloud account?
No persistent access is required by default. If support access is needed, it should be customer-approved, scoped, logged, time-bound, and revoked after use.
### Who controls updates?
The customer controls update approval and deployment cadence. Subconscious provides software releases, patches, release notes, and upgrade guidance. Customers can approve updates through their own change-management process.
### How are production updates approved?
Production updates should be customer-approved and deployed according to the customer's policy.
### What is the main security risk we should evaluate?
The main risk category is software supply-chain risk. Customers should review how Subconscious builds, signs, scans, documents, and distributes software artifacts that run in the customer environment.
### Do you provide vulnerability scan results?
Subconscious can provide vulnerability scan results for release artifacts as part of the security assurance packet. Customers may also scan images, packages, and deployed components inside their own environment.
### Do you provide an SBOM?
Subconscious can provide or work toward providing software bill of materials documentation for releases as part of the software supply-chain assurance package.
### Can we scan the software ourselves?
Yes. Customers can scan release artifacts and deployed components using their own tools before approving deployment.
### Is telemetry enabled by default?
Customer-hosted deployments should not send customer data telemetry by default. Any telemetry should be explicit, documented, configurable, and opt-in. Subconscious helps the customer to configure monitoring and logging into their systems of record for complete ability to support their deployments.
### What happens during an incident?
The customer owns incident response for the API Gateway and Inference Runtime in its environment. Subconscious supports investigation and remediation when requested by the customer. Any access or data sharing during an incident should follow the customer's approval and logging process.
### Can we run in a restricted or air-gapped environment?
Subconscious can support planning for restricted, private registry, semi-air-gapped, or air-gapped deployment workflows. Exact support depends on the customer's environment and release distribution requirements.
# Subconscious
Source: https://docs.subconscious.dev/overview
Subconscious is an AI lab that makes language models dramatically more capable with our inference runtime TIMRUN and complementary post-trained TIM family of models. Our API is compatible with both the OpenAI Completions and Anthropic Messages APIs, so you can get started with our API with three lines of code with the SDK you already use.
## Quick Look
```python Python (OpenAI) theme={null}
from openai import OpenAI
client = OpenAI(
api_key="your-api-key", # Step 1: add your API key
base_url="https://api.subconscious.dev/v1", # Step 2: point to our api
)
response = client.chat.completions.create(
model="subconscious/tim-qwen3.6-27b", # Step 3: set the model. You're all set!
messages=[{"role": "user", "content": "What is the capital of France?"}],
)
print(response.choices[0].message.content)
```
```typescript Node.js (OpenAI) theme={null}
import OpenAI from "openai";
const client = new OpenAI({
apiKey: "your-api-key",
baseURL: "https://api.subconscious.dev/v1",
});
const response = await client.chat.completions.create({
model: "subconscious/tim-qwen3.6-27b",
messages: [{ role: "user", content: "What is the capital of France?" }],
});
console.log(response.choices[0].message.content);
```
```python Python (Anthropic) theme={null}
from anthropic import Anthropic
client = Anthropic(
auth_token="your-api-key",
base_url="https://api.subconscious.dev",
)
message = client.messages.create(
model="subconscious/tim-qwen3.6-27b",
max_tokens=1024,
messages=[{"role": "user", "content": "What is the capital of France?"}],
)
print(message.content[0].text)
```
```typescript Node.js (Anthropic) theme={null}
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic({
authToken: "your-api-key",
baseURL: "https://api.subconscious.dev",
});
const message = await client.messages.create({
model: "subconscious/tim-qwen3.6-27b",
max_tokens: 1024,
messages: [{ role: "user", content: "What is the capital of France?" }],
});
console.log(message.content[0].text);
```
```bash cURL (Chat Completions) theme={null}
curl https://api.subconscious.dev/v1/chat/completions \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "subconscious/tim-qwen3.6-27b",
"messages": [{"role": "user", "content": "What is the capital of France?"}]
}'
```
```bash cURL (Messages) theme={null}
curl https://api.subconscious.dev/v1/messages \
-H "x-api-key: YOUR_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-H "content-type: application/json" \
-d '{
"model": "subconscious/tim-qwen3.6-27b",
"max_tokens": 1024,
"messages": [{"role": "user", "content": "What is the capital of France?"}]
}'
```
To try this yourself, grab an API key on our [platform](https://subconscious.dev).
## Available Models
See [Pricing](https://www.subconscious.dev/pricing) for the full list of available models and current per-token rates.
## Get started
Make your first API call in under 5 minutes
Full endpoint documentation
### AI Readable
* [subconscious.dev/llms.txt](https://docs.subconscious.dev/llms.txt): Documentation index with links
* [subconscious.dev/llms-full.txt](https://docs.subconscious.dev/llms-full.txt): Complete documentation in one file
# Quickstart
Source: https://docs.subconscious.dev/quickstart
Make your first API call
## 1. Create an Account
[Sign up on the platform](https://subconscious.dev/platform) and generate an API key from your dashboard.
## 2. Try the Playground
Before writing code, try the [Playground](https://www.subconscious.dev/playground) to test prompts interactively.
## 3. Install an SDK
Subconscious is compatible with both the OpenAI and Anthropic SDKs — use whichever you
prefer. Install it for your language:
```bash Python (OpenAI) theme={null}
pip install openai
```
```bash Node.js (OpenAI) theme={null}
npm install openai
```
```bash Python (Anthropic) theme={null}
pip install anthropic
```
```bash Node.js (Anthropic) theme={null}
npm install @anthropic-ai/sdk
```
## 4. Make Your First Request
Set your API key and base URL, then create a chat completion:
```python Python (OpenAI) theme={null}
from openai import OpenAI
client = OpenAI(
api_key="your-api-key",
base_url="https://api.subconscious.dev/v1",
)
response = client.chat.completions.create(
model="subconscious/tim-qwen3.6-27b",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Explain quantum computing in three sentences."},
],
)
print(response.choices[0].message.content)
```
```typescript Node.js (OpenAI) theme={null}
import OpenAI from "openai";
const client = new OpenAI({
apiKey: "your-api-key",
baseURL: "https://api.subconscious.dev/v1",
});
const response = await client.chat.completions.create({
model: "subconscious/tim-qwen3.6-27b",
messages: [
{ role: "system", content: "You are a helpful assistant." },
{ role: "user", content: "Explain quantum computing in three sentences." },
],
});
console.log(response.choices[0].message.content);
```
```python Python (Anthropic) theme={null}
from anthropic import Anthropic
client = Anthropic(
auth_token="your-api-key",
base_url="https://api.subconscious.dev",
)
message = client.messages.create(
model="subconscious/tim-qwen3.6-27b",
max_tokens=1024,
system="You are a helpful assistant.",
messages=[
{"role": "user", "content": "Explain quantum computing in three sentences."},
],
)
print(message.content[0].text)
```
```typescript Node.js (Anthropic) theme={null}
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic({
authToken: "your-api-key",
baseURL: "https://api.subconscious.dev",
});
const message = await client.messages.create({
model: "subconscious/tim-qwen3.6-27b",
max_tokens: 1024,
system: "You are a helpful assistant.",
messages: [
{ role: "user", content: "Explain quantum computing in three sentences." },
],
});
console.log(message.content[0].text);
```
```bash cURL (Chat Completions) theme={null}
curl https://api.subconscious.dev/v1/chat/completions \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "subconscious/tim-qwen3.6-27b",
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Explain quantum computing in three sentences."}
]
}'
```
```bash cURL (Messages) theme={null}
curl https://api.subconscious.dev/v1/messages \
-H "x-api-key: YOUR_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-H "content-type: application/json" \
-d '{
"model": "subconscious/tim-qwen3.6-27b",
"max_tokens": 1024,
"system": "You are a helpful assistant.",
"messages": [
{"role": "user", "content": "Explain quantum computing in three sentences."}
]
}'
```
### Using an agents SDK
Want to use an SDK build for handling agents? Try one of these:
```python OpenAI Agents theme={null}
from openai import AsyncOpenAI
from agents import (
Agent,
Runner,
OpenAIChatCompletionsModel,
set_tracing_disabled,
)
API_KEY = "your-api-key"
MODEL = "subconscious/tim-qwen3.6-27b"
def run_openai_agent():
client = AsyncOpenAI(
api_key=API_KEY,
base_url="https://api.subconscious.dev/v1",
)
# Tracing would otherwise try to reach OpenAI's backend.
set_tracing_disabled(True)
# Pass an explicit Chat Completions model bound to our client. A bare
# model string would route through the MultiProvider, which treats the
# "subconscious/" segment as a provider prefix and fails.
agent = Agent(
name="Assistant",
instructions="You are a helpful assistant.",
model=OpenAIChatCompletionsModel(model=MODEL, openai_client=client),
)
result = Runner.run_sync(agent, "What is the capital of France?")
print("OpenAI Agents SDK response:")
print(result.final_output)
run_openai_agent()
```
```python Claude Agent SDK theme={null}
# The SDK reads its endpoint/credentials from env vars and drives the Claude
# Code CLI under the hood (so Node.js + the CLI must be installed). Set these
# before importing/using the SDK so the spawned CLI inherits them.
import asyncio
import os
from claude_agent_sdk import query, ClaudeAgentOptions
API_KEY = "your-api-key"
os.environ["ANTHROPIC_BASE_URL"] = "https://api.subconscious.dev"
os.environ["ANTHROPIC_AUTH_TOKEN"] = API_KEY
MODEL = "subconscious/tim-qwen3.6-27b"
async def run_claude_agent():
print("Claude Agent SDK response:")
options = ClaudeAgentOptions(model=MODEL)
async for message in query(
prompt="What is the capital of France?",
options=options,
):
print(message)
asyncio.run(run_claude_agent())
```
```python LangChain theme={null}
# Dependencies: langchain, langchain-openai, langchain-anthropic
from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
from langchain_anthropic import ChatAnthropic
API_KEY = "your-api-key"
MODEL = "subconscious/tim-qwen3.6-27b"
PROMPT = {"messages": [{"role": "user", "content": "What is the capital of France?"}]}
# ---------------------------------------------------------------------------
# OpenAI chat completions format
# ---------------------------------------------------------------------------
openai_model = ChatOpenAI(
model=MODEL,
api_key=API_KEY,
base_url="https://api.subconscious.dev/v1",
)
openai_agent = create_agent(openai_model, tools=[])
print("OpenAI format:")
print(openai_agent.invoke(PROMPT)["messages"][-1].content)
# ---------------------------------------------------------------------------
# Anthropic messages format
# ---------------------------------------------------------------------------
anthropic_model = ChatAnthropic(
model=MODEL,
base_url="https://api.subconscious.dev",
api_key="unused", # auth goes through the Bearer header below
default_headers={"Authorization": f"Bearer {API_KEY}"},
)
anthropic_agent = create_agent(anthropic_model, tools=[])
print("\nAnthropic format:")
print(anthropic_agent.invoke(PROMPT)["messages"][-1].content)
```
```typescript Vercel AI SDK theme={null}
// Dependencies: ai, @ai-sdk/openai-compatible
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { generateText } from "ai";
const API_KEY = "your-api-key";
const MODEL = "subconscious/tim-qwen3.6-27b";
const subconscious = createOpenAICompatible({
name: "subconscious",
apiKey: API_KEY,
baseURL: "https://api.subconscious.dev/v1",
});
const { text } = await generateText({
model: subconscious.chatModel(MODEL),
system: "You are a helpful assistant.",
prompt: "What is the capital of France?",
});
console.log(text);
```
## 5. Try Streaming
You can stream responses token by token for real-time output:
```python Python (OpenAI) theme={null}
stream = client.chat.completions.create(
model="subconscious/tim-qwen3.6-27b",
messages=[{"role": "user", "content": "Write a haiku about programming."}],
stream=True,
)
for chunk in stream:
content = chunk.choices[0].delta.content
if content:
print(content, end="", flush=True)
```
```typescript Node.js (OpenAI) theme={null}
const stream = await client.chat.completions.create({
model: "subconscious/tim-qwen3.6-27b",
messages: [{ role: "user", content: "Write a haiku about programming." }],
stream: true,
});
for await (const chunk of stream) {
const content = chunk.choices[0]?.delta?.content;
if (content) process.stdout.write(content);
}
```
```python Python (Anthropic) theme={null}
with client.messages.stream(
model="subconscious/tim-qwen3.6-27b",
max_tokens=1024,
messages=[{"role": "user", "content": "Write a haiku about programming."}],
) as stream:
for text in stream.text_stream:
print(text, end="", flush=True)
```
```typescript Node.js (Anthropic) theme={null}
const stream = client.messages.stream({
model: "subconscious/tim-qwen3.6-27b",
max_tokens: 1024,
messages: [{ role: "user", content: "Write a haiku about programming." }],
});
for await (const event of stream) {
if (
event.type === "content_block_delta" &&
event.delta.type === "text_delta"
) {
process.stdout.write(event.delta.text);
}
}
```
See [Streaming](/features/streaming) for more details.
## 6. Try Structured Output
You can also get typed JSON responses by providing a schema:
```python Python (OpenAI) theme={null}
response = client.chat.completions.create(
model="subconscious/tim-qwen3.6-27b",
messages=[{"role": "user", "content": "Analyze the sentiment of: 'I love this product!'"}],
response_format={
"type": "json_schema",
"json_schema": {
"name": "sentiment_analysis",
"schema": {
"type": "object",
"properties": {
"sentiment": {"type": "string", "enum": ["positive", "negative", "neutral"]},
"confidence": {"type": "number"},
},
"required": ["sentiment", "confidence"],
},
},
},
)
import json
result = json.loads(response.choices[0].message.content)
print(result) # {"sentiment": "positive", "confidence": 0.95}
```
```typescript Node.js (OpenAI) theme={null}
const response = await client.chat.completions.create({
model: "subconscious/tim-qwen3.6-27b",
messages: [{ role: "user", content: "Analyze the sentiment of: 'I love this product!'" }],
response_format: {
type: "json_schema",
json_schema: {
name: "sentiment_analysis",
schema: {
type: "object",
properties: {
sentiment: { type: "string", enum: ["positive", "negative", "neutral"] },
confidence: { type: "number" },
},
required: ["sentiment", "confidence"],
},
},
},
});
const result = JSON.parse(response.choices[0].message.content!);
console.log(result); // { sentiment: "positive", confidence: 0.95 }
```
```python Python (Anthropic) theme={null}
# The Messages API has no response_format; force a tool call whose
# input_schema is your target schema, then read the tool_use input.
tool = {
"name": "sentiment_analysis",
"description": "Record the sentiment analysis result.",
"input_schema": {
"type": "object",
"properties": {
"sentiment": {"type": "string", "enum": ["positive", "negative", "neutral"]},
"confidence": {"type": "number"},
},
"required": ["sentiment", "confidence"],
},
}
message = client.messages.create(
model="subconscious/tim-qwen3.6-27b",
max_tokens=1024,
tools=[tool],
tool_choice={"type": "tool", "name": "sentiment_analysis"},
messages=[{"role": "user", "content": "Analyze the sentiment of: 'I love this product!'"}],
)
result = next(b.input for b in message.content if b.type == "tool_use")
print(result) # {"sentiment": "positive", "confidence": 0.95}
```
```typescript Node.js (Anthropic) theme={null}
// The Messages API has no response_format; force a tool call whose
// input_schema is your target schema, then read the tool_use input.
const message = await client.messages.create({
model: "subconscious/tim-qwen3.6-27b",
max_tokens: 1024,
tools: [
{
name: "sentiment_analysis",
description: "Record the sentiment analysis result.",
input_schema: {
type: "object",
properties: {
sentiment: { type: "string", enum: ["positive", "negative", "neutral"] },
confidence: { type: "number" },
},
required: ["sentiment", "confidence"],
},
},
],
tool_choice: { type: "tool", name: "sentiment_analysis" },
messages: [{ role: "user", content: "Analyze the sentiment of: 'I love this product!'" }],
});
const block = message.content.find((b) => b.type === "tool_use");
console.log(block?.input); // { sentiment: "positive", confidence: 0.95 }
```
See [Structured Output](/features/structured-output) for more details.
## Next Steps
Real-time token-by-token output
Typed JSON responses with schemas
Enable step-by-step reasoning
Full endpoint documentation
# Learn More
Source: https://docs.subconscious.dev/resources/learn-more
Explore more about our work and our team
* [Blog](https://www.subconscious.dev/blog?ref=docs): Read more from the team on our recent releases.
* [Research](https://www.subconscious.dev/research?ref=docs): Read the technical report for our release and experiment with TIM.
* **Alpha**: Alpha is Hongyin's dog. She's the best of us.
![Alpha, Hongyin's dog]()
# API
Source: https://docs.subconscious.dev/ways-to-use/cloud-api
The standard Subconscious inference endpoint
The Subconscious Cloud API is the fastest way to access our inference infrastructure. All requests go through `api.subconscious.dev`, our managed gateway that handles authentication, rate limiting, and routing.
## How It Works
Use the standard OpenAI or Anthropic SDK with no custom libraries needed
Low-latency access from anywhere
We handle scaling, reliability, and uptime
## API Keys
API keys are created and managed from your [Subconscious dashboard](https://subconscious.dev/platform/api-keys).
```python Python (OpenAI) theme={null}
from openai import OpenAI
client = OpenAI(
api_key="your-api-key", # From your dashboard
base_url="https://api.subconscious.dev/v1",
)
```
```python Python (Anthropic) theme={null}
from anthropic import Anthropic
client = Anthropic(
auth_token="your-api-key", # From your dashboard
base_url="https://api.subconscious.dev",
)
```
## Billing
Usage is billed per token. See [Pricing](https://www.subconscious.dev/pricing) for current rates.
* Credits are added from your dashboard
* Usage is deducted automatically per request
* Auto-pay is available for uninterrupted service
* Usage dashboards show real-time consumption
# Dedicated Endpoints
Source: https://docs.subconscious.dev/ways-to-use/dedicated
Reserved compute with guaranteed capacity
Dedicated endpoints give your organization reserved inference capacity with guaranteed throughput, custom rate limits, and stricter isolation.
Reserved capacity with no noisy neighbors
Higher token and request limits tailored to your workload
Dedicated compute for regulated environments
## What You Get
* **Reserved GPU capacity** so your requests always have compute available, with no queuing behind other customers.
* **Custom rate limits** with token-per-minute and request-per-minute limits configured to your needs.
* **SLA guarantees** including uptime and latency commitments for production workloads.
* **Priority support** with direct access to our engineering team.
## How It Works
Dedicated endpoints use the same OpenAI- and Anthropic-compatible API, so your code doesn't change. We provision isolated compute for your organization and configure your API keys to route to dedicated infrastructure.
```python Python (OpenAI) theme={null}
from openai import OpenAI
# Same API, same SDK, just higher limits and dedicated compute
client = OpenAI(
api_key="your-api-key",
base_url="https://api.subconscious.dev/v1",
)
```
```python Python (Anthropic) theme={null}
from anthropic import Anthropic
# Same API, same SDK, just higher limits and dedicated compute
client = Anthropic(
auth_token="your-api-key",
base_url="https://api.subconscious.dev",
)
```
## Pricing
Dedicated endpoint pricing is based on reserved capacity and varies by workload. [Fill out our contact form](https://www.subconscious.dev/product/coding-agents/contact) for a quote.
Fill out our contact form to discuss your dedicated endpoint needs.
# Local Devices
Source: https://docs.subconscious.dev/ways-to-use/local-devices
Run inference on workstations, laptops, and mobile devices
Deploy Subconscious models directly on local hardware for offline inference, ultra-low latency, and complete data privacy. All processing happens on-device, so no data ever leaves the machine.
Inference runs directly on device with no network round-trip
Works without internet connectivity
Data never leaves the device
## Supported Devices
### Workstations & Servers
* Desktop machines with dedicated GPUs
* Development workstations
* Edge servers and on-site hardware
### Laptops
* GPU-equipped laptops (NVIDIA, Apple Silicon)
* Development and field use
### Mobile Devices
* iOS and Android deployment
* On-device inference for mobile applications
## Use Cases
* **Sensitive data processing** for healthcare, legal, and financial documents that cannot leave the device
* **Field operations** where deployments don't have reliable internet access
* **Development** for local testing and iteration without API costs
* **Edge computing** for real-time inference at the point of data collection
## How It Works
We provide optimized model packages for different hardware targets. The local runtime exposes the same OpenAI- and Anthropic-compatible API on localhost, so your application code works unchanged:
```python Python (OpenAI) theme={null}
from openai import OpenAI
# Same API, running locally
client = OpenAI(
api_key="local",
base_url="http://localhost:8080/v1",
)
response = client.chat.completions.create(
model="subconscious/tim-qwen3.6-27b",
messages=[{"role": "user", "content": "Hello!"}],
)
```
```python Python (Anthropic) theme={null}
from anthropic import Anthropic
# Same API, running locally
client = Anthropic(
auth_token="local",
base_url="http://localhost:8080",
)
message = client.messages.create(
model="subconscious/tim-qwen3.6-27b",
max_tokens=1024,
messages=[{"role": "user", "content": "Hello!"}],
)
```
Fill out our contact form to discuss local deployment for your organization.
# On-Prem & Customer Cloud
Source: https://docs.subconscious.dev/ways-to-use/on-prem
Deploy Subconscious inference in your own infrastructure
For organizations with strict data residency, compliance, or network isolation requirements, we deploy Subconscious inference directly into your infrastructure, whether that's your own cloud (AWS, GCP, Azure) or an on-premises data center.
The full on-prem guide now lives in the dedicated On-Prem section. Start with [On-Prem overview](/on-prem/overview) for the Subconscious Inference System overview, component guides, deployment methods, integration journey, and compliance details.
All data stays within your network boundary, and nothing leaves your environment
Model serving, API gateway, and monitoring deployed in your infrastructure
Our team works directly with your infrastructure and security teams
Learn how the API Gateway, Inference Runtime, Distribution Platform, Deployments, Integration journey, and Trust center fit together for customer-hosted deployments.
## Deployment Options
### Customer Cloud
We deploy into your existing cloud environment:
* **AWS**: EKS, EC2, or SageMaker-based deployment
* **GCP**: GKE or Compute Engine deployment
* **Azure**: AKS or VM-based deployment
* **BaseTen**: Deploy dedicated GPUs
* **Together AI**: Deploy our inference system on Dedicated GPUs
* **Crusoe**: Deploy our inference system on Dedicated GPUs
Your data never leaves your cloud account. We provide the container images, configuration, and deployment automation.
### On-Premises
For air-gapped or fully on-premises environments:
* Deploy to bare metal or virtualized GPU infrastructure
* No internet connectivity required after initial setup
* Full control over network policies and access
## What's Included
* **Model weights and serving infrastructure** optimized for your hardware
* **OpenAI- and Anthropic-compatible API endpoint** running inside your network with the same API you already use
* **Monitoring and observability** including health checks, metrics, and logging
* **Ongoing support** with updates, patches, and direct engineering support
## Requirements
* GPU infrastructure (specific requirements depend on deployment size)
* Container orchestration (Kubernetes preferred)
* Network access for initial setup and updates (can be air-gapped post-setup)
Fill out our contact form to scope your on-prem deployment.
# Privacy & Security
Source: https://docs.subconscious.dev/ways-to-use/privacy
How we handle and protect your data
Security is foundational to how we build Subconscious. We treat your prompts, completions, and workflows with the highest level of care.
## Your Query Data Is Not Retained
We do not store, log, or retain the content of your inference requests or responses. Prompts and completions are processed transiently to serve your request and are never persisted.
The only usage data we retain is the token-accounting metadata required to operate and bill the service: your input tokens, your output tokens, and your cached tokens.
This metadata records *how much* you used, never *what* you sent or received. We have no record of the substance of your queries, and your data is never used to train or improve models.
## Security
All API traffic is encrypted using industry-standard TLS.
API keys are issued per organization, required for every request, and revocable at any time.
Internal access to production systems is restricted to authorized personnel.
We build on a small set of established, security-conscious infrastructure providers.
## Privacy
We collect only the personal data needed to provide and support the service, such as account identity, contact details, and basic technical and usage data. You retain rights over your personal data, including access, correction, and erasure. For full details, see our [Privacy Policy](https://www.subconscious.dev/privacy) and [Security](https://www.subconscious.dev/security) pages.
## Enterprise
For organizations with additional requirements, we offer:
* **Dedicated endpoints** with isolated compute and no shared infrastructure. See [Dedicated Endpoints](/ways-to-use/dedicated).
* **On-prem deployment** to run inference in your own cloud or data center. See [On-Prem](/ways-to-use/on-prem) or the detailed [On-Prem guide](/on-prem/overview).
* **Custom security reviews** with your security team.
## Contact
Found a vulnerability or have a question about our practices? Reach us at [security@subconscious.dev](mailto:security@subconscious.dev) (security) or [privacy@subconscious.dev](mailto:privacy@subconscious.dev) (privacy).