Skip to main content

Managing LLM Configurations

Before starting a chat session or constructing an AI agent, you'll first need to configure an LLM (Large Language Model).

Nexus supports multiple providers like OpenAI, Google, Deepseek, io.net or self-hosted through Ollama. Each configuration includes the model, API key, optional base URL (required for self-hosted models), and parameters like temperature and top-p.

Create a new LLM Configuration​

POST /llm_config

ℹ️ Note: When using Ollama (self-hosted), you must provide a valid base_url.

Payload example:

{
  "provider": "openai",
  "model": "gpt-4o",
  "api_key": "[api_key]",
  "base_url": null,
  "name": "My Model",
  "description": "My Model Description",
  "params": {
    "temperature": 0.1,
    "top_p": 0.1
  }
}

Response example:

{
  "status": "ok",
  "data": {
    "id": 2,
    "user_id": "[user_id]",
    "provider": "openai",
    "model": "gpt-4o",
    "base_url": null,
    "api_key": "[api_key]",
    "params": {
      "temperature": 0.1,
      "top_p": 0.1
    },
    "name": "My Model",
    "description": "My Model Description",
    "created_at": "2025-05-20T09:05:33.773043"
  }
}

Get a specific LLM Configuration​

GET /llm_config/{config_id}

Returns the details of a specific configuration.

Response example:

{
  "status": "ok",
  "data": {
    "id": 2,
    "user_id": "[user_id]",
    "provider": "openai",
    "model": "gpt-4o",
    "base_url": null,
    "api_key": "[api_key]",
    "params": {
      "temperature": 0.1,
      "top_p": 0.1
    },
    "name": "My Model",
    "description": "My Model Description",
    "created_at": "2025-05-20T09:05:33.773043"
  }
}

List all LLM Configurations​

GET /llm_config

Lists all the LLM configurations for the authenticated user.

Response example:

{
  "status": "ok",
  "data": {
    "models": [
      {
        "id": 2,
        "user_id": "user_id",
        "provider": "openai",
        "model": "gpt-4o",
        "base_url": null,
        "api_key": "[api_key]",
        "params": {
          "temperature": 0.1,
          "top_p": 0.1
        },
        "name": "My OpenAI Model",
        "description": "My OpenAI Model Description",
        "created_at": "2025-05-21T21:47:56.722169"
      },
      {
        "id": 3,
        "user_id": "user_id",
        "provider": "google",
        "model": "gemini-2.5-pro-preview-05-06",
        "base_url": null,
        "api_key": "[api_key]",
        "params": {
          "temperature": 0.1,
          "top_p": 0.1
        },
        "name": "My Google Model",
        "description": "My Google Model Description",
        "created_at": "2025-05-22T11:01:32.433674"
      }
    ]
  }
}

Update an LLM Configuration​

PATCH /llm_config/{config_id}

You only need to send the fields you want to update.

Full payload example:

{
  "provider": "openai",
  "model": "gpt-4",
  "api_key": "sk-NEWKEY0987654321",
  "base_url": null,
  "params": {
    "temperature": 0.7,
    "top_p": 0.9
  },
  "name": "My Model",
  "description": "My Model Description"
}

Partial update example:

{
  "params": {
    "temperature": 0.7,
    "top_p": 0.9
  }
}

Response example:

{
  "status": "ok",
  "data": {
    "id": 2,
    "user_id": "[user_id]",
    "provider": "openai",
    "model": "gpt-4o",
    "base_url": null,
    "api_key": "[api_key]",
    "params": {
      "temperature": 0.7,
      "top_p": 0.9
    },
    "name": "My Model",
    "description": "My Model Description",
    "created_at": "2025-05-20T09:05:33.773043"
  }
}

Delete an LLM Configuration​

DELETE /llm_config/{config_id}

Deletes the configuration.

Response example:

{
  "status": "ok",
  "data": {}
}

Testing an LLM Configuration​

POST /chat/test

Test an LLM configuration without saving it immediately. Will give human friendly errors when there's a configuration error, or a friendly LLM response when the configuration is successful.

Payload example:

{
    "provider": "openai",
    "model": "gpt-4o",
    "api_key": "",
    "base_url": "",
    "params": {
        "top_p": "0.1",
        "temperature": "0.1"
    },
    "name": "My First LLM!",
    "description": "Just to test the platform"
}

Error response example:

{
    "status": "error",
    "message": "No API key provided. Please set your OpenAI API key."
}

Success response example:

{
    "status": "ok",
    "data": {
        "response": "Hello! I'm \"My First LLM,\" and I'm here to help you test the platform. It looks like everything is set up successfully. If you have any more tests or questions, feel free to reach out. Have a great day!"
    }
}