Skip to main content

Models

Tower Models provide easy access to Large Language Models (LLMs) through a unified interface. They abstract away the complexity of different inference providers and model APIs, allowing you to work with local and remote models seamlessly.

Overview​

Tower offers two main components for working with language models:

  • The Llm class: A wrapper around language models that provides methods for text generation and chat completions
  • The llms helper function: A convenient way to create and configure Llm instances

Instantiating Models​

To create a Llm instance, you need to:

  1. Specify the model name (can be a model family name or specific model identifier)
  2. Optionally set the maximum tokens for responses
  3. Use the llms() helper function

Here's a basic example:

import tower

# Create a language model instance
llm = tower.llms("llama3.2", max_tokens=1000)

The returned llm object is of the Tower Llm class, which provides a unified interface for working with different language model providers. Currently, Tower supports Ollama for local inference and Hugging Face Hub for remote inference.

Model Name Resolution​

Tower automatically resolves model names based on available inference providers. You can specify either:

  • Model family names (e.g., "llama3.2", "gemma3.2", "deepseek-r1")
  • Specific model identifiers (e.g., "deepseek-r1:14b", "deepseek-ai/DeepSeek-R1-0528")

The system will automatically select the appropriate inference provider and resolve the exact model name. For a list of supported model family names, see the Tower LLM module.

Model Operations​

Model Instantiation​

The llms helper function creates an Llm instance with these parameters:

Configuration​

  • model_name (str): The name of the language model to use
  • max_tokens (int): Maximum number of tokens to generate in responses (defaults to 1000)

Llm Methods​

Once you have a language model instance, you can perform these operations:

Chat Completions​

  • complete_chat() - Sends a list of messages and returns the generated response (OpenAI Chat Completions API format)

Simple Prompts​

  • prompt() - Sends a single prompt string and returns the generated response (legacy OpenAI Completions API format)

Usage Examples​

Chat-based Interactions​

import tower

# Create a language model
llm = tower.llms("llama3.2", max_tokens=500)

# Use for chat completions
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello, how are you?"}
]
response = llm.complete_chat(messages)
print(response)

Simple Prompt Interactions​

import tower

# Create a language model
llm = tower.llms("deepseek-r1", max_tokens=1000)

# Use for simple prompts
response = llm.prompt("What is the capital of France?")
print(response)

Learning More​

For a more in-depth review of working with models in Tower, see the Working with Models guide.