DeepInfra

Use open-source models hosted on DeepInfra with Agno agents. DeepInfra supports a wide range of models. See their library of models here. We recommend experimenting to find the best-suited model for your use-case. Here are some general recommendations:

deepseek-ai/DeepSeek-R1-Distill-Llama-70B model is good for reasoning.
meta-llama/Llama-2-70b-chat-hf model is good for basic use-cases.
meta-llama/Llama-3.3-70B-Instruct model is good for multi-step tasks.

DeepInfra has rate limits. See the rate limit docs.

Authentication

Set your DEEPINFRA_API_KEY environment variable. Get your key from here.

export DEEPINFRA_API_KEY=***

setx DEEPINFRA_API_KEY ***

Example

Use DeepInfra with your Agent:

from agno.agent import Agent
from agno.models.deepinfra import DeepInfra

agent = Agent(
    model=DeepInfra(id="meta-llama/Llama-2-70b-chat-hf"),
    markdown=True
)

# Print the response in the terminal
agent.print_response("Share a 2 sentence horror story.")

View more examples here.

Params

Parameter	Type	Default	Description
`id`	`str`	`"meta-llama/Llama-2-70b-chat-hf"`	The id of the DeepInfra model to use
`name`	`str`	`"DeepInfra"`	The name of the model
`provider`	`str`	`"DeepInfra"`	The provider of the model
`api_key`	`Optional[str]`	`None`	The API key for DeepInfra (defaults to DEEPINFRA_API_KEY env var)
`base_url`	`str`	`"https://api.deepinfra.com/v1/openai"`	The base URL for the DeepInfra API

DeepInfra also supports the parameters of OpenAI.

Get Started

Basics

Advanced

Production

Providers

Other

Additional Resources

Authentication

Example

Params

​Authentication

​Example

​Params

Authentication

Example

Params