Skip to content

Debug Model Loading Issues

This guide helps you troubleshoot common errors when loading models with inference-models.

Common Error Types

Missing Dependencies

Error: 

MissingDependencyError: Required dependency 'onnxruntime' is not installed

Cause: The backend required for the model is not installed.

Solution: 

# Install the missing backend
pip install "inference-models[onnx-cpu]"

# Or for GPU
pip install "inference-models[onnx-cu12]"

Check what's installed: 

from inference_models import AutoModel

AutoModel.describe_compute_environment()

No Model Packages Available

Error: 

NoModelPackagesAvailableError: Could not find any model package announced by weights provider

Cause: No model package is registered for the requested model.

Solutions:

  1. Verify model ID: 

    # Correct format for Roboflow models
model = AutoModel.from_pretrained("project-id/version", api_key="your_key")

# Correct format for pre-trained models
model = AutoModel.from_pretrained("yolov8n-640")

  2. Check if model is ready:

  3. For Roboflow models, ensure training is complete

  4. Check the Roboflow dashboard for model status

  5. Install required backends: 

    # Install multiple backends for better compatibility
pip install "inference-models[torch-cpu,onnx-cpu]"

Unauthorized Access

Error: 

UnauthorizedModelAccessError: Access to model denied

Cause: Missing or invalid API key for private Roboflow models.

Solution: 

from inference_models import AutoModel

# Provide API key
model = AutoModel.from_pretrained(
    "your-project/1",
    api_key="your_roboflow_api_key"
)

Get your API key: See how to find your Roboflow API key

Corrupted Model Package

Error: 

CorruptedModelPackageError: Model package is corrupted or incomplete

Cause: Downloaded model files are corrupted or incomplete, or bug in the code.

Solution:

  1. Clear the cache: 

    import shutil
from inference_models.configuration import INFERENCE_HOME

# Remove cache directory
cache_dir = INFERENCE_HOME / "cache"
if cache_dir.exists():
    shutil.rmtree(cache_dir)

  2. Re-download the model: 

    from inference_models import AutoModel

model = AutoModel.from_pretrained("yolov8n-640")

  3. If the issue persists, report it:

After verifying the cache is cleared and the problem continues, open a GitHub issue with details about the model and error.

Backend Compatibility Issues

Error: 

Could not load any of model package candidate

Cause: No compatible backend for your environment.

Solution:

  1. Check your environment: 

    from inference_models import AutoModel

AutoModel.describe_compute_environment()

  2. Install compatible backends: 

    # For CPU
pip install "inference-models[torch-cpu,onnx-cpu]"

# For NVIDIA GPU
pip install "inference-models[torch-cu128,onnx-cu12,trt10]" tensorrt

  3. Force a specific backend: 

    # Try different backends
model = AutoModel.from_pretrained("yolov8n-640", backend="torch")
# or
model = AutoModel.from_pretrained("yolov8n-640", backend="onnx")

Common Problems

"Model not found" on Roboflow Platform

Problem: Model ID is correct but still getting errors.

Checklist:

  • ✅ Model training is complete
  • ✅ Model version exists (check dashboard)
  • ✅ API key is provided and valid
  • ✅ Project ID format is correct: project-id/version

Example: 

# Correct format
model = AutoModel.from_pretrained(
    "my-project-abc123/2",  # project-id/version
    api_key="your_api_key"
)

CUDA Out of Memory

Error: 

RuntimeError: CUDA out of memory

Solutions:

  1. Use smaller batch size: 

    from inference_models.developer_tools import generate_batch_chunks

# For list of images
batch_size = 4
for i in range(0, len(images), batch_size):
    batch = images[i:i + batch_size]
    predictions = model(batch)

# For 4D tensor (batch, channels, height, width)
import torch
images_tensor = torch.stack([...])  # Your images as tensor
for chunk, padding_size in generate_batch_chunks(images_tensor, chunk_size=4):
    predictions = model(chunk)
    # Remove padding from results if needed
    if padding_size > 0:
        predictions = predictions[:-padding_size]

  2. Use CPU instead: 

    model = AutoModel.from_pretrained("yolov8n-640", device="cpu")

  3. Clear CUDA cache: 

    import torch

torch.cuda.empty_cache()

Advanced Debugging

Inspect Model Package

from inference_models.developer_tools import get_model_package_contents

# Get model package details
contents = get_model_package_contents("path/to/model/package")

print(f"Config: {contents.config}")
print(f"Files: {contents.files}")

Test Specific Backend

from inference_models import AutoModel

# Test each backend individually
backends = ["torch", "onnx", "trt"]

for backend in backends:
    try:
        model = AutoModel.from_pretrained(
            "yolov8n-640",
            backend=backend,
            verbose=True
        )
        print(f"✅ {backend} backend works")
    except Exception as e:
        print(f"❌ {backend} backend failed: {e}")

Check Model Metadata

from inference_models.developer_tools import get_model_from_provider

# Get model metadata without loading
metadata = get_model_from_provider(
    model_id="yolov8n-640",
    api_key=None  # Not needed for public models
)

print(f"Architecture: {metadata.model_architecture}")
print(f"Task: {metadata.task_type}")
print(f"Available packages: {len(metadata.model_packages)}")

Getting Help

If you're still experiencing issues:

  1. Check GitHub Issues: roboflow/inference/issues

  2. Provide debug information: 

    from inference_models import AutoModel
import sys

print(f"Python version: {sys.version}")
print(f"Platform: {sys.platform}")

AutoModel.describe_compute_environment()

  3. Include error traceback: 

    import traceback

try:
    model = AutoModel.from_pretrained("your-model-id")
except Exception as e:
    print(traceback.format_exc())

  4. Contact Roboflow Support: For Roboflow-hosted models, contact support@roboflow.com

Next Steps