Modal

Summary: Serverless GPU platform for ML workloads. Spin up A100/H100 in seconds, Python-native decorators, pay-per-second billing. Ideal for training, fine-tuning, batch inference, and episodic compute.

Details

For Tools

  • Category: Cloud Infrastructure / Serverless GPU
  • Key Features:
    • GPU Access: A100 (40/80GB), H100, L4, T4
    • Cold Start: ~1-3 seconds (container pre-warm)
    • Billing: Per-second, no minimums
    • Interface: Python decorators (@stub.function(gpu="A100"))
    • Storage: Network filesystem (Modal Volumes)
    • Secrets: Secure env var management
    • Web Endpoints: @stub.web_endpoint() for APIs
  • Website: modal.com
  • Pricing: ~`1.10/hr (A100 40GB), ~`3.50/hr (A100 80GB), ~$4.50/hr (H100)

Significance for Your Workflow

Consumer-GPU project enabler:

  • Your RTX 4090 (24GB) limits batch size for ViT-B/16 + DSA
  • Modal A100 80GB → 8-16× larger batches, multi-GPU (FSDP/DS)
  • Cost estimate: 100 GPU-hrs on A100 80GB ≈ $350 — affordable for paper experiments

Use cases:

Task Modal Setup Est. Cost
ViT-DSA CIFAR-100 sanity 1×A100 40GB, 4 hrs ~$4
ViT-DSA ImageNet-1k 4×A100 80GB, 24 hrs ~$336
LoRA/QLoRA 7B fine-tune 1×A100 80GB, 8 hrs ~$28
Batch inference (10k images) CPU + 1×L4, 2 hrs ~$0.50

Integration:

import modal

stub = modal.Stub("vit-dsa")
image = modal.Image.debian_slim().pip_install("torch", "timm", "flash-attn")

@stub.function(gpu="A100:4", image=image, timeout=86400, volumes={"/data": modal.Volume.from_name("data")})
def train():
    # Your training code — runs on 4×A100 80GB
    ...

# Run: modal run train.py::train
# Or: modal deploy train.py (for web endpoint)

Related

Sources

  • web-modal-com: modal.com documentation