TensorDictSchema

Use TensorDictSchema to validate TensorDict objects.

Define a schema

Define a dictionary-based schema with Tensor components:

import torch
from tensordict import TensorDict
import pandera.tensordict as pa

schema = pa.TensorDictSchema(
    keys={
        "observation": pa.Tensor(dtype=torch.float32, shape=(None, 10)),
        "action": pa.Tensor(dtype=torch.float32, shape=(None, 5)),
    },
    batch_size=(32,),
)
  • keys: A dictionary mapping key names to Tensor objects

  • batch_size: Required batch dimension sizes. Use None for flexible dimensions.

Nested (tuple) keys

TensorDicts can nest (TorchRL uses keys like ("next", "observation")). Schema keys may be plain strings or nested tuples:

nested_schema = pa.TensorDictSchema(
    keys={
        ("next", "observation"): pa.Tensor(
            dtype=torch.float32, shape=(None, 10)
        ),
        ("next", "reward"): pa.Tensor(dtype=torch.float32, shape=(None,)),
        "action": pa.Tensor(dtype=torch.int64, shape=(None,)),
    },
    batch_size=(4,),
)

nested_td = TensorDict(
    {
        "action": torch.randint(0, 4, (4,)),
        "next": TensorDict(
            {
                "observation": torch.randn(4, 10),
                "reward": torch.randn(4),
            },
            batch_size=[4],
        ),
    },
    batch_size=[4],
)
nested_schema.validate(nested_td)
TensorDict batch_size=[4]
  • action: i64[]
  • next: 2 entries
    • observation: f32[10]
    • reward: f32[]

Validate a TensorDict

td = TensorDict(
    {"observation": torch.randn(32, 10), "action": torch.randn(32, 5)},
    batch_size=[32],
)
validated = schema.validate(td)

Lazy validation

Set lazy=True to collect all errors instead of fail-fast. Note that TensorDict validates batch dimensions on creation, so we need to use matching dimensions:

td_wrong_dtype = TensorDict(
    {"observation": torch.randn(32, 10), "action": torch.randint(0, 5, (32,))},
    batch_size=[32],
)

try:
    schema.validate(td_wrong_dtype, lazy=True)
except pa.errors.SchemaErrors as exc:
    print(f"Total errors: {len(exc.schema_errors)}")
    for err in exc.schema_errors:
        print(f"  - {str(err)}")
Total errors: 2
  - Key 'action': expected dtype torch.float32, got torch.int64
  - Key 'action': expected shape (None, 5), got (32,)

Tensor component

The Tensor component validates individual tensor keys.

Parameters

  • dtype: Torch dtype (e.g., torch.float32, torch.int64)

  • shape: Expected shape tuple. Use None for flexible dimensions.

  • checks: Optional list of Check instances

Example

from pandera import Check

schema = pa.TensorDictSchema(
    keys={
        "values": pa.Tensor(
            dtype=torch.float32,
            shape=(None,),
            checks=[Check.greater_than(0.0), Check.less_than(1.0)],
        ),
    },
    batch_size=(10,),
)

Type Coercion

Set coerce=True to automatically convert tensor dtypes during validation:

schema = pa.TensorDictSchema(
    keys={
        "observation": pa.Tensor(dtype=torch.float32, shape=(None, 10)),
        "action": pa.Tensor(dtype=torch.int64),
    },
    batch_size=(32,),
    coerce=True,
)

# Input has wrong dtypes (float64, int32)
td = TensorDict(
    {
        "observation": torch.randn(32, 10).to(torch.float64),
        "action": torch.randint(0, 5, (32,)).to(torch.int32),
    },
    batch_size=[32],
)

# Dtypes are automatically coerced to float32 and int64
validated = schema.validate(td)
assert validated["observation"].dtype == torch.float32
assert validated["action"].dtype == torch.int64

Type coercion is applied before validation checks, so any dtype or shape constraints will be evaluated on the coerced data.

Schema Inference

Automatically infer a schema from existing data using pa.infer_schema():

import torch
from tensordict import TensorDict
import pandera.tensordict as pa

# Existing TensorDict with sample data
sample_td = TensorDict({
    "observation": torch.randn(100, 64),
    "action": torch.randint(0, 4, (100,)),
}, batch_size=[100])

# Infer schema from data
inferred_schema = pa.infer_schema(sample_td)

See Schema Inference for more details on schema inference.

Serialization

Save and load schemas using YAML or JSON:

import tempfile
from pathlib import Path
import torch
import pandera.tensordict as pa

schema = pa.TensorDictSchema(
    keys={
        "observation": pa.Tensor(dtype=torch.float32, shape=(None, 10)),
    },
    batch_size=(32,),
)

with tempfile.TemporaryDirectory() as tmpdir:
    # Save to YAML
    yaml_path = Path(tmpdir) / "schema.yaml"
    pa.to_yaml(schema, yaml_path)

    # Load from YAML
    loaded_schema = pa.from_yaml(yaml_path)

See Serialization and Deserialization for more details on serialization.

See also