TensorDictSchema¶
Use TensorDictSchema to validate
TensorDict objects.
Define a schema¶
Define a dictionary-based schema with Tensor
components:
import torch
from tensordict import TensorDict
import pandera.tensordict as pa
schema = pa.TensorDictSchema(
keys={
"observation": pa.Tensor(dtype=torch.float32, shape=(None, 10)),
"action": pa.Tensor(dtype=torch.float32, shape=(None, 5)),
},
batch_size=(32,),
)
keys: A dictionary mapping key names toTensorobjectsbatch_size: Required batch dimension sizes. UseNonefor flexible dimensions.
Nested (tuple) keys¶
TensorDicts can nest (TorchRL uses keys like ("next", "observation")).
Schema keys may be plain strings or nested tuples:
nested_schema = pa.TensorDictSchema(
keys={
("next", "observation"): pa.Tensor(
dtype=torch.float32, shape=(None, 10)
),
("next", "reward"): pa.Tensor(dtype=torch.float32, shape=(None,)),
"action": pa.Tensor(dtype=torch.int64, shape=(None,)),
},
batch_size=(4,),
)
nested_td = TensorDict(
{
"action": torch.randint(0, 4, (4,)),
"next": TensorDict(
{
"observation": torch.randn(4, 10),
"reward": torch.randn(4),
},
batch_size=[4],
),
},
batch_size=[4],
)
nested_schema.validate(nested_td)
TensorDict batch_size=[4]
- action: i64[]
next: 2 entries
- observation: f32[10]
- reward: f32[]
Validate a TensorDict¶
td = TensorDict(
{"observation": torch.randn(32, 10), "action": torch.randn(32, 5)},
batch_size=[32],
)
validated = schema.validate(td)
Lazy validation¶
Set lazy=True to collect all errors instead of fail-fast. Note that TensorDict
validates batch dimensions on creation, so we need to use matching dimensions:
td_wrong_dtype = TensorDict(
{"observation": torch.randn(32, 10), "action": torch.randint(0, 5, (32,))},
batch_size=[32],
)
try:
schema.validate(td_wrong_dtype, lazy=True)
except pa.errors.SchemaErrors as exc:
print(f"Total errors: {len(exc.schema_errors)}")
for err in exc.schema_errors:
print(f" - {str(err)}")
Total errors: 2
- Key 'action': expected dtype torch.float32, got torch.int64
- Key 'action': expected shape (None, 5), got (32,)
Tensor component¶
The Tensor component validates individual tensor keys.
Parameters¶
dtype: Torch dtype (e.g.,torch.float32,torch.int64)shape: Expected shape tuple. UseNonefor flexible dimensions.checks: Optional list ofCheckinstances
Example¶
from pandera import Check
schema = pa.TensorDictSchema(
keys={
"values": pa.Tensor(
dtype=torch.float32,
shape=(None,),
checks=[Check.greater_than(0.0), Check.less_than(1.0)],
),
},
batch_size=(10,),
)
Type Coercion¶
Set coerce=True to automatically convert tensor dtypes during validation:
schema = pa.TensorDictSchema(
keys={
"observation": pa.Tensor(dtype=torch.float32, shape=(None, 10)),
"action": pa.Tensor(dtype=torch.int64),
},
batch_size=(32,),
coerce=True,
)
# Input has wrong dtypes (float64, int32)
td = TensorDict(
{
"observation": torch.randn(32, 10).to(torch.float64),
"action": torch.randint(0, 5, (32,)).to(torch.int32),
},
batch_size=[32],
)
# Dtypes are automatically coerced to float32 and int64
validated = schema.validate(td)
assert validated["observation"].dtype == torch.float32
assert validated["action"].dtype == torch.int64
Type coercion is applied before validation checks, so any dtype or shape constraints will be evaluated on the coerced data.
Schema Inference¶
Automatically infer a schema from existing data using pa.infer_schema():
import torch
from tensordict import TensorDict
import pandera.tensordict as pa
# Existing TensorDict with sample data
sample_td = TensorDict({
"observation": torch.randn(100, 64),
"action": torch.randint(0, 4, (100,)),
}, batch_size=[100])
# Infer schema from data
inferred_schema = pa.infer_schema(sample_td)
See Schema Inference for more details on schema inference.
Serialization¶
Save and load schemas using YAML or JSON:
import tempfile
from pathlib import Path
import torch
import pandera.tensordict as pa
schema = pa.TensorDictSchema(
keys={
"observation": pa.Tensor(dtype=torch.float32, shape=(None, 10)),
},
batch_size=(32,),
)
with tempfile.TemporaryDirectory() as tmpdir:
# Save to YAML
yaml_path = Path(tmpdir) / "schema.yaml"
pa.to_yaml(schema, yaml_path)
# Load from YAML
loaded_schema = pa.from_yaml(yaml_path)
See Serialization and Deserialization for more details on serialization.
See also¶
TensorDictModel — class-based schema definition
Checks and Lazy Validation — checks for value validation
Error Reporting — error handling
Schema Inference — infer schemas from data
Serialization and Deserialization — save/load schemas