pandera.Check

class pandera.Check(check_fn, groups=None, groupby=None, ignore_na=True, element_wise=False, name=None, error=None, raise_warning=False, n_failure_cases=10, **check_kwargs)[source]

Check a pandas Series or DataFrame for certain properties.

Apply a validation function to each element, Series, or DataFrame.

Parameters
  • check_fn (Callable) –

    A function to check pandas data structure. For Column or SeriesSchema checks, if element_wise is True, this function should have the signature: Callable[[pd.Series], Union[pd.Series, bool]], where the output series is a boolean vector.

    If element_wise is False, this function should have the signature: Callable[[Any], bool], where Any is an element in the column.

    For DataFrameSchema checks, if element_wise=True, fn should have the signature: Callable[[pd.DataFrame], Union[pd.DataFrame, pd.Series, bool]], where the output dataframe or series contains booleans.

    If element_wise is True, fn is applied to each row in the dataframe with the signature Callable[[pd.Series], bool] where the series input is a row in the dataframe.

  • groups (Union[str, List[str], None]) – The dict input to the fn callable will be constrained to the groups specified by groups.

  • groupby (Union[str, List[str], Callable, None]) –

    If a string or list of strings is provided, these columns are used to group the Column series. If a callable is passed, the expected signature is: Callable[ [pd.DataFrame], pd.core.groupby.DataFrameGroupBy]

    The the case of Column checks, this function has access to the entire dataframe, but Column.name is selected from this DataFrameGroupby object so that a SeriesGroupBy object is passed into check_fn.

    Specifying the groupby argument changes the check_fn signature to:

    Callable[[Dict[Union[str, Tuple[str]], pd.Series]], Union[bool, pd.Series]] # noqa

    where the input is a dictionary mapping keys to subsets of the column/dataframe.

  • ignore_na (bool) – If True, drops null values on the checked series or dataframe before passing into the check_fn. For dataframes, drops rows with any null value. New in version 0.4.0

  • element_wise (bool) – Whether or not to apply validator in an element-wise fashion. If bool, assumes that all checks should be applied to the column element-wise. If list, should be the same number of elements as checks.

  • name (Optional[str]) – optional name for the check.

  • error (Optional[str]) – custom error message if series fails validation check.

  • raise_warning (bool) – if True, raise a UserWarning and do not throw exception instead of raising a SchemaError for a specific check. This option should be used carefully in cases where a failing check is informational and shouldn’t stop execution of the program.

  • n_failure_cases (Optional[int]) – report the top n failure cases. If None, then report all failure cases.

  • check_kwargs – key-word arguments to pass into check_fn

Example

>>> import pandas as pd
>>> import pandera as pa
>>>
>>>
>>> # column checks are vectorized by default
>>> check_positive = pa.Check(lambda s: s > 0)
>>>
>>> # define an element-wise check
>>> check_even = pa.Check(lambda x: x % 2 == 0, element_wise=True)
>>>
>>> # specify assertions across categorical variables using `groupby`,
>>> # for example, make sure the mean measure for group "A" is always
>>> # larger than the mean measure for group "B"
>>> check_by_group = pa.Check(
...     lambda measures: measures["A"].mean() > measures["B"].mean(),
...     groupby=["group"],
... )
>>>
>>> # define a wide DataFrame-level check
>>> check_dataframe = pa.Check(
...     lambda df: df["measure_1"] > df["measure_2"])
>>>
>>> measure_checks = [check_positive, check_even, check_by_group]
>>>
>>> schema = pa.DataFrameSchema(
...     columns={
...         "measure_1": pa.Column(pa.Int, checks=measure_checks),
...         "measure_2": pa.Column(pa.Int, checks=measure_checks),
...         "group": pa.Column(pa.String),
...     },
...     checks=check_dataframe
... )
>>>
>>> df = pd.DataFrame({
...     "measure_1": [10, 12, 14, 16],
...     "measure_2": [2, 4, 6, 8],
...     "group": ["B", "B", "A", "A"]
... })
>>>
>>> schema.validate(df)[["measure_1", "measure_2", "group"]]
    measure_1  measure_2 group
0         10          2     B
1         12          4     B
2         14          6     A
3         16          8     A

See here for more usage details.

Attributes

statistics

Get check statistics.

Methods

eq

Ensure all elements of a series equal a certain value.

equal_to

Ensure all elements of a series equal a certain value.

ge

Ensure all values are greater or equal a certain value.

greater_than

Ensure values of a series are strictly greater than a minimum value.

greater_than_or_equal_to

Ensure all values are greater or equal a certain value.

gt

Ensure values of a series are strictly greater than a minimum value.

in_range

Ensure all values of a series are within an interval.

isin

Ensure only allowed values occur within a series.

le

Ensure values are less than or equal to a maximum value.

less_than

Ensure values of a series are strictly below a maximum value.

less_than_or_equal_to

Ensure values are less than or equal to a maximum value.

lt

Ensure values of a series are strictly below a maximum value.

ne

Ensure no elements of a series equals a certain value.

not_equal_to

Ensure no elements of a series equals a certain value.

notin

Ensure some defined values don’t occur within a series.

str_contains

Ensure that a pattern can be found within each row.

str_endswith

Ensure that all values end with a certain string.

str_length

Ensure that the length of strings is within a specified range.

str_matches

Ensure that string values match a regular expression.

str_startswith

Ensure that all values start with a certain string.

__call__

Validate pandas DataFrame or Series.