causalml package

Submodules

causalml.inference.tree module

causalml.inference.meta module

class causalml.inference.meta.BaseDRLearner(learner=None, control_outcome_learner=None, treatment_outcome_learner=None, treatment_effect_learner=None, ate_alpha=0.05, control_name=0)[source]

Bases: causalml.inference.meta.base.BaseLearner

A parent class for DR-learner regressor classes.

A DR-learner estimates treatment effects with machine learning models.

Details of DR-learner are available at Kennedy (2020) (https://arxiv.org/abs/2004.14497).

estimate_ate(X, treatment, y, p=None, bootstrap_ci=False, n_bootstraps=1000, bootstrap_size=10000, seed=None)[source]

Estimate the Average Treatment Effect (ATE).

Parameters
  • X (np.matrix or np.array or pd.Dataframe) – a feature matrix

  • treatment (np.array or pd.Series) – a treatment vector

  • y (np.array or pd.Series) – an outcome vector

  • p (np.ndarray or pd.Series or dict, optional) – an array of propensity scores of float (0,1) in the single-treatment case; or, a dictionary of treatment groups that map to propensity vectors of float (0,1); if None will run ElasticNetPropensityModel() to generate the propensity scores.

  • bootstrap_ci (bool) – whether run bootstrap for confidence intervals

  • n_bootstraps (int) – number of bootstrap iterations

  • bootstrap_size (int) – number of samples per bootstrap

  • seed (int) – random seed for cross-fitting

Returns

The mean and confidence interval (LB, UB) of the ATE estimate.

fit(X, treatment, y, p=None, seed=None)[source]

Fit the inference model.

Parameters
  • X (np.matrix or np.array or pd.Dataframe) – a feature matrix

  • treatment (np.array or pd.Series) – a treatment vector

  • y (np.array or pd.Series) – an outcome vector

  • p (np.ndarray or pd.Series or dict, optional) – an array of propensity scores of float (0,1) in the single-treatment case; or, a dictionary of treatment groups that map to propensity vectors of float (0,1); if None will run ElasticNetPropensityModel() to generate the propensity scores.

  • seed (int) – random seed for cross-fitting

fit_predict(X, treatment, y, p=None, return_ci=False, n_bootstraps=1000, bootstrap_size=10000, return_components=False, verbose=True, seed=None)[source]

Fit the treatment effect and outcome models of the R learner and predict treatment effects.

Parameters
  • X (np.matrix or np.array or pd.Dataframe) – a feature matrix

  • treatment (np.array or pd.Series) – a treatment vector

  • y (np.array or pd.Series) – an outcome vector

  • p (np.ndarray or pd.Series or dict, optional) – an array of propensity scores of float (0,1) in the single-treatment case; or, a dictionary of treatment groups that map to propensity vectors of float (0,1); if None will run ElasticNetPropensityModel() to generate the propensity scores.

  • return_ci (bool) – whether to return confidence intervals

  • n_bootstraps (int) – number of bootstrap iterations

  • bootstrap_size (int) – number of samples per bootstrap

  • return_components (bool, optional) – whether to return outcome for treatment and control seperately

  • verbose (str) – whether to output progress logs

  • seed (int) – random seed for cross-fitting

Returns

Predictions of treatment effects. Output dim: [n_samples, n_treatment]

If return_ci, returns CATE [n_samples, n_treatment], LB [n_samples, n_treatment], UB [n_samples, n_treatment]

Return type

(numpy.ndarray)

predict(X, treatment=None, y=None, p=None, return_components=False, verbose=True)[source]

Predict treatment effects.

Parameters
  • X (np.matrix or np.array or pd.Dataframe) – a feature matrix

  • treatment (np.array or pd.Series, optional) – a treatment vector

  • y (np.array or pd.Series, optional) – an outcome vector

  • verbose (bool, optional) – whether to output progress logs

Returns

Predictions of treatment effects.

Return type

(numpy.ndarray)

class causalml.inference.meta.BaseDRRegressor(learner=None, control_outcome_learner=None, treatment_outcome_learner=None, treatment_effect_learner=None, ate_alpha=0.05, control_name=0)[source]

Bases: causalml.inference.meta.drlearner.BaseDRLearner

A parent class for DR-learner regressor classes.

class causalml.inference.meta.BaseRClassifier(outcome_learner=None, effect_learner=None, propensity_learner=LogisticRegressionCV(Cs=array([1.00230524, 2.15608891, 4.63802765, 9.97700064]), cv=StratifiedKFold(n_splits=4, random_state=42, shuffle=True), l1_ratios=array([0.001, 0.33366667, 0.66633333, 0.999]), penalty='elasticnet', random_state=42, solver='saga'), ate_alpha=0.05, control_name=0, n_fold=5, random_state=None)[source]

Bases: causalml.inference.meta.rlearner.BaseRLearner

A parent class for R-learner classifier classes.

fit(X, treatment, y, p=None, verbose=True)[source]

Fit the treatment effect and outcome models of the R learner.

Parameters
  • X (np.matrix or np.array or pd.Dataframe) – a feature matrix

  • treatment (np.array or pd.Series) – a treatment vector

  • y (np.array or pd.Series) – an outcome vector

  • p (np.ndarray or pd.Series or dict, optional) – an array of propensity scores of float (0,1) in the single-treatment case; or, a dictionary of treatment groups that map to propensity vectors of float (0,1); if None will run ElasticNetPropensityModel() to generate the propensity scores.

  • verbose (bool, optional) – whether to output progress logs

predict(X, p=None)[source]

Predict treatment effects.

Parameters

X (np.matrix or np.array or pd.Dataframe) – a feature matrix

Returns

Predictions of treatment effects.

Return type

(numpy.ndarray)

class causalml.inference.meta.BaseRLearner(learner=None, outcome_learner=None, effect_learner=None, propensity_learner=LogisticRegressionCV(Cs=array([1.00230524, 2.15608891, 4.63802765, 9.97700064]), cv=StratifiedKFold(n_splits=4, random_state=42, shuffle=True), l1_ratios=array([0.001, 0.33366667, 0.66633333, 0.999]), penalty='elasticnet', random_state=42, solver='saga'), ate_alpha=0.05, control_name=0, n_fold=5, random_state=None)[source]

Bases: causalml.inference.meta.base.BaseLearner

A parent class for R-learner classes.

An R-learner estimates treatment effects with two machine learning models and the propensity score.

Details of R-learner are available at Nie and Wager (2019) (https://arxiv.org/abs/1712.04912).

estimate_ate(X, treatment, y, p=None, bootstrap_ci=False, n_bootstraps=1000, bootstrap_size=10000)[source]

Estimate the Average Treatment Effect (ATE).

Parameters
  • X (np.matrix or np.array or pd.Dataframe) – a feature matrix

  • treatment (np.array or pd.Series) – a treatment vector

  • y (np.array or pd.Series) – an outcome vector

  • p (np.ndarray or pd.Series or dict, optional) – an array of propensity scores of float (0,1) in the single-treatment case; or, a dictionary of treatment groups that map to propensity vectors of float (0,1); if None will run ElasticNetPropensityModel() to generate the propensity scores.

  • bootstrap_ci (bool) – whether run bootstrap for confidence intervals

  • n_bootstraps (int) – number of bootstrap iterations

  • bootstrap_size (int) – number of samples per bootstrap

Returns

The mean and confidence interval (LB, UB) of the ATE estimate.

fit(X, treatment, y, p=None, verbose=True)[source]

Fit the treatment effect and outcome models of the R learner.

Parameters
  • X (np.matrix or np.array or pd.Dataframe) – a feature matrix

  • treatment (np.array or pd.Series) – a treatment vector

  • y (np.array or pd.Series) – an outcome vector

  • p (np.ndarray or pd.Series or dict, optional) – an array of propensity scores of float (0,1) in the single-treatment case; or, a dictionary of treatment groups that map to propensity vectors of float (0,1); if None will run ElasticNetPropensityModel() to generate the propensity scores.

  • verbose (bool, optional) – whether to output progress logs

fit_predict(X, treatment, y, p=None, return_ci=False, n_bootstraps=1000, bootstrap_size=10000, verbose=True)[source]

Fit the treatment effect and outcome models of the R learner and predict treatment effects.

Parameters
  • X (np.matrix or np.array or pd.Dataframe) – a feature matrix

  • treatment (np.array or pd.Series) – a treatment vector

  • y (np.array or pd.Series) – an outcome vector

  • p (np.ndarray or pd.Series or dict, optional) – an array of propensity scores of float (0,1) in the single-treatment case; or, a dictionary of treatment groups that map to propensity vectors of float (0,1); if None will run ElasticNetPropensityModel() to generate the propensity scores.

  • return_ci (bool) – whether to return confidence intervals

  • n_bootstraps (int) – number of bootstrap iterations

  • bootstrap_size (int) – number of samples per bootstrap

  • verbose (bool) – whether to output progress logs

Returns

Predictions of treatment effects. Output dim: [n_samples, n_treatment].

If return_ci, returns CATE [n_samples, n_treatment], LB [n_samples, n_treatment], UB [n_samples, n_treatment]

Return type

(numpy.ndarray)

predict(X, p=None)[source]

Predict treatment effects.

Parameters

X (np.matrix or np.array or pd.Dataframe) – a feature matrix

Returns

Predictions of treatment effects.

Return type

(numpy.ndarray)

class causalml.inference.meta.BaseRRegressor(learner=None, outcome_learner=None, effect_learner=None, propensity_learner=LogisticRegressionCV(Cs=array([1.00230524, 2.15608891, 4.63802765, 9.97700064]), cv=StratifiedKFold(n_splits=4, random_state=42, shuffle=True), l1_ratios=array([0.001, 0.33366667, 0.66633333, 0.999]), penalty='elasticnet', random_state=42, solver='saga'), ate_alpha=0.05, control_name=0, n_fold=5, random_state=None)[source]

Bases: causalml.inference.meta.rlearner.BaseRLearner

A parent class for R-learner regressor classes.

class causalml.inference.meta.BaseSClassifier(learner=None, ate_alpha=0.05, control_name=0)[source]

Bases: causalml.inference.meta.slearner.BaseSLearner

A parent class for S-learner classifier classes.

predict(X, treatment=None, y=None, p=None, return_components=False, verbose=True)[source]

Predict treatment effects. :param X: a feature matrix :type X: np.matrix or np.array or pd.Dataframe :param treatment: a treatment vector :type treatment: np.array or pd.Series, optional :param y: an outcome vector :type y: np.array or pd.Series, optional :param return_components: whether to return outcome for treatment and control seperately :type return_components: bool, optional :param verbose: whether to output progress logs :type verbose: bool, optional

Returns

Predictions of treatment effects.

Return type

(numpy.ndarray)

class causalml.inference.meta.BaseSLearner(learner=None, ate_alpha=0.05, control_name=0)[source]

Bases: causalml.inference.meta.base.BaseLearner

A parent class for S-learner classes. An S-learner estimates treatment effects with one machine learning model. Details of S-learner are available at Kunzel et al. (2018) (https://arxiv.org/abs/1706.03461).

estimate_ate(X, treatment, y, p=None, return_ci=False, bootstrap_ci=False, n_bootstraps=1000, bootstrap_size=10000)[source]

Estimate the Average Treatment Effect (ATE).

Parameters
  • X (np.matrix, np.array, or pd.Dataframe) – a feature matrix

  • treatment (np.array or pd.Series) – a treatment vector

  • y (np.array or pd.Series) – an outcome vector

  • return_ci (bool, optional) – whether to return confidence intervals

  • bootstrap_ci (bool) – whether to return confidence intervals

  • n_bootstraps (int) – number of bootstrap iterations

  • bootstrap_size (int) – number of samples per bootstrap

Returns

The mean and confidence interval (LB, UB) of the ATE estimate.

fit(X, treatment, y, p=None)[source]

Fit the inference model :param X: a feature matrix :type X: np.matrix, np.array, or pd.Dataframe :param treatment: a treatment vector :type treatment: np.array or pd.Series :param y: an outcome vector :type y: np.array or pd.Series

fit_predict(X, treatment, y, p=None, return_ci=False, n_bootstraps=1000, bootstrap_size=10000, return_components=False, verbose=True)[source]

Fit the inference model of the S learner and predict treatment effects. :param X: a feature matrix :type X: np.matrix, np.array, or pd.Dataframe :param treatment: a treatment vector :type treatment: np.array or pd.Series :param y: an outcome vector :type y: np.array or pd.Series :param return_ci: whether to return confidence intervals :type return_ci: bool, optional :param n_bootstraps: number of bootstrap iterations :type n_bootstraps: int, optional :param bootstrap_size: number of samples per bootstrap :type bootstrap_size: int, optional :param return_components: whether to return outcome for treatment and control seperately :type return_components: bool, optional :param verbose: whether to output progress logs :type verbose: bool, optional

Returns

Predictions of treatment effects. Output dim: [n_samples, n_treatment].

If return_ci, returns CATE [n_samples, n_treatment], LB [n_samples, n_treatment], UB [n_samples, n_treatment]

Return type

(numpy.ndarray)

predict(X, treatment=None, y=None, p=None, return_components=False, verbose=True)[source]

Predict treatment effects. :param X: a feature matrix :type X: np.matrix or np.array or pd.Dataframe :param treatment: a treatment vector :type treatment: np.array or pd.Series, optional :param y: an outcome vector :type y: np.array or pd.Series, optional :param return_components: whether to return outcome for treatment and control seperately :type return_components: bool, optional :param verbose: whether to output progress logs :type verbose: bool, optional

Returns

Predictions of treatment effects.

Return type

(numpy.ndarray)

class causalml.inference.meta.BaseSRegressor(learner=None, ate_alpha=0.05, control_name=0)[source]

Bases: causalml.inference.meta.slearner.BaseSLearner

A parent class for S-learner regressor classes.

class causalml.inference.meta.BaseTClassifier(learner=None, control_learner=None, treatment_learner=None, ate_alpha=0.05, control_name=0)[source]

Bases: causalml.inference.meta.tlearner.BaseTLearner

A parent class for T-learner classifier classes.

predict(X, treatment=None, y=None, p=None, return_components=False, verbose=True)[source]

Predict treatment effects.

Parameters
  • X (np.matrix or np.array or pd.Dataframe) – a feature matrix

  • treatment (np.array or pd.Series, optional) – a treatment vector

  • y (np.array or pd.Series, optional) – an outcome vector

  • verbose (bool, optional) – whether to output progress logs

Returns

Predictions of treatment effects.

Return type

(numpy.ndarray)

class causalml.inference.meta.BaseTLearner(learner=None, control_learner=None, treatment_learner=None, ate_alpha=0.05, control_name=0)[source]

Bases: causalml.inference.meta.base.BaseLearner

A parent class for T-learner regressor classes.

A T-learner estimates treatment effects with two machine learning models.

Details of T-learner are available at Kunzel et al. (2018) (https://arxiv.org/abs/1706.03461).

estimate_ate(X, treatment, y, p=None, bootstrap_ci=False, n_bootstraps=1000, bootstrap_size=10000)[source]

Estimate the Average Treatment Effect (ATE).

Parameters
  • X (np.matrix or np.array or pd.Dataframe) – a feature matrix

  • treatment (np.array or pd.Series) – a treatment vector

  • y (np.array or pd.Series) – an outcome vector

  • bootstrap_ci (bool) – whether to return confidence intervals

  • n_bootstraps (int) – number of bootstrap iterations

  • bootstrap_size (int) – number of samples per bootstrap

Returns

The mean and confidence interval (LB, UB) of the ATE estimate.

fit(X, treatment, y, p=None)[source]

Fit the inference model

Parameters
  • X (np.matrix or np.array or pd.Dataframe) – a feature matrix

  • treatment (np.array or pd.Series) – a treatment vector

  • y (np.array or pd.Series) – an outcome vector

fit_predict(X, treatment, y, p=None, return_ci=False, n_bootstraps=1000, bootstrap_size=10000, return_components=False, verbose=True)[source]

Fit the inference model of the T learner and predict treatment effects.

Parameters
  • X (np.matrix or np.array or pd.Dataframe) – a feature matrix

  • treatment (np.array or pd.Series) – a treatment vector

  • y (np.array or pd.Series) – an outcome vector

  • return_ci (bool) – whether to return confidence intervals

  • n_bootstraps (int) – number of bootstrap iterations

  • bootstrap_size (int) – number of samples per bootstrap

  • return_components (bool, optional) – whether to return outcome for treatment and control seperately

  • verbose (str) – whether to output progress logs

Returns

Predictions of treatment effects. Output dim: [n_samples, n_treatment].

If return_ci, returns CATE [n_samples, n_treatment], LB [n_samples, n_treatment], UB [n_samples, n_treatment]

Return type

(numpy.ndarray)

predict(X, treatment=None, y=None, p=None, return_components=False, verbose=True)[source]

Predict treatment effects.

Parameters
  • X (np.matrix or np.array or pd.Dataframe) – a feature matrix

  • treatment (np.array or pd.Series, optional) – a treatment vector

  • y (np.array or pd.Series, optional) – an outcome vector

  • return_components (bool, optional) – whether to return outcome for treatment and control seperately

  • verbose (bool, optional) – whether to output progress logs

Returns

Predictions of treatment effects.

Return type

(numpy.ndarray)

class causalml.inference.meta.BaseTRegressor(learner=None, control_learner=None, treatment_learner=None, ate_alpha=0.05, control_name=0)[source]

Bases: causalml.inference.meta.tlearner.BaseTLearner

A parent class for T-learner regressor classes.

class causalml.inference.meta.BaseXClassifier(outcome_learner=None, effect_learner=None, control_outcome_learner=None, treatment_outcome_learner=None, control_effect_learner=None, treatment_effect_learner=None, ate_alpha=0.05, control_name=0)[source]

Bases: causalml.inference.meta.xlearner.BaseXLearner

A parent class for X-learner classifier classes.

fit(X, treatment, y, p=None)[source]

Fit the inference model.

Parameters
  • X (np.matrix or np.array or pd.Dataframe) – a feature matrix

  • treatment (np.array or pd.Series) – a treatment vector

  • y (np.array or pd.Series) – an outcome vector

  • p (np.ndarray or pd.Series or dict, optional) – an array of propensity scores of float (0,1) in the single-treatment case; or, a dictionary of treatment groups that map to propensity vectors of float (0,1); if None will run ElasticNetPropensityModel() to generate the propensity scores.

predict(X, treatment=None, y=None, p=None, return_components=False, verbose=True)[source]

Predict treatment effects.

Parameters
  • X (np.matrix or np.array or pd.Dataframe) – a feature matrix

  • treatment (np.array or pd.Series, optional) – a treatment vector

  • y (np.array or pd.Series, optional) – an outcome vector

  • p (np.ndarray or pd.Series or dict, optional) – an array of propensity scores of float (0,1) in the single-treatment case; or, a dictionary of treatment groups that map to propensity vectors of float (0,1); if None will run ElasticNetPropensityModel() to generate the propensity scores.

  • return_components (bool, optional) – whether to return outcome for treatment and control seperately

  • return_p_score (bool, optional) – whether to return propensity score

  • verbose (bool, optional) – whether to output progress logs

Returns

Predictions of treatment effects.

Return type

(numpy.ndarray)

class causalml.inference.meta.BaseXLearner(learner=None, control_outcome_learner=None, treatment_outcome_learner=None, control_effect_learner=None, treatment_effect_learner=None, ate_alpha=0.05, control_name=0)[source]

Bases: causalml.inference.meta.base.BaseLearner

A parent class for X-learner regressor classes.

An X-learner estimates treatment effects with four machine learning models.

Details of X-learner are available at Kunzel et al. (2018) (https://arxiv.org/abs/1706.03461).

estimate_ate(X, treatment, y, p=None, bootstrap_ci=False, n_bootstraps=1000, bootstrap_size=10000)[source]

Estimate the Average Treatment Effect (ATE).

Parameters
  • X (np.matrix or np.array or pd.Dataframe) – a feature matrix

  • treatment (np.array or pd.Series) – a treatment vector

  • y (np.array or pd.Series) – an outcome vector

  • p (np.ndarray or pd.Series or dict, optional) – an array of propensity scores of float (0,1) in the single-treatment case; or, a dictionary of treatment groups that map to propensity vectors of float (0,1); if None will run ElasticNetPropensityModel() to generate the propensity scores.

  • bootstrap_ci (bool) – whether run bootstrap for confidence intervals

  • n_bootstraps (int) – number of bootstrap iterations

  • bootstrap_size (int) – number of samples per bootstrap

Returns

The mean and confidence interval (LB, UB) of the ATE estimate.

fit(X, treatment, y, p=None)[source]

Fit the inference model.

Parameters
  • X (np.matrix or np.array or pd.Dataframe) – a feature matrix

  • treatment (np.array or pd.Series) – a treatment vector

  • y (np.array or pd.Series) – an outcome vector

  • p (np.ndarray or pd.Series or dict, optional) – an array of propensity scores of float (0,1) in the single-treatment case; or, a dictionary of treatment groups that map to propensity vectors of float (0,1); if None will run ElasticNetPropensityModel() to generate the propensity scores.

fit_predict(X, treatment, y, p=None, return_ci=False, n_bootstraps=1000, bootstrap_size=10000, return_components=False, verbose=True)[source]

Fit the treatment effect and outcome models of the R learner and predict treatment effects.

Parameters
  • X (np.matrix or np.array or pd.Dataframe) – a feature matrix

  • treatment (np.array or pd.Series) – a treatment vector

  • y (np.array or pd.Series) – an outcome vector

  • p (np.ndarray or pd.Series or dict, optional) – an array of propensity scores of float (0,1) in the single-treatment case; or, a dictionary of treatment groups that map to propensity vectors of float (0,1); if None will run ElasticNetPropensityModel() to generate the propensity scores.

  • return_ci (bool) – whether to return confidence intervals

  • n_bootstraps (int) – number of bootstrap iterations

  • bootstrap_size (int) – number of samples per bootstrap

  • return_components (bool, optional) – whether to return outcome for treatment and control seperately

  • verbose (str) – whether to output progress logs

Returns

Predictions of treatment effects. Output dim: [n_samples, n_treatment]

If return_ci, returns CATE [n_samples, n_treatment], LB [n_samples, n_treatment], UB [n_samples, n_treatment]

Return type

(numpy.ndarray)

predict(X, treatment=None, y=None, p=None, return_components=False, verbose=True)[source]

Predict treatment effects.

Parameters
  • X (np.matrix or np.array or pd.Dataframe) – a feature matrix

  • treatment (np.array or pd.Series, optional) – a treatment vector

  • y (np.array or pd.Series, optional) – an outcome vector

  • p (np.ndarray or pd.Series or dict, optional) – an array of propensity scores of float (0,1) in the single-treatment case; or, a dictionary of treatment groups that map to propensity vectors of float (0,1); if None will run ElasticNetPropensityModel() to generate the propensity scores.

  • return_components (bool, optional) – whether to return outcome for treatment and control seperately

  • verbose (bool, optional) – whether to output progress logs

Returns

Predictions of treatment effects.

Return type

(numpy.ndarray)

class causalml.inference.meta.BaseXRegressor(learner=None, control_outcome_learner=None, treatment_outcome_learner=None, control_effect_learner=None, treatment_effect_learner=None, ate_alpha=0.05, control_name=0)[source]

Bases: causalml.inference.meta.xlearner.BaseXLearner

A parent class for X-learner regressor classes.

class causalml.inference.meta.LRSRegressor(ate_alpha=0.05, control_name=0)[source]

Bases: causalml.inference.meta.slearner.BaseSRegressor

estimate_ate(X, treatment, y, p=None)[source]

Estimate the Average Treatment Effect (ATE). :param X: a feature matrix :type X: np.matrix, np.array, or pd.Dataframe :param treatment: a treatment vector :type treatment: np.array or pd.Series :param y: an outcome vector :type y: np.array or pd.Series

Returns

The mean and confidence interval (LB, UB) of the ATE estimate.

class causalml.inference.meta.MLPTRegressor(ate_alpha=0.05, control_name=0, *args, **kwargs)[source]

Bases: causalml.inference.meta.tlearner.BaseTRegressor

class causalml.inference.meta.TMLELearner(learner, ate_alpha=0.05, control_name=0, cv=None, calibrate_propensity=True)[source]

Bases: object

Targeted maximum likelihood estimation.

Ref: Gruber, S., & Van Der Laan, M. J. (2009). Targeted maximum likelihood estimation: A gentle introduction.

estimate_ate(X, treatment, y, p, segment=None, return_ci=False)[source]

Estimate the Average Treatment Effect (ATE).

Parameters
  • X (np.matrix or np.array or pd.Dataframe) – a feature matrix

  • treatment (np.array or pd.Series) – a treatment vector

  • y (np.array or pd.Series) – an outcome vector

  • p (np.ndarray or pd.Series or dict) – an array of propensity scores of float (0,1) in the single-treatment case; or, a dictionary of treatment groups that map to propensity vectors of float (0,1)

  • segment (np.array, optional) – An optional segment vector of int. If given, the ATE and its CI will be estimated for each segment.

  • return_ci (bool, optional) – Whether to return confidence intervals

Returns

The ATE and its confidence interval (LB, UB) for each treatment, t and segment, s

Return type

(tuple)

class causalml.inference.meta.XGBDRRegressor(ate_alpha=0.05, control_name=0, *args, **kwargs)[source]

Bases: causalml.inference.meta.drlearner.BaseDRRegressor

class causalml.inference.meta.XGBRRegressor(early_stopping=True, test_size=0.3, early_stopping_rounds=30, effect_learner_objective='rank:pairwise', effect_learner_n_estimators=500, random_state=42, *args, **kwargs)[source]

Bases: causalml.inference.meta.rlearner.BaseRRegressor

fit(X, treatment, y, p=None, verbose=True)[source]

Fit the treatment effect and outcome models of the R learner.

Parameters
  • X (np.matrix or np.array or pd.Dataframe) – a feature matrix

  • y (np.array or pd.Series) – an outcome vector

  • p (np.ndarray or pd.Series or dict, optional) – an array of propensity scores of float (0,1) in the single-treatment case; or, a dictionary of treatment groups that map to propensity vectors of float (0,1); if None will run ElasticNetPropensityModel() to generate the propensity scores.

  • verbose (bool, optional) – whether to output progress logs

class causalml.inference.meta.XGBTRegressor(ate_alpha=0.05, control_name=0, *args, **kwargs)[source]

Bases: causalml.inference.meta.tlearner.BaseTRegressor

causalml.optimize module

class causalml.optimize.CounterfactualUnitSelector(learner, nevertaker_payoff, alwaystaker_payoff, complier_payoff, defier_payoff, organic_conversion=None)[source]

Bases: object

A highly experimental implementation of the counterfactual unit selection model proposed by Li and Pearl (2019).

Parameters
  • learner (object) – The base learner used to estimate the segment probabilities.

  • nevertaker_payoff (float) – The payoff from targeting a never-taker

  • alwaystaker_payoff (float) – The payoff from targeting an always-taker

  • complier_payoff (float) – The payoff from targeting a complier

  • defier_payoff (float) – The payoff from targeting a defier

  • organic_conversion (float, optional (default=None)) –

    The organic conversion rate in the population without an intervention. If None, the organic conversion rate is obtained from tne control group.

    NB: The organic conversion in the control group is not always the same as the organic conversion rate without treatment.

  • data (DataFrame) – A pandas DataFrame containing the features, treatment assignment indicator and the outcome of interest.

  • treatment (string) – A string corresponding to the name of the treatment column. The assumed coding in the column is 1 for treatment and 0 for control.

  • outcome (string) – A string corresponding to the name of the outcome column. The assumed coding in the column is 1 for conversion and 0 for no conversion.

References

Li, Ang, and Judea Pearl. 2019. “Unit Selection Based on Counterfactual Logic.” https://ftp.cs.ucla.edu/pub/stat_ser/r488.pdf.

fit(data, treatment, outcome)[source]

Fits the class.

predict(data, treatment, outcome)[source]

Predicts an individual-level payoff. If gain equality is satisfied, uses the exact function; if not, uses the midpoint between bounds.

class causalml.optimize.CounterfactualValueEstimator(treatment, control_name, treatment_names, y_proba, cate, value, conversion_cost, impression_cost, *args, **kwargs)[source]

Bases: object

Parameters
  • treatment (array, shape = (num_samples, )) – An array of treatment group indicator values.

  • control_name (string) – The name of the control condition as a string. Must be contained in the treatment array.

  • treatment_names (list, length = cate.shape[1]) – A list of treatment group names. NB: The order of the items in the list must correspond to the order in which the conditional average treatment effect estimates are in cate_array.

  • y_proba (array, shape = (num_samples, )) – The predicted probability of conversion using the Y ~ X model across the total sample.

  • cate (array, shape = (num_samples, len(set(treatment)))) – Conditional average treatment effect estimations from any model.

  • value (array, shape = (num_samples, )) – Value of converting each unit.

  • conversion_cost (shape = (num_samples, len(set(treatment)))) – The cost of a treatment that is triggered if a unit converts after having been in the treatment, such as a promotion code.

  • impression_cost (shape = (num_samples, len(set(treatment)))) – The cost of a treatment that is the same for each unit whether or not they convert, such as a cost associated with a promotion channel.

Notes

Because we get the conditional average treatment effects from cate-learners relative to the control condition, we subtract the cate for the unit in their actual treatment group from y_proba for that unit, in order to recover the control outcome. We then add the cates to the control outcome to obtain y_proba under each condition. These outcomes are counterfactual because just one of them is actually observed.

predict_best()[source]

Predict the best treatment group based on the highest counterfactual value for a treatment.

predict_counterfactuals()[source]

Predict the counterfactual values for each treatment group.

class causalml.optimize.PolicyLearner(outcome_learner=GradientBoostingRegressor(), treatment_learner=GradientBoostingClassifier(), policy_learner=DecisionTreeClassifier(), clip_bounds=(0.001, 0.999), n_fold=5, random_state=None, calibration=False)[source]

Bases: object

A Learner that learns a treatment assignment policy with observational data using doubly robust estimator of causal effect for binary treatment.

Details of the policy learner are available at Athey and Wager (2018) (https://arxiv.org/abs/1702.02896).

fit(X, treatment, y, p=None, dhat=None)[source]

Fit the treatment assignment policy learner.

Parameters
  • X (np.matrix) – a feature matrix

  • treatment (np.array) – a treatment vector (1 if treated, otherwise 0)

  • y (np.array) – an outcome vector

  • p (optional, np.array) – user provided propensity score vector between 0 and 1

  • dhat (optinal, np.array) – user provided predicted treatment effect vector

Returns

returns an instance of self.

Return type

self

predict(X)[source]

Predict treatment assignment that optimizes the outcome.

Parameters

X (np.matrix) – a feature matrix

Returns

predictions of treatment assignment.

Return type

(numpy.ndarray)

predict_proba(X)[source]

Predict treatment assignment score that optimizes the outcome.

Parameters

X (np.matrix) – a feature matrix

Returns

predictions of treatment assignment score.

Return type

(numpy.ndarray)

causalml.optimize.get_actual_value(treatment, observed_outcome, conversion_value, conditions, conversion_cost, impression_cost)[source]

Set the conversion and impression costs based on a dict of parameters.

Calculate the actual value of targeting a user with the actual treatment group using the above parameters.

treatmentarray, shape = (num_samples, )

Treatment array.

observed_outcomearray, shape = (num_samples, )

Observed outcome array, aka y.

conversion_valuearray, shape = (num_samples, )

The value of converting a given user.

conditionslist, len = len(set(treatment))

List of treatment conditions.

conversion_costarray, shape = (num_samples, num_treatment)

Array of conversion costs for each unit in each treatment.

impression_costarray, shape = (num_samples, num_treatment)

Array of impression costs for each unit in each treatment.

Returns

  • actual_value (array, shape = (num_samples, )) – Array of actual values of havng a user in their actual treatment group.

  • conversion_value (array, shape = (num_samples, )) – Array of payoffs from converting a user.

causalml.optimize.get_treatment_costs(treatment, control_name, cc_dict, ic_dict)[source]

Set the conversion and impression costs based on a dict of parameters.

Calculate the actual cost of targeting a user with the actual treatment group using the above parameters.

treatmentarray, shape = (num_samples, )

Treatment array.

control_name, str

Control group name as string.

cc_dictdict

Dict containing the conversion cost for each treatment.

ic_dict

Dict containing the impression cost for each treatment.

Returns

  • conversion_cost (ndarray, shape = (num_samples, num_treatments)) – An array of conversion costs for each treatment.

  • impression_cost (ndarray, shape = (num_samples, num_treatments)) – An array of impression costs for each treatment.

  • conditions (list, len = len(set(treatment))) – A list of experimental conditions.

causalml.optimize.get_uplift_best(cate, conditions)[source]

Takes the CATE prediction from a learner, adds the control outcome array and finds the name of the argmax conditon.

catearray, shape = (num_samples, )

The conditional average treatment effect prediction.

conditions : list, len = len(set(treatment))

Returns

uplift_recomm_name – The experimental group recommended by the learner.

Return type

array, shape = (num_samples, )

causalml.dataset module

causalml.match module

class causalml.match.MatchOptimizer(treatment_col='is_treatment', ps_col='pihat', user_col=None, matching_covariates=['pihat'], max_smd=0.1, max_deviation=0.1, caliper_range=(0.01, 0.5), max_pihat_range=(0.95, 0.999), max_iter_per_param=5, min_users_per_group=1000, smd_cols=['pihat'], dev_cols_transformations={'pihat': <function mean>}, dev_factor=1.0, verbose=True)[source]

Bases: object

check_table_one(tableone, matched, score_cols, pihat_threshold, caliper)[source]
match_and_check(score_cols, pihat_threshold, caliper)[source]
search_best_match(df)[source]
single_match(score_cols, pihat_threshold, caliper)[source]
class causalml.match.NearestNeighborMatch(caliper=0.2, replace=False, ratio=1, shuffle=True, random_state=None)[source]

Bases: object

Propensity score matching based on the nearest neighbor algorithm.

caliper

threshold to be considered as a match.

Type

float

replace

whether to match with replacement or not

Type

bool

ratio

ratio of control / treatment to be matched. used only if replace=True.

Type

int

shuffle

whether to shuffle the treatment group data before matching

Type

bool

random_state

RandomState or an int seed

Type

numpy.random.RandomState or int

match(data, treatment_col, score_cols)[source]

Find matches from the control group by matching on specified columns (propensity preferred).

Parameters
  • data (pandas.DataFrame) – total input data

  • treatment_col (str) – the column name for the treatment

  • score_cols (list) – list of column names for matching (propensity column should be included)

Returns

The subset of data consisting of matched

treatment and control group data.

Return type

(pandas.DataFrame)

match_by_group(data, treatment_col, score_cols, groupby_col)[source]

Find matches from the control group stratified by groupby_col, by matching on specified columns (propensity preferred).

Parameters
  • data (pandas.DataFrame) – total sample data

  • treatment_col (str) – the column name for the treatment

  • score_cols (list) – list of column names for matching (propensity column should be included)

  • groupby_col (str) – the column name to be used for stratification

Returns

The subset of data consisting of matched

treatment and control group data.

Return type

(pandas.DataFrame)

causalml.match.create_table_one(data, treatment_col, features)[source]

Report balance in input features between the treatment and control groups.

References

R’s tableone at CRAN: https://github.com/kaz-yos/tableone Python’s tableone at PyPi: https://github.com/tompollard/tableone

Parameters
  • data (pandas.DataFrame) – total or matched sample data

  • treatment_col (str) – the column name for the treatment

  • features (list of str) – the column names of features

Returns

A table with the means and standard deviations in

the treatment and control groups, and the SMD between two groups for the features.

Return type

(pandas.DataFrame)

causalml.match.smd(feature, treatment)[source]

Calculate the standard mean difference (SMD) of a feature between the treatment and control groups.

The definition is available at https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3144483/#s11title

Parameters
  • feature (pandas.Series) – a column of a feature to calculate SMD for

  • treatment (pandas.Series) – a column that indicate whether a row is in the treatment group or not

Returns

The SMD of the feature

Return type

(float)

causalml.propensity module

class causalml.propensity.ElasticNetPropensityModel(clip_bounds=(0.001, 0.999), **model_kwargs)[source]

Bases: causalml.propensity.LogisticRegressionPropensityModel

class causalml.propensity.GradientBoostedPropensityModel(early_stop=False, clip_bounds=(0.001, 0.999), **model_kwargs)[source]

Bases: causalml.propensity.PropensityModel

Gradient boosted propensity score model with optional early stopping.

Notes

Please see the xgboost documentation for more information on gradient boosting tuning parameters: https://xgboost.readthedocs.io/en/latest/python/python_api.html

fit(X, y, early_stopping_rounds=10, stop_val_size=0.2)[source]

Fit a propensity model.

Parameters
  • X (numpy.ndarray) – a feature matrix

  • y (numpy.ndarray) – a binary target vector

predict(X)[source]

Predict propensity scores.

Parameters

X (numpy.ndarray) – a feature matrix

Returns

Propensity scores between 0 and 1.

Return type

(numpy.ndarray)

class causalml.propensity.LogisticRegressionPropensityModel(clip_bounds=(0.001, 0.999), **model_kwargs)[source]

Bases: causalml.propensity.PropensityModel

Propensity regression model based on the LogisticRegression algorithm.

class causalml.propensity.PropensityModel(clip_bounds=(0.001, 0.999), **model_kwargs)[source]

Bases: object

fit(X, y)[source]

Fit a propensity model.

Parameters
  • X (numpy.ndarray) – a feature matrix

  • y (numpy.ndarray) – a binary target vector

fit_predict(X, y)[source]

Fit a propensity model and predict propensity scores.

Parameters
  • X (numpy.ndarray) – a feature matrix

  • y (numpy.ndarray) – a binary target vector

Returns

Propensity scores between 0 and 1.

Return type

(numpy.ndarray)

predict(X)[source]

Predict propensity scores.

Parameters

X (numpy.ndarray) – a feature matrix

Returns

Propensity scores between 0 and 1.

Return type

(numpy.ndarray)

causalml.propensity.calibrate(ps, treatment)[source]

Calibrate propensity scores with logistic GAM.

Ref: https://pygam.readthedocs.io/en/latest/api/logisticgam.html

Parameters
  • ps (numpy.array) – a propensity score vector

  • treatment (numpy.array) – a binary treatment vector (0: control, 1: treated)

Returns

a calibrated propensity score vector

Return type

(numpy.array)

causalml.propensity.compute_propensity_score(X, treatment, p_model=None, X_pred=None, treatment_pred=None, calibrate_p=True)[source]

Generate propensity score if user didn’t provide

Parameters
  • X (np.matrix) – features for training

  • treatment (np.array or pd.Series) – a treatment vector for training

  • p_model (propensity model object, optional) – ElasticNetPropensityModel (default) / GradientBoostedPropensityModel

  • X_pred (np.matrix, optional) – features for prediction

  • treatment_pred (np.array or pd.Series, optional) – a treatment vector for prediciton

  • calibrate_p (bool, optional) – whether calibrate the propensity score

Returns

(tuple)
  • p (numpy.ndarray): propensity score

  • p_model (PropensityModel): a trained PropensityModel object

causalml.metrics module

class causalml.metrics.Sensitivity(df, inference_features, p_col, treatment_col, outcome_col, learner, *args, **kwargs)[source]

Bases: object

A Sensitivity Check class to support Placebo Treatment, Irrelevant Additional Confounder and Subset validation refutation methods to verify causal inference.

Reference: https://github.com/microsoft/dowhy/blob/master/dowhy/causal_refuters/

get_ate_ci(X, p, treatment, y)[source]

Return the confidence intervals for treatment effects prediction.

Parameters
  • X (np.matrix) – a feature matrix

  • p (np.array) – a propensity score vector between 0 and 1

  • treatment (np.array) – a treatment vector (1 if treated, otherwise 0)

  • y (np.array) – an outcome vector

Returns

Mean and confidence interval (LB, UB) of the ATE estimate.

Return type

(numpy.ndarray)

static get_class_object(method_name, *args, **kwargs)[source]

Return class object based on input method :param method_name: a list of sensitivity analysis method :type method_name: list of str

Returns

Sensitivy Class

Return type

(class)

get_prediction(X, p, treatment, y)[source]

Return the treatment effects prediction.

Parameters
  • X (np.matrix) – a feature matrix

  • p (np.array) – a propensity score vector between 0 and 1

  • treatment (np.array) – a treatment vector (1 if treated, otherwise 0)

  • y (np.array) – an outcome vector

Returns

Predictions of treatment effects

Return type

(numpy.ndarray)

sensitivity_analysis(methods, sample_size=None, confound='one_sided', alpha_range=None)[source]

Return the sensitivity data by different method

Parameters
  • method (list of str) – a list of sensitivity analysis method

  • sample_size (float, optional) – ratio for subset the original data

  • confound (string, optional) – the name of confouding function

  • alpha_range (np.array, optional) – a parameter to pass the confounding function

Returns

a feature matrix p (np.array): a propensity score vector between 0 and 1 treatment (np.array): a treatment vector (1 if treated, otherwise 0) y (np.array): an outcome vector

Return type

X (np.matrix)

sensitivity_estimate()[source]
summary(method)[source]

Summary report :param method_name: sensitivity analysis method :type method_name: str

Returns

a summary dataframe

Return type

(pd.DataFrame)

class causalml.metrics.SensitivityPlaceboTreatment(*args, **kwargs)[source]

Bases: causalml.metrics.sensitivity.Sensitivity

Replaces the treatment variable with a new variable randomly generated.

sensitivity_estimate()[source]

Summary report :param return_ci: sensitivity analysis method :type return_ci: str

Returns

a summary dataframe

Return type

(pd.DataFrame)

class causalml.metrics.SensitivityRandomCause(*args, **kwargs)[source]

Bases: causalml.metrics.sensitivity.Sensitivity

Adds an irrelevant random covariate to the dataframe.

sensitivity_estimate()[source]
class causalml.metrics.SensitivityRandomReplace(*args, **kwargs)[source]

Bases: causalml.metrics.sensitivity.Sensitivity

Replaces a random covariate with an irrelevant variable.

sensitivity_estimate()[source]

Replaces a random covariate with an irrelevant variable.

class causalml.metrics.SensitivitySelectionBias(*args, confound='one_sided', alpha_range=None, sensitivity_features=None, **kwargs)[source]

Bases: causalml.metrics.sensitivity.Sensitivity

Reference:

[1] Blackwell, Matthew. “A selection bias approach to sensitivity analysis for causal effects.” Political Analysis 22.2 (2014): 169-182. https://www.mattblackwell.org/files/papers/causalsens.pdf

[2] Confouding parameter alpha_range using the same range as in: https://github.com/mattblackwell/causalsens/blob/master/R/causalsens.R

causalsens()[source]
static partial_rsqs_confounding(sens_df, feature_name, partial_rsqs_value, range=0.01)[source]

Check partial rsqs values of feature corresponding confounding amonunt of ATE :param sens_df: a data frame output from causalsens :type sens_df: pandas.DataFrame :param feature_name: feature name to check :type feature_name: str :param partial_rsqs_value: partial rsquare value of feature :type partial_rsqs_value: float :param range: range to search from sens_df :type range: float

Return: min and max value of confounding amount

static plot(sens_df, partial_rsqs_df=None, type='raw', ci=False, partial_rsqs=False)[source]

Plot the results of a sensitivity analysis against unmeasured :param sens_df: a data frame output from causalsens :type sens_df: pandas.DataFrame :param partial_rsqs_d: a data frame output from causalsens including partial rsqure :type partial_rsqs_d: pandas.DataFrame :param type: the type of plot to draw, ‘raw’ or ‘r.squared’ are supported :type type: str, optional :param ci: whether plot confidence intervals :type ci: bool, optional :param partial_rsqs: whether plot partial rsquare results :type partial_rsqs: bool, optional

summary(method='Selection Bias')[source]

Summary report for Selection Bias Method :param method_name: sensitivity analysis method :type method_name: str

Returns

a summary dataframe

Return type

(pd.DataFrame)

class causalml.metrics.SensitivitySubsetData(*args, **kwargs)[source]

Bases: causalml.metrics.sensitivity.Sensitivity

Takes a random subset of size sample_size of the data.

sensitivity_estimate()[source]
causalml.metrics.ape(y, p)[source]

Absolute Percentage Error (APE). :param y: target :type y: float :param p: prediction :type p: float

Returns

APE

Return type

e (float)

causalml.metrics.auuc_score(df, outcome_col='y', treatment_col='w', treatment_effect_col='tau', normalize=True, tmle=False, *args, **kwarg)[source]

Calculate the AUUC (Area Under the Uplift Curve) score.

Args:

df (pandas.DataFrame): a data frame with model estimates and actual data as columns outcome_col (str, optional): the column name for the actual outcome treatment_col (str, optional): the column name for the treatment indicator (0 or 1) treatment_effect_col (str, optional): the column name for the true treatment effect normalize (bool, optional): whether to normalize the y-axis to 1 or not

Returns

the AUUC score

Return type

(float)

causalml.metrics.classification_metrics(y, p, w=None, metrics={'AUC': <function roc_auc_score>, 'Log Loss': <function logloss>})[source]

Log metrics for classifiers.

Parameters
  • y (numpy.array) – target

  • p (numpy.array) – prediction

  • w (numpy.array, optional) – a treatment vector (1 or True: treatment, 0 or False: control). If given, log metrics for the treatment and control group separately

  • metrics (dict, optional) – a dictionary of the metric names and functions

causalml.metrics.get_cumgain(df, outcome_col='y', treatment_col='w', treatment_effect_col='tau', normalize=False, random_seed=42)[source]

Get cumulative gains of model estimates in population.

If the true treatment effect is provided (e.g. in synthetic data), it’s calculated as the cumulative gain of the true treatment effect in each population. Otherwise, it’s calculated as the cumulative difference between the mean outcomes of the treatment and control groups in each population.

For details, see Section 4.1 of Gutierrez and G{‘e}rardy (2016), Causal Inference and Uplift Modeling: A review of the literature.

For the former, treatment_effect_col should be provided. For the latter, both outcome_col and treatment_col should be provided.

Parameters
  • df (pandas.DataFrame) – a data frame with model estimates and actual data as columns

  • outcome_col (str, optional) – the column name for the actual outcome

  • treatment_col (str, optional) – the column name for the treatment indicator (0 or 1)

  • treatment_effect_col (str, optional) – the column name for the true treatment effect

  • normalize (bool, optional) – whether to normalize the y-axis to 1 or not

  • random_seed (int, optional) – random seed for numpy.random.rand()

Returns

cumulative gains of model estimates in population

Return type

(pandas.DataFrame)

causalml.metrics.get_cumlift(df, outcome_col='y', treatment_col='w', treatment_effect_col='tau', random_seed=42)[source]

Get average uplifts of model estimates in cumulative population.

If the true treatment effect is provided (e.g. in synthetic data), it’s calculated as the mean of the true treatment effect in each of cumulative population. Otherwise, it’s calculated as the difference between the mean outcomes of the treatment and control groups in each of cumulative population.

For details, see Section 4.1 of Gutierrez and G{‘e}rardy (2016), Causal Inference and Uplift Modeling: A review of the literature.

For the former, treatment_effect_col should be provided. For the latter, both outcome_col and treatment_col should be provided.

Parameters
  • df (pandas.DataFrame) – a data frame with model estimates and actual data as columns

  • outcome_col (str, optional) – the column name for the actual outcome

  • treatment_col (str, optional) – the column name for the treatment indicator (0 or 1)

  • treatment_effect_col (str, optional) – the column name for the true treatment effect

  • random_seed (int, optional) – random seed for numpy.random.rand()

Returns

average uplifts of model estimates in cumulative population

Return type

(pandas.DataFrame)

causalml.metrics.get_qini(df, outcome_col='y', treatment_col='w', treatment_effect_col='tau', normalize=False, random_seed=42)[source]

Get Qini of model estimates in population.

If the true treatment effect is provided (e.g. in synthetic data), it’s calculated as the cumulative gain of the true treatment effect in each population. Otherwise, it’s calculated as the cumulative difference between the mean outcomes of the treatment and control groups in each population.

For details, see Radcliffe (2007), Using Control Group to Target on Predicted Lift: Building and Assessing Uplift Models

For the former, treatment_effect_col should be provided. For the latter, both outcome_col and treatment_col should be provided.

Parameters
  • df (pandas.DataFrame) – a data frame with model estimates and actual data as columns

  • outcome_col (str, optional) – the column name for the actual outcome

  • treatment_col (str, optional) – the column name for the treatment indicator (0 or 1)

  • treatment_effect_col (str, optional) – the column name for the true treatment effect

  • normalize (bool, optional) – whether to normalize the y-axis to 1 or not

  • random_seed (int, optional) – random seed for numpy.random.rand()

Returns

cumulative gains of model estimates in population

Return type

(pandas.DataFrame)

causalml.metrics.get_tmlegain(df, inference_col, learner=LGBMRegressor(learning_rate=0.05, n_estimators=300, num_leaves=64), outcome_col='y', treatment_col='w', p_col='p', n_segment=5, cv=None, calibrate_propensity=True, ci=False)[source]

Get TMLE based average uplifts of model estimates of segments.

Parameters
  • df (pandas.DataFrame) – a data frame with model estimates and actual data as columns

  • inferenece_col (list of str) – a list of columns that used in learner for inference

  • learner (optional) – a model used by TMLE to estimate the outcome

  • outcome_col (str, optional) – the column name for the actual outcome

  • treatment_col (str, optional) – the column name for the treatment indicator (0 or 1)

  • p_col (str, optional) – the column name for propensity score

  • n_segment (int, optional) – number of segment that TMLE will estimated for each

  • cv (sklearn.model_selection._BaseKFold, optional) – sklearn CV object

  • calibrate_propensity (bool, optional) – whether calibrate propensity score or not

  • ci (bool, optional) – whether return confidence intervals for ATE or not

Returns

cumulative gains of model estimates based of TMLE

Return type

(pandas.DataFrame)

causalml.metrics.get_tmleqini(df, inference_col, learner=LGBMRegressor(learning_rate=0.05, n_estimators=300, num_leaves=64), outcome_col='y', treatment_col='w', p_col='p', n_segment=5, cv=None, calibrate_propensity=True, ci=False, normalize=False)[source]

Get TMLE based Qini of model estimates by segments.

Parameters
  • df (pandas.DataFrame) – a data frame with model estimates and actual data as columns

  • inferenece_col (list of str) – a list of columns that used in learner for inference

  • learner (optional) – a model used by TMLE to estimate the outcome

  • outcome_col (str, optional) – the column name for the actual outcome

  • treatment_col (str, optional) – the column name for the treatment indicator (0 or 1)

  • p_col (str, optional) – the column name for propensity score

  • n_segment (int, optional) – number of segment that TMLE will estimated for each

  • cv (sklearn.model_selection._BaseKFold, optional) – sklearn CV object

  • calibrate_propensity (bool, optional) – whether calibrate propensity score or not

  • ci (bool, optional) – whether return confidence intervals for ATE or not

Returns

cumulative gains of model estimates based of TMLE

Return type

(pandas.DataFrame)

causalml.metrics.gini(y, p)[source]

Normalized Gini Coefficient.

Parameters
  • y (numpy.array) – target

  • p (numpy.array) – prediction

Returns

normalized Gini coefficient

Return type

e (numpy.float64)

causalml.metrics.logloss(y, p)[source]

Bounded log loss error. :param y: target :type y: numpy.array :param p: prediction :type p: numpy.array

Returns

bounded log loss error

causalml.metrics.mae(y_true, y_pred, *, sample_weight=None, multioutput='uniform_average')

Mean absolute error regression loss.

Read more in the User Guide.

Parameters
  • y_true (array-like of shape (n_samples,) or (n_samples, n_outputs)) – Ground truth (correct) target values.

  • y_pred (array-like of shape (n_samples,) or (n_samples, n_outputs)) – Estimated target values.

  • sample_weight (array-like of shape (n_samples,), default=None) – Sample weights.

  • multioutput ({'raw_values', 'uniform_average'} or array-like of shape (n_outputs,), default='uniform_average') –

    Defines aggregating of multiple output values. Array-like value defines weights used to average errors.

    ’raw_values’ :

    Returns a full set of errors in case of multioutput input.

    ’uniform_average’ :

    Errors of all outputs are averaged with uniform weight.

Returns

loss – If multioutput is ‘raw_values’, then mean absolute error is returned for each output separately. If multioutput is ‘uniform_average’ or an ndarray of weights, then the weighted average of all output errors is returned.

MAE output is non-negative floating point. The best value is 0.0.

Return type

float or ndarray of floats

Examples

>>> from sklearn.metrics import mean_absolute_error
>>> y_true = [3, -0.5, 2, 7]
>>> y_pred = [2.5, 0.0, 2, 8]
>>> mean_absolute_error(y_true, y_pred)
0.5
>>> y_true = [[0.5, 1], [-1, 1], [7, -6]]
>>> y_pred = [[0, 2], [-1, 2], [8, -5]]
>>> mean_absolute_error(y_true, y_pred)
0.75
>>> mean_absolute_error(y_true, y_pred, multioutput='raw_values')
array([0.5, 1. ])
>>> mean_absolute_error(y_true, y_pred, multioutput=[0.3, 0.7])
0.85...
causalml.metrics.mape(y, p)[source]

Mean Absolute Percentage Error (MAPE). :param y: target :type y: numpy.array :param p: prediction :type p: numpy.array

Returns

MAPE

Return type

e (numpy.float64)

causalml.metrics.plot(df, kind='gain', tmle=False, n=100, figsize=(8, 8), *args, **kwarg)[source]

Plot one of the lift/gain/Qini charts of model estimates.

A factory method for plot_lift(), plot_gain(), plot_qini(), plot_tmlegain() and plot_tmleqini(). For details, pleas see docstrings of each function.

Parameters
  • df (pandas.DataFrame) – a data frame with model estimates and actual data as columns.

  • kind (str, optional) – the kind of plot to draw. ‘lift’, ‘gain’, and ‘qini’ are supported.

  • n (int, optional) – the number of samples to be used for plotting.

causalml.metrics.plot_gain(df, outcome_col='y', treatment_col='w', treatment_effect_col='tau', normalize=False, random_seed=42, n=100, figsize=(8, 8))[source]

Plot the cumulative gain chart (or uplift curve) of model estimates.

If the true treatment effect is provided (e.g. in synthetic data), it’s calculated as the cumulative gain of the true treatment effect in each population. Otherwise, it’s calculated as the cumulative difference between the mean outcomes of the treatment and control groups in each population.

For details, see Section 4.1 of Gutierrez and G{‘e}rardy (2016), Causal Inference and Uplift Modeling: A review of the literature.

For the former, treatment_effect_col should be provided. For the latter, both outcome_col and treatment_col should be provided.

Parameters
  • df (pandas.DataFrame) – a data frame with model estimates and actual data as columns

  • outcome_col (str, optional) – the column name for the actual outcome

  • treatment_col (str, optional) – the column name for the treatment indicator (0 or 1)

  • treatment_effect_col (str, optional) – the column name for the true treatment effect

  • normalize (bool, optional) – whether to normalize the y-axis to 1 or not

  • random_seed (int, optional) – random seed for numpy.random.rand()

  • n (int, optional) – the number of samples to be used for plotting

causalml.metrics.plot_lift(df, outcome_col='y', treatment_col='w', treatment_effect_col='tau', random_seed=42, n=100, figsize=(8, 8))[source]

Plot the lift chart of model estimates in cumulative population.

If the true treatment effect is provided (e.g. in synthetic data), it’s calculated as the mean of the true treatment effect in each of cumulative population. Otherwise, it’s calculated as the difference between the mean outcomes of the treatment and control groups in each of cumulative population.

For details, see Section 4.1 of Gutierrez and G{‘e}rardy (2016), Causal Inference and Uplift Modeling: A review of the literature.

For the former, treatment_effect_col should be provided. For the latter, both outcome_col and treatment_col should be provided.

Parameters
  • df (pandas.DataFrame) – a data frame with model estimates and actual data as columns

  • outcome_col (str, optional) – the column name for the actual outcome

  • treatment_col (str, optional) – the column name for the treatment indicator (0 or 1)

  • treatment_effect_col (str, optional) – the column name for the true treatment effect

  • random_seed (int, optional) – random seed for numpy.random.rand()

  • n (int, optional) – the number of samples to be used for plotting

causalml.metrics.plot_qini(df, outcome_col='y', treatment_col='w', treatment_effect_col='tau', normalize=False, random_seed=42, n=100, figsize=(8, 8))[source]

Plot the Qini chart (or uplift curve) of model estimates.

If the true treatment effect is provided (e.g. in synthetic data), it’s calculated as the cumulative gain of the true treatment effect in each population. Otherwise, it’s calculated as the cumulative difference between the mean outcomes of the treatment and control groups in each population.

For details, see Radcliffe (2007), Using Control Group to Target on Predicted Lift: Building and Assessing Uplift Models

For the former, treatment_effect_col should be provided. For the latter, both outcome_col and treatment_col should be provided.

Parameters
  • df (pandas.DataFrame) – a data frame with model estimates and actual data as columns

  • outcome_col (str, optional) – the column name for the actual outcome

  • treatment_col (str, optional) – the column name for the treatment indicator (0 or 1)

  • treatment_effect_col (str, optional) – the column name for the true treatment effect

  • normalize (bool, optional) – whether to normalize the y-axis to 1 or not

  • random_seed (int, optional) – random seed for numpy.random.rand()

  • n (int, optional) – the number of samples to be used for plotting

  • ci (bool, optional) – whether return confidence intervals for ATE or not

causalml.metrics.plot_tmlegain(df, inference_col, learner=LGBMRegressor(learning_rate=0.05, n_estimators=300, num_leaves=64), outcome_col='y', treatment_col='w', p_col='tau', n_segment=5, cv=None, calibrate_propensity=True, ci=False, figsize=(8, 8))[source]

Plot the lift chart based of TMLE estimation

Parameters
  • df (pandas.DataFrame) – a data frame with model estimates and actual data as columns

  • inferenece_col (list of str) – a list of columns that used in learner for inference

  • learner (optional) – a model used by TMLE to estimate the outcome

  • outcome_col (str, optional) – the column name for the actual outcome

  • treatment_col (str, optional) – the column name for the treatment indicator (0 or 1)

  • p_col (str, optional) – the column name for propensity score

  • n_segment (int, optional) – number of segment that TMLE will estimated for each

  • cv (sklearn.model_selection._BaseKFold, optional) – sklearn CV object

  • calibrate_propensity (bool, optional) – whether calibrate propensity score or not

  • ci (bool, optional) – whether return confidence intervals for ATE or not

causalml.metrics.plot_tmleqini(df, inference_col, learner=LGBMRegressor(learning_rate=0.05, n_estimators=300, num_leaves=64), outcome_col='y', treatment_col='w', p_col='tau', n_segment=5, cv=None, calibrate_propensity=True, ci=False, figsize=(8, 8))[source]

Plot the qini chart based of TMLE estimation

Parameters
  • df (pandas.DataFrame) – a data frame with model estimates and actual data as columns

  • inferenece_col (list of str) – a list of columns that used in learner for inference

  • learner (optional) – a model used by TMLE to estimate the outcome

  • outcome_col (str, optional) – the column name for the actual outcome

  • treatment_col (str, optional) – the column name for the treatment indicator (0 or 1)

  • p_col (str, optional) – the column name for propensity score

  • n_segment (int, optional) – number of segment that TMLE will estimated for each

  • cv (sklearn.model_selection._BaseKFold, optional) – sklearn CV object

  • calibrate_propensity (bool, optional) – whether calibrate propensity score or not

  • ci (bool, optional) – whether return confidence intervals for ATE or not

causalml.metrics.qini_score(df, outcome_col='y', treatment_col='w', treatment_effect_col='tau', normalize=True, tmle=False, *args, **kwarg)[source]

Calculate the Qini score: the area between the Qini curves of a model and random.

For details, see Radcliffe (2007), Using Control Group to Target on Predicted Lift: Building and Assessing Uplift Models

Args:

df (pandas.DataFrame): a data frame with model estimates and actual data as columns outcome_col (str, optional): the column name for the actual outcome treatment_col (str, optional): the column name for the treatment indicator (0 or 1) treatment_effect_col (str, optional): the column name for the true treatment effect normalize (bool, optional): whether to normalize the y-axis to 1 or not

Returns

the Qini score

Return type

(float)

causalml.metrics.r2_score(y_true, y_pred, *, sample_weight=None, multioutput='uniform_average')[source]

\(R^2\) (coefficient of determination) regression score function.

Best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse). A constant model that always predicts the expected value of y, disregarding the input features, would get a \(R^2\) score of 0.0.

Read more in the User Guide.

Parameters
  • y_true (array-like of shape (n_samples,) or (n_samples, n_outputs)) – Ground truth (correct) target values.

  • y_pred (array-like of shape (n_samples,) or (n_samples, n_outputs)) – Estimated target values.

  • sample_weight (array-like of shape (n_samples,), default=None) – Sample weights.

  • multioutput ({'raw_values', 'uniform_average', 'variance_weighted'}, array-like of shape (n_outputs,) or None, default='uniform_average') –

    Defines aggregating of multiple output scores. Array-like value defines weights used to average scores. Default is “uniform_average”.

    ’raw_values’ :

    Returns a full set of scores in case of multioutput input.

    ’uniform_average’ :

    Scores of all outputs are averaged with uniform weight.

    ’variance_weighted’ :

    Scores of all outputs are averaged, weighted by the variances of each individual output.

    Changed in version 0.19: Default value of multioutput is ‘uniform_average’.

Returns

z – The \(R^2\) score or ndarray of scores if ‘multioutput’ is ‘raw_values’.

Return type

float or ndarray of floats

Notes

This is not a symmetric function.

Unlike most other scores, \(R^2\) score may be negative (it need not actually be the square of a quantity R).

This metric is not well-defined for single samples and will return a NaN value if n_samples is less than two.

References

1

Wikipedia entry on the Coefficient of determination

Examples

>>> from sklearn.metrics import r2_score
>>> y_true = [3, -0.5, 2, 7]
>>> y_pred = [2.5, 0.0, 2, 8]
>>> r2_score(y_true, y_pred)
0.948...
>>> y_true = [[0.5, 1], [-1, 1], [7, -6]]
>>> y_pred = [[0, 2], [-1, 2], [8, -5]]
>>> r2_score(y_true, y_pred,
...          multioutput='variance_weighted')
0.938...
>>> y_true = [1, 2, 3]
>>> y_pred = [1, 2, 3]
>>> r2_score(y_true, y_pred)
1.0
>>> y_true = [1, 2, 3]
>>> y_pred = [2, 2, 2]
>>> r2_score(y_true, y_pred)
0.0
>>> y_true = [1, 2, 3]
>>> y_pred = [3, 2, 1]
>>> r2_score(y_true, y_pred)
-3.0
causalml.metrics.regression_metrics(y, p, w=None, metrics={'Gini': <function gini>, 'RMSE': <function rmse>, 'sMAPE': <function smape>})[source]

Log metrics for regressors.

Parameters
  • y (numpy.array) – target

  • p (numpy.array) – prediction

  • w (numpy.array, optional) – a treatment vector (1 or True: treatment, 0 or False: control). If given, log metrics for the treatment and control group separately

  • metrics (dict, optional) – a dictionary of the metric names and functions

causalml.metrics.rmse(y, p)[source]

Root Mean Squared Error (RMSE). :param y: target :type y: numpy.array :param p: prediction :type p: numpy.array

Returns

RMSE

Return type

e (numpy.float64)

causalml.metrics.roc_auc_score(y_true, y_score, *, average='macro', sample_weight=None, max_fpr=None, multi_class='raise', labels=None)[source]

Compute Area Under the Receiver Operating Characteristic Curve (ROC AUC) from prediction scores.

Note: this implementation can be used with binary, multiclass and multilabel classification, but some restrictions apply (see Parameters).

Read more in the User Guide.

Parameters
  • y_true (array-like of shape (n_samples,) or (n_samples, n_classes)) – True labels or binary label indicators. The binary and multiclass cases expect labels with shape (n_samples,) while the multilabel case expects binary label indicators with shape (n_samples, n_classes).

  • y_score (array-like of shape (n_samples,) or (n_samples, n_classes)) –

    Target scores.

    • In the binary case, it corresponds to an array of shape (n_samples,). Both probability estimates and non-thresholded decision values can be provided. The probability estimates correspond to the probability of the class with the greater label, i.e. estimator.classes_[1] and thus estimator.predict_proba(X, y)[:, 1]. The decision values corresponds to the output of estimator.decision_function(X, y). See more information in the User guide;

    • In the multiclass case, it corresponds to an array of shape (n_samples, n_classes) of probability estimates provided by the predict_proba method. The probability estimates must sum to 1 across the possible classes. In addition, the order of the class scores must correspond to the order of labels, if provided, or else to the numerical or lexicographical order of the labels in y_true. See more information in the User guide;

    • In the multilabel case, it corresponds to an array of shape (n_samples, n_classes). Probability estimates are provided by the predict_proba method and the non-thresholded decision values by the decision_function method. The probability estimates correspond to the probability of the class with the greater label for each output of the classifier. See more information in the User guide.

  • average ({'micro', 'macro', 'samples', 'weighted'} or None, default='macro') –

    If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data: Note: multiclass ROC AUC currently only handles the ‘macro’ and ‘weighted’ averages.

    'micro':

    Calculate metrics globally by considering each element of the label indicator matrix as a label.

    'macro':

    Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.

    'weighted':

    Calculate metrics for each label, and find their average, weighted by support (the number of true instances for each label).

    'samples':

    Calculate metrics for each instance, and find their average.

    Will be ignored when y_true is binary.

  • sample_weight (array-like of shape (n_samples,), default=None) – Sample weights.

  • max_fpr (float > 0 and <= 1, default=None) – If not None, the standardized partial AUC 2 over the range [0, max_fpr] is returned. For the multiclass case, max_fpr, should be either equal to None or 1.0 as AUC ROC partial computation currently is not supported for multiclass.

  • multi_class ({'raise', 'ovr', 'ovo'}, default='raise') –

    Only used for multiclass targets. Determines the type of configuration to use. The default value raises an error, so either 'ovr' or 'ovo' must be passed explicitly.

    'ovr':

    Stands for One-vs-rest. Computes the AUC of each class against the rest 3 4. This treats the multiclass case in the same way as the multilabel case. Sensitive to class imbalance even when average == 'macro', because class imbalance affects the composition of each of the ‘rest’ groupings.

    'ovo':

    Stands for One-vs-one. Computes the average AUC of all possible pairwise combinations of classes 5. Insensitive to class imbalance when average == 'macro'.

  • labels (array-like of shape (n_classes,), default=None) – Only used for multiclass targets. List of labels that index the classes in y_score. If None, the numerical or lexicographical order of the labels in y_true is used.

Returns

auc

Return type

float

References

1

Wikipedia entry for the Receiver operating characteristic

2

Analyzing a portion of the ROC curve. McClish, 1989

3

Provost, F., Domingos, P. (2000). Well-trained PETs: Improving probability estimation trees (Section 6.2), CeDER Working Paper #IS-00-04, Stern School of Business, New York University.

4

Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27(8), 861-874.

5

Hand, D.J., Till, R.J. (2001). A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems. Machine Learning, 45(2), 171-186.

See also

average_precision_score

Area under the precision-recall curve.

roc_curve

Compute Receiver operating characteristic (ROC) curve.

plot_roc_curve

Plot Receiver operating characteristic (ROC) curve.

Examples

Binary case:

>>> from sklearn.datasets import load_breast_cancer
>>> from sklearn.linear_model import LogisticRegression
>>> from sklearn.metrics import roc_auc_score
>>> X, y = load_breast_cancer(return_X_y=True)
>>> clf = LogisticRegression(solver="liblinear", random_state=0).fit(X, y)
>>> roc_auc_score(y, clf.predict_proba(X)[:, 1])
0.99...
>>> roc_auc_score(y, clf.decision_function(X))
0.99...

Multiclass case:

>>> from sklearn.datasets import load_iris
>>> X, y = load_iris(return_X_y=True)
>>> clf = LogisticRegression(solver="liblinear").fit(X, y)
>>> roc_auc_score(y, clf.predict_proba(X), multi_class='ovr')
0.99...

Multilabel case:

>>> from sklearn.datasets import make_multilabel_classification
>>> from sklearn.multioutput import MultiOutputClassifier
>>> X, y = make_multilabel_classification(random_state=0)
>>> clf = MultiOutputClassifier(clf).fit(X, y)
>>> # get a list of n_output containing probability arrays of shape
>>> # (n_samples, n_classes)
>>> y_pred = clf.predict_proba(X)
>>> # extract the positive columns for each output
>>> y_pred = np.transpose([pred[:, 1] for pred in y_pred])
>>> roc_auc_score(y, y_pred, average=None)
array([0.82..., 0.86..., 0.94..., 0.85... , 0.94...])
>>> from sklearn.linear_model import RidgeClassifierCV
>>> clf = RidgeClassifierCV().fit(X, y)
>>> roc_auc_score(y, clf.decision_function(X), average=None)
array([0.81..., 0.84... , 0.93..., 0.87..., 0.94...])
causalml.metrics.smape(y, p)[source]

Symmetric Mean Absolute Percentage Error (sMAPE). :param y: target :type y: numpy.array :param p: prediction :type p: numpy.array

Returns

sMAPE

Return type

e (numpy.float64)

Module contents