Rule Generator (Decision Tree algorithm) Example

This notebook contains an example of how the Rule Generator (Decision Tree algorithm) can be used to create rules based on a labelled dataset. This algorithm generate rules by extracting the highest performing branches from a tree ensemble model.

Requirements

To run, you’ll need the following:

  • A labelled, processed dataset (nulls imputed, categorical features encoded).


Import packages

[6]:
from iguanas.rule_generation import RuleGeneratorDT
from iguanas.metrics.classification import FScore

import pandas as pd
import numpy as np
from sklearn.ensemble import RandomForestClassifier

Read in data

Let’s read in some labelled, processed dummy data.

[7]:
X_train = pd.read_csv(
    'dummy_data/X_train.csv',
    index_col='eid'
)
y_train = pd.read_csv(
    'dummy_data/y_train.csv',
    index_col='eid'
).squeeze()
X_test = pd.read_csv(
    'dummy_data/X_test.csv',
    index_col='eid'
)
y_test = pd.read_csv(
    'dummy_data/y_test.csv',
    index_col='eid'
).squeeze()

Generate rules

Set up class parameters

Now we can set our class parameters for the Rule Generator. Here, we’re using the F1 score as the main rule performance metric (you can choose a different function from the metrics.classification module or create your own).

Note that if you’re using the FScore, Precision or Recall score as the optimisation function, use the FScore, Precision or Recall classes in the metrics.classification module rather than the same functions from Sklearn’s metrics module, as the former are ~100 times faster on larger datasets.

Please see the class docstring for more information on each parameter.

[8]:
fs = FScore(beta=1)
[9]:
params = {
    'n_total_conditions': 4,
    'opt_func': fs.fit,
    'tree_ensemble': RandomForestClassifier(n_estimators=100, random_state=0),
    'precision_threshold': 0.5,
    'num_cores': 1,
    'target_feat_corr_types': 'Infer',
    'verbose': 1
}

Instantiate class and run fit method

Once the parameters have been set, we can run the .fit() method to generate rules.

[10]:
rg = RuleGeneratorDT(**params)
[11]:
X_rules = rg.fit(
    X=X_train,
    y=y_train,
    sample_weight=None
)
--- Calculating correlation of features with respect to the target ---
--- Returning column datatypes ---
--- Training tree ensemble ---
[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.
[Parallel(n_jobs=1)]: Done 100 out of 100 | elapsed:    0.2s finished
--- Extracting rules from tree ensemble ---
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:00<00:00, 1865.53it/s]

Outputs

The .fit() method returns a dataframe giving the binary columns of the generated rules as applied to the training dataset.

Useful attributes created by running the .fit() method are:

  • rule_strings: The generated rules, defined using the standard Iguanas string format (values) and their names (keys).

  • rule_descriptions: A dataframe showing the logic of the generated rules and their performance metrics as applied to the training dataset.

[12]:
X_rules.head()
[12]:
Rule RGDT_Rule_20211123_100 RGDT_Rule_20211123_104 RGDT_Rule_20211123_130 RGDT_Rule_20211123_158 RGDT_Rule_20211123_157 RGDT_Rule_20211123_156 RGDT_Rule_20211123_94 RGDT_Rule_20211123_95 RGDT_Rule_20211123_173 RGDT_Rule_20211123_124 ... RGDT_Rule_20211123_112 RGDT_Rule_20211123_13 RGDT_Rule_20211123_18 RGDT_Rule_20211123_76 RGDT_Rule_20211123_208 RGDT_Rule_20211123_111 RGDT_Rule_20211123_68 RGDT_Rule_20211123_106 RGDT_Rule_20211123_107 RGDT_Rule_20211123_39
eid
867-8837095-9305559 0 0 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0 0 0
974-5306287-3527394 0 0 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0 0 0
584-0112844-9158928 0 0 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0 0 0
956-4190732-7014837 0 0 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0 0 0
349-7005645-8862067 0 0 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0 0 0

5 rows × 218 columns

[13]:
rg.rule_descriptions.head()
[13]:
Precision Recall PercDataFlagged OptMetric Logic nConditions
Rule
RGDT_Rule_20211123_100 0.991837 1.0 0.027547 0.995902 (X['account_number_num_fraud_transactions_per_... 2
RGDT_Rule_20211123_104 0.991837 1.0 0.027547 0.995902 (X['account_number_num_fraud_transactions_per_... 2
RGDT_Rule_20211123_130 0.991837 1.0 0.027547 0.995902 (X['account_number_num_fraud_transactions_per_... 2
RGDT_Rule_20211123_158 0.991837 1.0 0.027547 0.995902 (X['account_number_num_fraud_transactions_per_... 2
RGDT_Rule_20211123_157 0.991837 1.0 0.027547 0.995902 (X['account_number_num_fraud_transactions_per_... 2

Apply rules to a separate dataset

Use the .transform() method to apply the generated rules to a separate dataset.

[14]:
X_rules_test = rg.transform(
    X=X_test,
    y=y_test,
    sample_weight=None
)

Outputs

The .transform() method returns a dataframe giving the binary columns of the rules as applied to the given dataset.

A useful attribute created by running the .transform() method is:

  • rule_descriptions: A dataframe showing the logic of the generated rules and their performance metrics as applied to the given dataset.

[15]:
rg.rule_descriptions.head()
[15]:
Precision Recall PercDataFlagged OptMetric Logic nConditions
Rule
RGDT_Rule_20211123_100 0.991453 1.0 0.0267 0.995708 (X['account_number_num_fraud_transactions_per_... 2
RGDT_Rule_20211123_156 0.991453 1.0 0.0267 0.995708 (X['account_number_num_fraud_transactions_per_... 1
RGDT_Rule_20211123_95 0.991453 1.0 0.0267 0.995708 (X['account_number_num_fraud_transactions_per_... 2
RGDT_Rule_20211123_94 0.991453 1.0 0.0267 0.995708 (X['account_number_num_fraud_transactions_per_... 1
RGDT_Rule_20211123_104 0.991453 1.0 0.0267 0.995708 (X['account_number_num_fraud_transactions_per_... 2
[16]:
X_rules_test.head()
[16]:
Rule RGDT_Rule_20211123_100 RGDT_Rule_20211123_156 RGDT_Rule_20211123_95 RGDT_Rule_20211123_94 RGDT_Rule_20211123_104 RGDT_Rule_20211123_157 RGDT_Rule_20211123_130 RGDT_Rule_20211123_158 RGDT_Rule_20211123_173 RGDT_Rule_20211123_124 ... RGDT_Rule_20211123_107 RGDT_Rule_20211123_77 RGDT_Rule_20211123_13 RGDT_Rule_20211123_83 RGDT_Rule_20211123_76 RGDT_Rule_20211123_111 RGDT_Rule_20211123_68 RGDT_Rule_20211123_106 RGDT_Rule_20211123_208 RGDT_Rule_20211123_39
eid
975-8351797-7122581 0 0 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0 0 0
785-6259585-7858053 0 0 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0 0 0
057-4039373-1790681 0 0 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0 0 0
095-5263240-3834186 0 0 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0 0 0
980-3802574-0009480 0 0 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0 0 0

5 rows × 218 columns