skmultilearn.problem_transform package

The skmultilearn.problem_transform module provides classifiers that follow the problem transformation approaches to multi-label classification.

The problem transformation approach to multi-label classification converts multi-label problems to single-label problems: single-class or multi-class.

Classifier	Description
BinaryRelevance	treats each label as a separate single-class classification problem
ClassifierChain	treats each label as a part of a conditioned chain of single-class classification problems
ProbabilisticClassifierChain	extends Classifier Chains by modeling joint label distributions and estimating the probability of label sets. It trains a series of classifiers, each predicting the probability of a label, conditioned on the input features and preceding label predictions
ClassificationHeterogeneousFeature	augments the feature set with extra features derived from label probabilities and resolves cyclic dependencies between features and labels iteratively
LabelPowerset	treats each label combination as a separate class with one multi-class classification problem
InstanceBasedLogisticRegression	combines instance-based learning with logistic regression, using neighbors’ information as features. It has a K-Nearest Neighbor layer followed by Logistic Regression classifiers.
StructuredGridSearchCV	performs hyperparameter tuning for each label classifier, considering BR&CC structural properties. It searches for optimal classifiers with fine-tuned parameters for each label.

class skmultilearn.problem_transform.BinaryRelevance(classifier=None, require_dense=None)

Bases: ProblemTransformationBase

Performs classification per label

Transforms a multi-label classification problem with L labels into L single-label separate binary classification problems using the same base classifier provided in the constructor. The prediction output is the union of all per label classifiers

Parameters

classifierBaseEstimator

scikit-learn compatible base classifier

require_dense[bool, bool], optional

whether the base classifier requires dense representations for input features and classes/labels matrices in fit/predict. If value not provided, sparse representations are used if base classifier is an instance of MLClassifierBase and dense otherwise.

Attributes

model_count_int

number of trained models, in this classifier equal to n_labels

partition_List[List[int]], shape=(model_count_,)

list of lists of label indexes, used to index the output space matrix, set in _generate_partition() via fit()

classifiers_List[BaseEstimator] of shape model_count

list of classifiers trained per partition, set in fit()

Notes

Note

This is one of the most basic approaches to multi-label classification, it ignores relationships between labels.

Examples

An example use case for Binary Relevance classification with an sklearn.svm.SVC base classifier which supports sparse input:

from skmultilearn.problem_transform import BinaryRelevance
from sklearn.svm import SVC

# initialize Binary Relevance multi-label classifier
# with an SVM classifier
# SVM in scikit only supports the X matrix in sparse representation

classifier = BinaryRelevance(
    classifier = SVC(),
    require_dense = [False, True]
)

# train
classifier.fit(X_train, y_train)

# predict
predictions = classifier.predict(X_test)

Another way to use this classifier is to select the best scenario from a set of single-label classifiers used with Binary Relevance, this can be done using cross validation grid search. In the example below, the model with highest accuracy results is selected from either a sklearn.naive_bayes.MultinomialNB or sklearn.svm.SVC base classifier, alongside with best parameters for that base classifier.

from skmultilearn.problem_transform import BinaryRelevance
from sklearn.model_selection import GridSearchCV
from sklearn.naive_bayes import MultinomialNB
from sklearn.svm import SVC

parameters = [
    {
        'classifier': [MultinomialNB()],
        'classifier__alpha': [0.7, 1.0],
    },
    {
        'classifier': [SVC()],
        'classifier__kernel': ['rbf', 'linear'],
    },
]

clf = GridSearchCV(BinaryRelevance(), parameters, scoring='accuracy')
clf.fit(x, y)

print (clf.best_params_, clf.best_score_)

# result:
#
# {
#   'classifier': SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,
#   decision_function_shape='ovr', degree=3, gamma='auto', kernel='linear',
#   max_iter=-1, probability=False, random_state=None, shrinking=True,
#   tol=0.001, verbose=False), 'classifier__kernel': 'linear'
# } 0.17

fit(X, y)

Fits classifier to training data

Parameters

Xarray_like, numpy.matrix or scipy.sparse matrix, shape=(n_samples, n_features)

input feature matrix

yarray_like, numpy.matrix or scipy.sparse matrix of {0, 1}, shape=(n_samples, n_labels)

binary indicator matrix with label assignments

Returns

self

fitted instance of self

Notes

Note

Input matrices are converted to sparse format internally if a numpy representation is passed

predict(X)

Predict labels for X

Parameters

Xarray_like, numpy.matrix or scipy.sparse matrix, shape=(n_samples, n_features)

input feature matrix

Returns

scipy.sparse matrix of {0, 1}, shape=(n_samples, n_labels)

binary indicator matrix with label assignments

predict_proba(X)

Predict probabilities of label assignments for X

Parameters

Xarray_like, numpy.matrix or scipy.sparse matrix, shape=(n_samples, n_features)

input feature matrix

Returns

scipy.sparse matrix of float in [0.0, 1.0], shape=(n_samples, n_labels)

matrix with label assignment probabilities

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → BinaryRelevance

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to score.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in score.

Returns

selfobject

The updated object.

class skmultilearn.problem_transform.ClassificationHeterogeneousFeature(classifier=None, require_dense=None)

Bases: ProblemTransformationBase

Classification with heterogeneous features

This model is to augment the feature set with extra features which are from one for each label in the dataset. The cyclic dependency between features and labels is resolved iteratively.

There are two BR layers, composed of given base classifiers for each layer. The first layer is proposed to reproduce heterogeneous features, which will be augment to the original feature set. Heterogeneous features are predict probabilities produced by base classifier per each label. The second layer is to predict each labels with features from first layer.

Parameters

classifierBaseEstimator

scikit-learn compatible base classifier

require_dense[bool, bool], optional

whether the base classifier requires dense representations for input features and classes/labels matrices in fit/predict. If value not provided, sparse representations are used if base classifier is an instance of MLClassifierBase and dense otherwise.

Attributes

model_count_int

number of trained models, in this classifier equal to n_labels

partition_List[List[int]], shape=(model_count_,)

list of lists of label indexes, used to index the output space matrix, set in _generate_partition() via fit()

classifiers_List[BaseEstimator] of shape model_count

list of classifiers trained per partition, set in fit()

first_layer_List[BaseEstimator] of shape model_count

list of classifiers trained per partition for obtaining heterogeneous feature, set in fit()

References

If used, please cite the scikit-multilearn library and the relevant paper:

@inproceedings{godbole2004discriminative,
    title={Discriminative methods for multi-labeled classification},
    author={Godbole, Shantanu and Sarawagi, Sunita},
    booktitle={Pacific-Asia conference on knowledge discovery and data mining},
    pages={22--30},
    year={2004},
    organization={Springer}
}

Examples

An example use case for Classification with heterogeneous features with an sklearn.svm.SVC base classifier which supports sparse input:

from skmultilearn.problem_transform import ClassificationHeterogeneousFeature
from sklearn.model_selection import GridSearchCV
from sklearn.naive_bayes import MultinomialNB
from sklearn.svm import SVC

parameters = [
    {
        'classifier': [MultinomialNB()],
        'classifier__alpha': [0.7, 1.0],
    },
    {
        'classifier': [SVC(probability=True)],
        'classifier__kernel': ['rbf', 'linear'],
    },
]

clf = GridSearchCV(ClassificationHeterogeneousFeature(), parameters, scoring='accuracy')
clf.fit(X, y)

# Output the best parameters and the corresponding score
print(clf.best_params_, clf.best_score_)

# Example output
# {'classifier': MultinomialNB(alpha=0.7), 'classifier__alpha': 0.7} 0.21

fit(X, y)

Fits classifier to training data

Parameters

Xarray_like, numpy.matrix or scipy.sparse matrix, shape=(n_samples, n_features)

input feature matrix

yarray_like, numpy.matrix or scipy.sparse matrix of {0, 1}, shape=(n_samples, n_labels)

binary indicator matrix with label assignments

Returns

self

fitted instance of self

Notes

Note

Input matrices are converted to sparse format internally if a numpy representation is passed

predict(X)

Predict labels for X

Parameters

Xarray_like, numpy.matrix or scipy.sparse matrix, shape=(n_samples, n_features)

input feature matrix

Returns

scipy.sparse matrix of {0, 1}, shape=(n_samples, n_labels)

binary indicator matrix with label assignments

predict_proba(X)

Predict probabilities of label assignments for X

Parameters

Xarray_like, numpy.matrix or scipy.sparse matrix, shape=(n_samples, n_features)

input feature matrix

Returns

scipy.sparse matrix of float in [0.0, 1.0], shape=(n_samples, n_labels)

matrix with label assignment probabilities

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → ClassificationHeterogeneousFeature

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to score.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in score.

Returns

selfobject

The updated object.

class skmultilearn.problem_transform.ClassifierChain(classifier=None, require_dense=None, order=None)

Bases: ProblemTransformationBase

Constructs a bayesian conditioned chain of per label classifiers

This class provides implementation of Jesse Read’s problem transformation method called Classifier Chains. For L labels it trains L classifiers ordered in a chain according to the Bayesian chain rule.

The first classifier is trained just on the input space, and then each next classifier is trained on the input space and all previous classifiers in the chain.

The default classifier chains follow the same ordering as provided in the training set, i.e. label in column 0, then 1, etc.

Parameters

classifierBaseEstimator

scikit-learn compatible base classifier

require_dense[bool, bool], optional

whether the base classifier requires dense representations for input features and classes/labels matrices in fit/predict. If value not provided, sparse representations are used if base classifier is an instance of MLClassifierBase and dense otherwise.

orderList[int], permutation of range(n_labels), optional

the order in which the chain should go through labels, the default is range(n_labels)

Attributes

classifiers_List[BaseEstimator] of shape n_labels

list of classifiers trained per partition, set in fit()

References

If used, please cite the scikit-multilearn library and the relevant paper:

@inproceedings{read2009classifier,
  title={Classifier chains for multi-label classification},
  author={Read, Jesse and Pfahringer, Bernhard and Holmes, Geoff and Frank, Eibe},
  booktitle={Joint European Conference on Machine Learning and Knowledge Discovery in Databases},
  pages={254--269},
  year={2009},
  organization={Springer}
}

Examples

An example use case for Classifier Chains with an sklearn.svm.SVC base classifier which supports sparse input:

from skmultilearn.problem_transform import ClassifierChain
from sklearn.svm import SVC

# initialize Classifier Chain multi-label classifier
# with an SVM classifier
# SVM in scikit only supports the X matrix in sparse representation

classifier = ClassifierChain(
    classifier = SVC(),
    require_dense = [False, True]
)

# train
classifier.fit(X_train, y_train)

# predict
predictions = classifier.predict(X_test)

Another way to use this classifier is to select the best scenario from a set of single-label classifiers used with Classifier Chain, this can be done using cross validation grid search. In the example below, the model with highest accuracy results is selected from either a sklearn.naive_bayes.MultinomialNB or sklearn.svm.SVC base classifier, alongside with best parameters for that base classifier.

from skmultilearn.problem_transform import ClassifierChain
from sklearn.model_selection import GridSearchCV
from sklearn.naive_bayes import MultinomialNB
from sklearn.svm import SVC

parameters = [
    {
        'classifier': [MultinomialNB()],
        'classifier__alpha': [0.7, 1.0],
    },
    {
        'classifier': [SVC()],
        'classifier__kernel': ['rbf', 'linear'],
    },
]


clf = GridSearchCV(ClassifierChain(), parameters, scoring='accuracy')
clf.fit(X, y)

print (clf.best_params_, clf.best_score_)

# result
# {'classifier': MultinomialNB(alpha=0.7, class_prior=None, fit_prior=True), 'classifier__alpha': 0.7} 0.16

fit(X, y)

Fits classifier to training data

Parameters

Xarray_like, numpy.matrix or scipy.sparse matrix, shape=(n_samples, n_features)

input feature matrix

yarray_like, numpy.matrix or scipy.sparse matrix of {0, 1}, shape=(n_samples, n_labels)

binary indicator matrix with label assignments

Returns

self

fitted instance of self

Notes

Note

Input matrices are converted to sparse format internally if a numpy representation is passed

predict(X)

Predict labels for X

Parameters

Xarray_like, numpy.matrix or scipy.sparse matrix, shape=(n_samples, n_features)

input feature matrix

Returns

scipy.sparse matrix of {0, 1}, shape=(n_samples, n_labels)

binary indicator matrix with label assignments

predict_proba(X)

Predict probabilities of label assignments for X

Parameters

Xarray_like, numpy.matrix or scipy.sparse matrix, shape=(n_samples, n_features)

input feature matrix

Returns

scipy.sparse matrix of float in [0.0, 1.0], shape=(n_samples, n_labels)

matrix with label assignment probabilities

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → ClassifierChain

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to score.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in score.

Returns

selfobject

The updated object.

class skmultilearn.problem_transform.InstanceBasedLogisticRegression(classifier=LogisticRegression(), require_dense=None)

Bases: ProblemTransformationBase

Combining Instance-Based Learning and Logistic Regression

This idea is put into practice by means of a learning algorithm that realizes instance-based classification as logistic regression, using the information coming from the neighbors of an instance x as a “feature”.

The first classifier layer is filled with K-Nearest Neighbor models, while the second classifier layer is filled with another classifiers, Logistic Regression as default.

Parameters

require_dense[bool, bool], optional

whether the base classifier requires dense representations for input features and classes/labels matrices in fit/predict. If value not provided, sparse representations are used if base classifier is an instance of MLClassifierBase and dense otherwise.

Attributes

model_count_int

number of trained models, in this classifier equal to n_labels

partition_List[List[int]], shape=(model_count_,)

list of lists of label indexes, used to index the output space matrix, set in _generate_partition() via fit()

classifiers_List[BaseEstimator] of shape model_count

list of classifiers trained per partition, set in fit()

knn_layer_List[BaseEstimator] of shape model_count

list of classifiers trained per partition in first layer, set in fit()

References

If used, please cite the scikit-multilearn library and the relevant paper:

@article{cheng2009combining,
    title={Combining instance-based learning and logistic regression for multilabel classification},
    author={Cheng, Weiwei and H{"u}llermeier, Eyke},
    journal={Machine Learning},
    volume={76},
    number={2-3},
    pages={211--225},
    year={2009},
    publisher={Springer}
}

Examples

An example use case for Classifier Chains with an sklearn.svm.SVC base classifier which supports sparse input:

from skmultilearn.problem_transform import InstanceBasedLogisticRegression
from sklearn.svm import SVC

# initialize Instance-Based Learning and Logistic Regression multi-label classifier
# with an SVM classifier
# SVM in scikit only supports the X matrix in sparse representation

classifier = InstanceBasedLogisticRegression(
    classifier = SVC(),
    require_dense = [False, True]
)

# train
classifier.fit(X_train, y_train)

# predict
predictions = classifier.predict(X_test)

Another way to use this classifier is to select the best scenario from a set of single-label classifiers used with Instance-Based Learning and Logistic Regression, this can be done using cross validation grid search. In the example below, the model with highest accuracy results is selected from either a sklearn.naive_bayes.MultinomialNB or sklearn.svm.SVC base classifier, alongside with best parameters for that base classifier.

from skmultilearn.problem_transform import InstanceBasedLogisticRegression
from sklearn.model_selection import GridSearchCV
from sklearn.naive_bayes import MultinomialNB
from sklearn.svm import SVC

parameters = [
    {
        'classifier': [MultinomialNB()],
        'classifier__alpha': [0.7, 1.0],
    },
    {
        'classifier': [SVC()],
        'classifier__kernel': ['rbf', 'linear'],
    },
]

# Create a ClassificationHeterogeneousFeature instance with GridSearchCV
clf = GridSearchCV(InstanceBasedLogisticRegression(), parameters, scoring='accuracy')
clf.fit(X, y)

print(clf.best_params_, clf.best_score_)

# Example output
# {'classifier': MultinomialNB(alpha=0.7), 'classifier__alpha': 0.7} 0.18

fit(X, y)

Fits classifier to training data

Parameters

Xarray_like, numpy.matrix or scipy.sparse matrix, shape=(n_samples, n_features)

input feature matrix

yarray_like, numpy.matrix or scipy.sparse matrix of {0, 1}, shape=(n_samples, n_labels)

binary indicator matrix with label assignments

Returns

self

fitted instance of self

Notes

Note

Input matrices are converted to sparse format internally if a numpy representation is passed

predict(X)

Predict labels from X

Parameters

Xarray_like, numpy.matrix or scipy.sparse matrix, shape=(n_samples, n_features)

input feature matrix

Returns

scipy.sparse matrix of {0, 1}, shape=(n_samples, n_labels)

binary indicator matrix with label assignments

predict_proba(X)

Predict probabilities of each labels from given X

Parameters

Xarray_like, numpy.matrix or scipy.sparse matrix, shape=(n_samples, n_features)

input feature matrix

Returns

scipy.sparse matrix of float in [0.0, 1.0], shape=(n_samples, n_labels)

matrix with label assignment probabilities

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → InstanceBasedLogisticRegression

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to score.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in score.

Returns

selfobject

The updated object.

class skmultilearn.problem_transform.LabelPowerset(classifier=None, require_dense=None)

Bases: ProblemTransformationBase

Transform multi-label problem to a multi-class problem

Label Powerset is a problem transformation approach to multi-label classification that transforms a multi-label problem to a multi-class problem with 1 multi-class classifier trained on all unique label combinations found in the training data.

The method maps each combination to a unique combination id number, and performs multi-class classification using the classifier as multi-class classifier and combination ids as classes.

Parameters

classifierBaseEstimator

scikit-learn compatible base classifier

require_dense[bool, bool], optional

whether the base classifier requires dense representations for input features and classes/labels matrices in fit/predict. If value not provided, sparse representations are used if base classifier is an instance of skmultilearn.base.MLClassifierBase and dense otherwise.

Attributes

unique_combinations_Dict[str, int]

mapping from label combination as string to label combination id transform:() via fit()

reverse_combinations_List[List[int]]

label combination id ordered list to list of label indexes for a given combination transform:() via fit()

Notes

Note

n_classes in this document denotes the number of unique label combinations present in the training y passed to fit(), in practice it is equal to len(self.unique_combinations)

Examples

An example use case for Label Powerset with an sklearn.ensemble.RandomForestClassifier base classifier which supports sparse input:

from skmultilearn.problem_transform import LabelPowerset
from sklearn.ensemble import RandomForestClassifier

# initialize LabelPowerset multi-label classifier with a RandomForest
classifier = LabelPowerset(
    classifier = RandomForestClassifier(n_estimators=100),
    require_dense = [False, True]
)

# train
classifier.fit(X_train, y_train)

# predict
predictions = classifier.predict(X_test)

Another way to use this classifier is to select the best scenario from a set of multi-class classifiers used with Label Powerset, this can be done using cross validation grid search. In the example below, the model with highest accuracy results is selected from either a sklearn.ensemble.RandomForestClassifier or sklearn.naive_bayes.MultinomialNB base classifier, alongside with best parameters for that base classifier.

from skmultilearn.problem_transform import LabelPowerset
from sklearn.model_selection import GridSearchCV
from sklearn.naive_bayes import MultinomialNB
from sklearn.ensemble import RandomForestClassifier

parameters = [
    {
        'classifier': [MultinomialNB()],
        'classifier__alpha': [0.7, 1.0],
    },
    {
        'classifier': [RandomForestClassifier()],
        'classifier__criterion': ['gini', 'entropy'],
        'classifier__n_estimators': [10, 20, 50],
    },
]

clf = GridSearchCV(LabelPowerset(), parameters, scoring='accuracy')
clf.fit(x, y)

print (clf.best_params_, clf.best_score_)

# result
# {
#   'classifier': RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',
#             max_depth=None, max_features='auto', max_leaf_nodes=None,
#             min_impurity_decrease=0.0, min_impurity_split=None,
#             min_samples_leaf=1, min_samples_split=2,
#             min_weight_fraction_leaf=0.0, n_estimators=50, n_jobs=1,
#             oob_score=False, random_state=None, verbose=0,
#             warm_start=False), 'classifier__criterion': 'gini', 'classifier__n_estimators': 50
# } 0.16

fit(X, y)

Fits classifier to training data

Parameters

Xarray_like, numpy.matrix or scipy.sparse matrix, shape=(n_samples, n_features)

input feature matrix

yarray_like, numpy.matrix or scipy.sparse matrix of {0, 1}, shape=(n_samples, n_labels)

binary indicator matrix with label assignments

Returns

self

fitted instance of self

Notes

Note

Input matrices are converted to sparse format internally if a numpy representation is passed

inverse_transform(y)

Transforms multi-class assignment to multi-label

Transforms a mutli-label problem into a single-label multi-class problem where each label combination is a separate class.

Parameters

ynumpy.ndarray of {0, … , n_classes-1}, shape=(n_samples,)

binary indicator matrix with label assignments

Returns

scipy.sparse matrix of {0, 1}, shape=(n_samples, n_labels)

binary indicator matrix with label assignments

predict(X)

Predict labels for X

Parameters

Xarray_like, numpy.matrix or scipy.sparse matrix, shape=(n_samples, n_features)

input feature matrix

Returns

scipy.sparse matrix of {0, 1}, shape=(n_samples, n_labels)

binary indicator matrix with label assignments

predict_proba(X)

Predict probabilities of label assignments for X

Parameters

Xarray_like, numpy.matrix or scipy.sparse matrix, shape=(n_samples, n_features)

input feature matrix

Returns

scipy.sparse matrix of float in [0.0, 1.0], shape=(n_samples, n_labels)

matrix with label assignment probabilities

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → LabelPowerset

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to score.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in score.

Returns

selfobject

The updated object.

transform(y)

Transform multi-label output space to multi-class

Transforms a mutli-label problem into a single-label multi-class problem where each label combination is a separate class.

Parameters

yarray_like, numpy.matrix or scipy.sparse matrix of {0, 1}, shape=(n_samples, n_labels)

binary indicator matrix with label assignments

Returns

numpy.ndarray of {0, … , n_classes-1}, shape=(n_samples,)

a multi-class output space vector

class skmultilearn.problem_transform.ProbabilisticClassifierChain(classifier=None, require_dense=None, order=None)

Bases: ClassifierChain

Probabilistic Classifier Chain for Multi-Label Classification

This class implements a Probabilistic Classifier Chain (PCC), an extension of Jesse Read’s Classifier Chains. It learns a chain of classifiers, each predicting a label conditioned on the input features and the predictions of preceding classifiers in the chain. This approach models joint label distributions to capture label correlations.

Each classifier in the chain is trained on an augmented input space that includes the original features (X) and the predictions of all previous classifiers in the chain.

The implementation adapted and changed from: https://github.com/ChristianSch/skml/blob/master/skml/problem_transformation/probabilistic_classifier_chain.py

Parameters

classifierBaseEstimator

A scikit-learn compatible base classifier. This classifier is used as the base model for each step in the classifier chain.

require_dense[bool, bool], optional

Indicates whether the base classifier requires dense representations for input features and label matrices in fit/predict. If not provided, it defaults to using sparse representations unless the base classifier is an instance of MLClassifierBase, in which case dense representations are used.

orderList[int], permutation of range(n_labels), optional

the order in which the chain should go through labels, the default is range(n_labels)

Attributes

classifiers_List[BaseEstimator] of shape n_labels

A list of classifiers, one for each label, trained per partition as per the chain order.

References

If using this implementation, please cite the scikit-multilearn library and the relevant paper:

@inproceedings{Dembczynski2010BayesOM,
  title={Bayes Optimal Multilabel Classification via Probabilistic Classifier Chains},
  author={Krzysztof Dembczynski and Weiwei Cheng and Eyke H{"u}llermeier},
  booktitle={International Conference on Machine Learning},
  year={2010},
  url={https://api.semanticscholar.org/CorpusID:6418797}
}

Examples

An example of using Probabilistic Classifier Chain with an sklearn.svm.SVC:

from skmultilearn.problem_transform import ProbabilisticClassifierChain
from sklearn.svm import SVC

# Initialize Probabilistic Classifier Chain with an SVM classifier
classifier = ProbabilisticClassifierChain(
    classifier = SVC(probability=True),
    require_dense = [False, True]
)

# Train
classifier.fit(X_train, y_train)

# Predict
predictions = classifier.predict(X_test)

To optimize the classifier chain with grid search, one can vary both the base classifier and its parameters:

from skmultilearn.problem_transform import ProbabilisticClassifierChain
from sklearn.model_selection import GridSearchCV
from sklearn.naive_bayes import MultinomialNB
from sklearn.svm import SVC

parameters = [
    {
        'classifier': [MultinomialNB()],
        'classifier__alpha': [0.7, 1.0],
    },
    {
        'classifier': [SVC(probability=True)],
        'classifier__kernel': ['rbf', 'linear'],
    },
]

clf = GridSearchCV(ProbabilisticClassifierChain(), parameters, scoring='accuracy')
clf.fit(X, y)

print(clf.best_params_, clf.best_score_)

predict(X)

Predict labels for X

Parameters

Xarray_like, numpy.matrix or scipy.sparse matrix, shape=(n_samples, n_features)

input feature matrix

Returns

scipy.sparse matrix of {0, 1}, shape=(n_samples, n_labels)

binary indicator matrix with label assignments

predict_proba(X)

Predict probabilities of label assignments for X

Parameters

Xarray_like, numpy.matrix or scipy.sparse matrix, shape=(n_samples, n_features)

input feature matrix

Returns

scipy.sparse matrix of float in [0.0, 1.0], shape=(n_samples, n_labels)

matrix with label assignment probabilities

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → ProbabilisticClassifierChain

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to score.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in score.

Returns

selfobject

The updated object.

class skmultilearn.problem_transform.StructuredGridSearchCV(estimator, param_grid, scoring=None, n_jobs=None, refit=True, cv=None, verbose=0, pre_dispatch='2*n_jobs', error_score=nan, return_train_score=False, print_best_param=False)

Bases: ProblemTransformationBase

Hyperparameter tuning per each label classifier

As original GridSearchCV provided by scikit-learn ignores BR&CC structural property, it cannot search best parameter and classifier for each label. Therefore, StructuredGridSearchCV was implemented for fine tuning with considering structural property. StructuredGridSearchCV searches best classifier with optimal hyper-parameters for each labels.

StructuredGridSearchCV provides “fit”, “predict”, “predict_proba” as its methods. It provides list of optimal classifiers with fine-tuned hyper-parameters via find_optm_classifier function.

If print_best_param is True, find_optm_classifier function prints best parameter for each label.

Parameters

Same as GridsearchCV in scikit-learn print_best_param : bool, default = False

whether print best parameter each label classifier or not

Attributes

classifiers_List[BaseEstimator] of shape n_labels

list of classifiers trained per partition, set in fit()

print_best_parambool, default = False

whether print best parameter each label classifier or not

estimatorestimator object.

This is assumed to implement the scikit-learn estimator interface. Either estimator needs to provide a score function, or scoring must be passed.

classifier_numint

The index which corresponds to each label classifier

Examples

An example use case for Structured GridSearchCV:

from skmultilearn.problem_transform import StructuredGridSearchCV, BinaryRelevance
from sklearn.naive_bayes import MultinomialNB

# Define parameter grid
parameters = {'classifier': MultinomialNB(), 'alpha':[1, 10]}

# Create StructuredGridSearchCV instance
structured_grid_search = StructuredGridSearchCV(
    estimator=BinaryRelevance(),
    param_grid=parameters,
    scoring='accuracy',
    print_best_param=True
)

# Fit and find optimal classifiers
structured_grid_search.fit(X_train, y_train)
optimal_classifiers = structured_grid_search.find_optm_classifiers(X, y)

find_optm_classifiers(X, y)

Find optimal classifier per label

Parameters

Xarray_like, numpy.matrix or scipy.sparse matrix, shape=(n_samples, n_features)

input feature matrix

yarray_like, numpy.matrix or scipy.sparse matrix of {0, 1}, shape=(n_samples, n_labels)

binary indicator matrix with label assignments

Returns

optimized_clfs_

list of optimal classifier per label

Notes

Note

Input matrices are converted to sparse format internally if a numpy representation is passed

fit(X, y)

Fits classifier to training data and finds optimal classifier

Parameters

Xarray_like, numpy.matrix or scipy.sparse matrix, shape=(n_samples, n_features)

input feature matrix

yarray_like, numpy.matrix or scipy.sparse matrix of {0, 1}, shape=(n_samples, n_labels)

binary indicator matrix with label assignments

Returns

self

fitted instance of self

Notes

Note

Input matrices are converted to sparse format internally if a numpy representation is passed

predict(X)

Predict labels for X using optimal classifiers

Parameters

Xarray_like, numpy.matrix or scipy.sparse matrix, shape=(n_samples, n_features)

input feature matrix

Returns

scipy.sparse matrix of {0, 1}, shape=(n_samples, n_labels)

binary indicator matrix with label assignments

predict_proba(X)

Predict probabilities of label assignments for X using optimal classifiers

Parameters

Xarray_like, numpy.matrix or scipy.sparse matrix, shape=(n_samples, n_features)

input feature matrix

Returns

scipy.sparse matrix of float in [0.0, 1.0], shape=(n_samples, n_labels)

matrix with label assignment probabilities

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → StructuredGridSearchCV

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to score.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in score.

Returns

selfobject

The updated object.

---

Cite us

If you use scikit-multilearn-ng in your research and publish it, please consider citing scikit-multilearn:

@ARTICLE{2017arXiv170201460S,
    author = {{Szyma{'n}ski}, P. and {Kajdanowicz}, T.},
    title = "{A scikit-based Python environment for performing multi-label classification}",
    journal = {ArXiv e-prints},
    archivePrefix = "arXiv",
    eprint = {1702.01460},
    primaryClass = "cs.LG",
    keywords = {Computer Science - Learning, Computer Science - Mathematical Software},
    year = 2017,
    month = feb,
}