Handling label encoding, transformation and estimation in one object

Question

Similar : Pipeline doesn't work with Label Encoder

I'd like to have an object that handles label encoding (in my case with a LabelEncoder), transformation and estimation. It is important to me that all theses functions can be executed through only one object.

I've tried using a pipeline this way :

import numpy as np
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import LabelEncoder

# mock training dataset
X = np.random.rand(1000, 100)
y = np.concatenate([["label1"] * 300, ["label2"] * 300, ["label3"] * 400])

le = LabelEncoder()
ss = StandardScaler()
clf = MyClassifier()
pl = Pipeline([('encoder', le),
               ('scaler', ss),
               ('clf', clf)])
pl.fit(X, y)

Which gives :

File "sklearn/pipeline.py", line 581, in _fit_transform_one
    res = transformer.fit_transform(X, y, **fit_params)
TypeError: fit_transform() takes exactly 2 arguments (3 given)

Clarifications :

• X and y are my training dataset, X being the values and y the targeted labels.

• X is a numpy.ndarray of shape (n_sample, n_features) and of type float, values ranging from 0 to 1.

• y is a numpy.ndarray of shape (n_sample,) and of type string

• I expect LabelEncoder to encode y, not X.

• I need y only for MyClassifier, and I need it encoded to integers for MyClassifier to work.

After some thoughts and facing the error above, I feel like it was naive to think that Pipeline could handle it. I figured out that Pipeline could very well handle my transformation and classifier together but it was the label encoding part that would fail.

What is the correct way to achieve what I want ? By correct I mean to do something that would allow reusability and some kind of consistency with sklearn. Is there a class in sklearn library that do what I want ?

I'm pretty surprised I haven't found an answer browsing the web because I feel like what I'm doing is nothing uncommon. I might be missing something here.

Answer 1

As Vivek Kumar wrote in the comments :

LabelEncoder will be automatically called on y when you call clf.fit(). So you dont need to worry about it. y can have integers, strings as classes, that will be handled correctly by the estimators in scikit. So there is no need to include LabelEncoder in the pipeline to work on y.

So here's a solution to my problem :

import numpy as np
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline

# mock training dataset
X = np.random.rand(1000, 100)
y = np.concatenate([["label1"] * 300, ["label2"] * 300, ["label3"] * 400])

ss = StandardScaler()
clf = MyClassifier()  # my own classifier
pl = Pipeline([('scaler', ss),
               ('clf', clf)])
pl.fit(X, y)

Only difference, now pl.predict(X) will return an array of strings containing the values "label1", "label2" or "label3" (which makes sense, since that's what we fed him with).

If needed, to get back the LabelEncoder that is used automatically by sklearn.pipeline, you can do :

from sklearn.preprocessing import LabelEncoder
le = LabelEncoder(pl.classes_)

Which would give me a copy of the label encoder used by the Pipeline pl.

Answer 2

I believe this isn't possible.

Firstly all transformers inherit from the sklearn.base.TransformerMixin. The fit_transform method takes X and optionally y arguments, but only returns X_new. scikit-learn isn't designed with this kind of transformation in mind.

Secondly, LabelEncoder would fail in a pipeline because fit and transform only accept one argument, y, not X, y.

In the end I wrote a function to do lookup in an Enum mapping string labels to integer labels. At least then the transformation is in code and trackable using version control.

Answer 3

I have implemented the categorical encoding with pandas, and as a classifier I have used SGDClassifier. Your code above calls MyClassifier(), but it is not defined inside the code itself.

import numpy as np
import pandas as pd
# from sklearn.preprocessing import LabelEncoder # No longer used
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.linear_model import SGDClassifier

X = np.random.randn(1000, 10)

y_initial = np.concatenate([["label1"] * 300, ["label2"] * 300, ["label3"] * 400])

df = pd.DataFrame({'y':y_initial})
df['y'] = df['y'].astype('category') # Same as the output of LabelEncoder

ss = StandardScaler()
clf = SGDClassifier()

y = df['y']

pl = Pipeline([('scaler', ss),
               ('clf', clf)])

pl.fit(X,y)

The output is the fit pipeline object:

Pipeline(memory=None,
     steps=[('scaler', StandardScaler(copy=True, with_mean=True, with_std=True)), ('clf', SGDClassifier(alpha=0.0001, average=False, class_weight=None, epsilon=0.1,
       eta0=0.0, fit_intercept=True, l1_ratio=0.15,
       learning_rate='optimal', loss='hinge', max_iter=None, n_iter=None,
       n_jobs=1, penalty='l2', power_t=0.5, random_state=None,
       shuffle=True, tol=None, verbose=0, warm_start=False))])