3

I'm trying and failing to pass parameters to a custom estimator in scikit learn. I'd like the parameter lr to change during the gridsearch. Problem is that the lr parameter is not changing...

The code sample is copied and updated from here

(the original code did neither work for me)

Any full working example of GridSearchCV with custom estimator, with changing parameters would be appreciated.

I'm in ubuntu 18.10 using scikit-learn 0.20.2

from sklearn.model_selection import GridSearchCV
from sklearn.base import BaseEstimator, ClassifierMixin
import numpy as np

class MyClassifier(BaseEstimator, ClassifierMixin):

     def __init__(self, lr=0.1):
         # Some code
         print('lr:', lr)
         return self

     def fit(self, X, y):
         # Some code
         return self

     def predict(self, X):
         # Some code
         return X % 3

params = {
    'lr': [0.1, 0.5, 0.7]
}
gs = GridSearchCV(MyClassifier(), param_grid=params, cv=4)

x = np.arange(30)
y = np.concatenate((np.zeros(10), np.ones(10), np.ones(10) * 2))
gs.fit(x, y)

Terveisin, Markus

Venkatachalam
  • 16,288
  • 9
  • 49
  • 77
Markus Kaukonen
  • 334
  • 4
  • 10

1 Answers1

5

You were not able to see the change in lr value since you are printing inside constructor function.

If we print inside .fit() function, we can see the change of lr values. It happens because of the way the different copies of estimators are created. See here to understand the process for creating multiple copies. 

from sklearn.model_selection import GridSearchCV
from sklearn.base import BaseEstimator, ClassifierMixin
import numpy as np

class MyClassifier(BaseEstimator, ClassifierMixin):

    def __init__(self, lr=0):
         # Some code
        print('lr:', lr)
        self.lr = lr

    def fit(self, X, y):
         # Some code
        print('lr:', self.lr)
        return self

    def predict(self, X):
         # Some code
         return X % 3

params = {
    'lr': [0.1, 0.5, 0.7]
}
gs = GridSearchCV(MyClassifier(), param_grid=params, cv=4)

x = np.arange(30)
y = np.concatenate((np.zeros(10), np.ones(10), np.ones(10) * 2))
gs.fit(x, y)
gs.predict(x)

Output:

lr: 0
lr: 0
lr: 0
lr: 0.1
lr: 0
lr: 0.1
lr: 0
lr: 0.1
lr: 0
lr: 0.1
lr: 0
lr: 0.5
lr: 0
lr: 0.5
lr: 0
lr: 0.5
lr: 0
lr: 0.5
lr: 0
lr: 0.7
lr: 0
lr: 0.7
lr: 0
lr: 0.7
lr: 0
lr: 0.7
lr: 0
lr: 0.1
Venkatachalam
  • 16,288
  • 9
  • 49
  • 77
  • So if I want to create an attribute of `MyClassifier` based on some argument value in the constructor, does this mean I can't have this code inside the `__init__()` of `MyClassifier`? For example `if lr != 0: self.nonzero = True else: self.nonzero = False` – Hawklaz Oct 23 '20 at 10:25
  • yes, you cann't have additional parameters defined inside `__init__()`. Btw, you can create these attributes outside constructor. like the `feature_importance_` attribute of decision tree – Venkatachalam Oct 23 '20 at 11:12
  • I see. So I would simply need to call a different method to do run the additional code before returning the value of my new attribute? For example `self.nonzero = check_non_zero_lr()` And `def check_non_zero_lr(): if self.lr != 0: return True else: return False` – Hawklaz Oct 24 '20 at 02:27
  • You can use it as property and not store that into a variable. example [here](https://github.com/scikit-learn/scikit-learn/blob/f1111be2fa9899a610843c36d203b0fab02f16c3/sklearn/ensemble/_voting.py#L51). – Venkatachalam Oct 26 '20 at 06:26
  • If you stil need clarification please ask a fresh question with more details. People would be happy to help you. – Venkatachalam Oct 26 '20 at 06:29
  • 1
    Done. Posted [here](https://stackoverflow.com/q/64532894/11542679) – Hawklaz Oct 26 '20 at 07:22
  • Good. You hav got a great answer. – Venkatachalam Oct 27 '20 at 11:33
  • 1
    Yes, I sorted it out. – Hawklaz Oct 27 '20 at 12:19