What does calling fit() multiple times on the same model do?

Question

After I instantiate a scikit model (e.g. LinearRegression), if I call its fit() method multiple times (with different X and y data), what happens? Does it fit the model on the data like if I just re-instantiated the model (i.e. from scratch), or does it keep into accounts data already fitted from the previous call to fit()?

Trying with LinearRegression (also looking at its source code) it seems to me that every time I call fit(), it fits from scratch, ignoring the result of any previous call to the same method. I wonder if this true in general, and I can rely on this behavior for all models/pipelines of scikit learn.

MaxU - stand with Ukraine · Accepted Answer · 2021-03-04T11:09:13.927

90

If you will execute model.fit(X_train, y_train) for a second time - it'll overwrite all previously fitted coefficients, weights, intercept (bias), etc.

If you want to fit just a portion of your data set and then to improve your model by fitting a new data, then you can use estimators, supporting "Incremental learning" (those, that implement partial_fit() method)

edited Mar 04 '21 at 11:09

answered Apr 15 '18 at 11:24

MaxU - stand with Ukraine

205,989
36
386
419

16

Some estimators (having the `warm_start` parameter) will reuse the solutions from the previous calls to `fit()` as initial solution in new call when `warm_start = True`. – Vivek Kumar Apr 16 '18 at 10:46
3

@VivekKumar, yes, this is true. But for "linear model" algorithms (like LogisticRegression) the previously fitted coefficients will be used just for the __initialization__ step, so they will/might be overwritten as well – MaxU - stand with Ukraine Apr 16 '18 at 10:55
1

can we use partial_fit in KNN? – Prvt_Yadav Apr 04 '19 at 04:48
2

Broken link, I believe the new location is https://scikit-learn.org/0.15/modules/scaling_strategies.html – LeeNeverGup Mar 04 '21 at 11:04
1

@LeeNeverGup, thank you! I've updated the link in the answer) – MaxU - stand with Ukraine Mar 04 '21 at 11:09
In my case, I want to `IterativeImputer.fit_transform(x)` before I use `model.fit(x,y)`. Is this a good way to use? – curiouscheese May 11 '22 at 03:03

score 7 · Answer 2 · answered Apr 15 '18 at 11:50

You can use term fit() and train() word interchangeably in machine learning. Based on classification model you have instantiated, may be a clf = GBNaiveBayes() or clf = SVC(), your model uses specified machine learning technique.
And as soon as you call clf.fit(features_train, label_train) your model starts training using the features and labels that you have passed.

you can use clf.predict(features_test) to predict.
If you will again call clf.fit(features_train2, label_train2) it will start training again using passed data and will remove the previous results. Your model will reset the following inside model:

Weights
Fitted Coefficients
Bias
And other training related stuff...

You can use partial_fit() method as well if you want your previous calculated stuff to stay and additionally train using next data

I want to implement imputation using IterativeImputer from sklearn, before training the data. To impute the data, I need to `fit_transform` (x-feature) than after that need to `fit ` it again for modelling. Can I use fit again for modelling? Is this a good method to use? — curiouscheese, May 11 '22 at 03:00

score 1 · Answer 3 · answered Sep 23 '22 at 10:40

Beware that the model is passed kind of "by reference". Here, model1 will be overwritten:

df1 = pd.DataFrame(np.random.rand(100).reshape(10,10))
df2 = df1.copy()
df2.iloc[0,0] = df2.iloc[0,0] -2 # change one value

pca = PCA()
model1 = pca.fit(df)
model2 = pca.fit(df2)

np.unique(model1.explained_variance_ == model2.explained_variance_)

Returns

array([ True])

To avoid this use

from copy import deepcopy
model1 = deepcopy(pca.fit(df))

What does calling fit() multiple times on the same model do?

3 Answers3

Linked