0

This is a really basic question, but I haven't found the answer on other sites, so I am kind of forced to ask about it here.

I fitted my "classif.ranger" learner usin benchmark(design,store_models) function form mlr3 library and I need to acces the fitted parameters (obv). I found nothing about it in the benchmark documentation, so I tried to do it the hard way: -> I set store_models to TRUE -> I tried to acces the model using fitted(), but it returned NULL.

I know the question is basic and that I probably am doing smth stupid (for ex. misreading the documentation or smth like that) but I just have no idea of how to acctualy access the parameters... please help.

If it is needed in such (probably) trivial situation, here comes the code:

library(data.table)
library(mlr3)
library(mlr3learners)
library(mlr3filters)
library(mlr3fselect)
library(mlr3tuning)
library(ranger)
library(paradox)

Here is some code that is irrelevant to the question Now the relevant code:

measure = msr("classif.auc")

tuner = tnr("random_search")

ranger_space = ParamSet$new(list(
  ParamInt$new("num.trees", lower = 700, upper = 2000),
  ParamInt$new("mtry", lower = 1, upper = 15)
))

rf_learner <- lrn("classif.ranger", predict_type = "prob")

at = AutoTuner$new(
    learner = rf_learner,
    resampling = rsmp("holdout"),
    measure = measure,
    search_space = ranger_space,
    terminator = trm("evals", n_evals = 25),
    tuner = tuner
  )

pred_task <- TaskClassif$new(id = "predict", backend = dataSet, target = "will_it_sell")

grid = benchmark_grid(
    task = pred_task,
    learner = list(at, rf_learner),
    resampling = rsmp("cv", folds = 3)
  )
  
rf_benchmark = benchmark(design = grid, store_models = TRUE)
result = rf_benchmark$aggregate(measure)
result
Phil
  • 7,287
  • 3
  • 36
  • 66
Jakub Radziwilski
  • 3
  • 2
  • 2
    It's easier to help you if you include a simple [reproducible example](https://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example) with sample input and desired output that can be used to test and verify possible solutions. If the problem isn't specific to your data, please make a simple example we can copy/paste to run and test. Please explicitly include the `library()` calls in your code. – MrFlick Jan 21 '21 at 20:50
  • I can't share the data, as I am not allowed to duplicate it, it was given to me simply for studying reasons. I am also not sure how the "reproducible example" should look like in this case... – Jakub Radziwilski Jan 21 '21 at 22:43
  • It doesn't have to be your actual data. Feel free to use example data from the packages themselves or generate random data. You just need to provide runnable code if you really want help. – MrFlick Jan 21 '21 at 22:44

2 Answers2

1

The mlr3book is the documentation for mlr3 and it has a section about your question: https://mlr3book.mlr-org.com/benchmarking.html#bm-resamp

And please do not mix R and RStudio, they are two different things. RStudio never has any function.

pat-s
  • 5,992
  • 1
  • 32
  • 60
0

You can use getBMRModels() to get the models, which will tell you what hyperparameters were used to fit them. See the benchmark section of the documentation.

Lars Kotthoff
  • 107,425
  • 16
  • 204
  • 204