diff --git a/docs/src/notebooks/UsingMLJ/01_basics/notebook.md b/docs/src/notebooks/UsingMLJ/01_basics/notebook.md
index afb15fd..662e437 100644
--- a/docs/src/notebooks/UsingMLJ/01_basics/notebook.md
+++ b/docs/src/notebooks/UsingMLJ/01_basics/notebook.md
@@ -11,6 +11,18 @@ Notebook supporting the video series "Using MLJ".
To run the code in this tutorial in a live Julia session, first follow the instructions
given [here](@ref instructions).
+### Video Timings
+
+- 00:00 Introduction
+- 01:05 Goals
+- 01:48 Prerequisites and Getting Help
+- 02:21 Supervised Learning Recap
+- 05:31 Models and Machines
+- 09:28 **Live Coding:** Regression
+- 28:55 Scientific Types
+- 32:29 **Live Coding:** Scitypes and Classification
+- 55:45 End
+
We start by inspecting the packages, and their exact versions, in the currently active
package environment:
@@ -30,7 +42,7 @@ Info Packages marked with ⌃ have new versions available and may be upgradable.
````
-# Part I. Regression
+## Part I. Regression
````@julia
using MLJ
@@ -187,8 +199,8 @@ fitted_params(mach)
````
(forest = Ensemble of Decision Trees
Trees: 100
-Avg Leaves: 147.25
-Avg Depth: 14.49,)
+Avg Leaves: 146.21
+Avg Depth: 14.61,)
````
````@julia
@@ -207,9 +219,9 @@ predict(mach, X)[1:3]
````
3-element Vector{Float64}:
- 26.984
- 22.682999999999993
- 35.31
+ 27.011
+ 22.410999999999994
+ 34.27
````
Predict in the `test` rows:
@@ -225,7 +237,7 @@ mae(ypred, y[test])
````
````
-4.410905940594058
+4.574237623762377
````
`mae` is actually just an alias:
@@ -272,14 +284,14 @@ PerformanceEvaluation object with these fields:
per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: RandomForestRegressor-162
+Tag: RandomForestRegressor-336
Extract:
┌────────────┬───────────┬─────────────┐
│ measure │ operation │ measurement │
├────────────┼───────────┼─────────────┤
-│ LPLoss( │ predict │ 4.41 │
+│ LPLoss( │ predict │ 4.57 │
│ p = 1) │ │ │
-│ RSquared() │ predict │ 0.369 │
+│ RSquared() │ predict │ 0.33 │
└────────────┴───────────┴─────────────┘
Apply `describe` to this result for a named tuple summary.
````
@@ -301,21 +313,21 @@ PerformanceEvaluation object with these fields:
per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: RandomForestRegressor-267
+Tag: RandomForestRegressor-817
Extract:
┌───┬────────────┬───────────┬─────────────┬─────────┐
│ │ measure │ operation │ measurement │ 1.96*SE │
├───┼────────────┼───────────┼─────────────┼─────────┤
-│ A │ LPLoss( │ predict │ 2.99 │ 0.92 │
+│ A │ LPLoss( │ predict │ 3.05 │ 0.89 │
│ │ p = 1) │ │ │ │
-│ B │ RSquared() │ predict │ 0.67 │ 0.17 │
+│ B │ RSquared() │ predict │ 0.65 │ 0.16 │
└───┴────────────┴───────────┴─────────────┴─────────┘
-┌───┬──────────────────────────────────────────┐
-│ │ per_fold │
-├───┼──────────────────────────────────────────┤
-│ A │ [2.14, 2.11, 3.32, 2.5, 4.9, 2.96] │
-│ B │ [0.74, 0.871, 0.73, 0.813, 0.454, 0.392] │
-└───┴──────────────────────────────────────────┘
+┌───┬─────────────────────────────────────────┐
+│ │ per_fold │
+├───┼─────────────────────────────────────────┤
+│ A │ [2.27, 2.36, 3.25, 2.48, 4.97, 2.96] │
+│ B │ [0.723, 0.81, 0.742, 0.806, 0.441, 0.4] │
+└───┴─────────────────────────────────────────┘
Apply `describe` to this result for a named tuple summary.
````
@@ -338,21 +350,21 @@ PerformanceEvaluation object with these fields:
per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: RandomForestRegressor-760
+Tag: RandomForestRegressor-621
Extract:
┌───┬────────────┬───────────┬─────────────┬─────────┐
│ │ measure │ operation │ measurement │ 1.96*SE │
├───┼────────────┼───────────┼─────────────┼─────────┤
-│ A │ LPLoss( │ predict │ 2.49 │ 0.14 │
+│ A │ LPLoss( │ predict │ 2.48 │ 0.1 │
│ │ p = 1) │ │ │ │
-│ B │ RSquared() │ predict │ 0.823 │ 0.033 │
+│ B │ RSquared() │ predict │ 0.817 │ 0.038 │
└───┴────────────┴───────────┴─────────────┴─────────┘
-┌───┬────────────────────────────────────────────────────────────────────────┐
-│ │ per_fold │
-├───┼────────────────────────────────────────────────────────────────────────┤
-│ A │ [2.64, 2.35, 2.17, 2.66, 2.86, 2.27, 2.67, 2.47, 2.54, 2.28] │
-│ B │ [0.752, 0.874, 0.897, 0.758, 0.777, 0.853, 0.835, 0.792, 0.824, 0.864] │
-└───┴────────────────────────────────────────────────────────────────────────┘
+┌───┬──────────────────────────────────────────────────────────────────────┐
+│ │ per_fold │
+├───┼──────────────────────────────────────────────────────────────────────┤
+│ A │ [2.37, 2.74, 2.27, 2.42, 2.75, 2.54, 2.36, 2.48, 2.41, 2.5] │
+│ B │ [0.87, 0.702, 0.857, 0.848, 0.824, 0.72, 0.843, 0.834, 0.839, 0.834] │
+└───┴──────────────────────────────────────────────────────────────────────┘
Apply `describe` to this result for a named tuple summary.
````
@@ -362,8 +374,8 @@ e.uncertainty_radius_95
````
2-element Vector{Float64}:
- 0.14383105341435692
- 0.032989951374955834
+ 0.10349121844149421
+ 0.03773282475420102
````
# Interlude on scientific types
@@ -400,7 +412,7 @@ scitype(["cat", "mouse", "dog"])
AbstractVector{Textual} (alias for AbstractArray{ScientificTypesBase.Textual, 1})
````
-# Part II. Classification
+## Part II. Classification
New data set for classification, the Adult Dataset (census data):
@@ -608,8 +620,8 @@ first(yprob, 5)
5-element CategoricalDistributions.UnivariateFiniteVector{ScientificTypesBase.Multiclass{2}, InlineStrings.String7, UInt32, Float64}:
UnivariateFinite{ScientificTypesBase.Multiclass{2}}(<=50K=>1.0, >50K=>0.0)
UnivariateFinite{ScientificTypesBase.Multiclass{2}}(<=50K=>1.0, >50K=>0.0)
- UnivariateFinite{ScientificTypesBase.Multiclass{2}}(<=50K=>0.91, >50K=>0.09)
- UnivariateFinite{ScientificTypesBase.Multiclass{2}}(<=50K=>0.33, >50K=>0.67)
+ UnivariateFinite{ScientificTypesBase.Multiclass{2}}(<=50K=>0.89, >50K=>0.11)
+ UnivariateFinite{ScientificTypesBase.Multiclass{2}}(<=50K=>0.28, >50K=>0.72)
UnivariateFinite{ScientificTypesBase.Multiclass{2}}(<=50K=>1.0, >50K=>0.0)
````
@@ -620,8 +632,8 @@ yprob[3]
````
UnivariateFinite{ScientificTypesBase.Multiclass{2}}
┌ ┐
- <=50K ┤■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 0.91
- >50K ┤■■■ 0.09
+ <=50K ┤■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 0.89
+ >50K ┤■■■■ 0.11
└ ┘
````
@@ -632,7 +644,7 @@ accuracy(ypoint, y[test])
````
````
-0.7823616727235502
+0.7821057480677689
````
````@julia
@@ -640,7 +652,7 @@ log_loss(yprob, y[test])
````
````
-1.5655836383744342
+1.53133125245395
````
Evaluate with one command:
@@ -660,12 +672,12 @@ PerformanceEvaluation object with these fields:
per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: RandomForestClassifier-308
+Tag: RandomForestClassifier-321
Extract:
┌──────────────────────┬──────────────┬─────────────┐
│ measure │ operation │ measurement │
├──────────────────────┼──────────────┼─────────────┤
-│ Accuracy() │ predict_mode │ 0.781 │
+│ Accuracy() │ predict_mode │ 0.782 │
│ LogLoss( │ predict │ 1.55 │
│ tol = 2.22045e-16) │ │ │
└──────────────────────┴──────────────┴─────────────┘
diff --git a/docs/src/notebooks/UsingMLJ/02_model_composition/notebook.md b/docs/src/notebooks/UsingMLJ/02_model_composition/notebook.md
index 54350fb..7dde1c6 100644
--- a/docs/src/notebooks/UsingMLJ/02_model_composition/notebook.md
+++ b/docs/src/notebooks/UsingMLJ/02_model_composition/notebook.md
@@ -11,6 +11,20 @@ Notebook supporting the video series "Using MLJ".
To run the code in this tutorial in a live Julia session, first follow the instructions
given [here](@ref instructions).
+### Video Timings
+
+- 00:00 Introduction
+- 00:08 Goals
+- 01:19 Prerequisites and Getting Help
+- 02:00 Composite Models Defined
+- 04:13 Model Pipelines
+- 05:59 Data Leakage
+- 08:37 Target Transformations
+- 12:43 Live Coding: Pipelines
+- 18:42 Live Coding: Target Transformations
+- 20:35 Other Model Wrappers
+- 25:02 End
+
````@julia
using MLJ
````
@@ -146,7 +160,7 @@ PerformanceEvaluation object with these fields:
per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: DeterministicPipeline-639
+Tag: DeterministicPipeline-500
Extract:
┌──────────┬───────────┬─────────────┬─────────┐
│ measure │ operation │ measurement │ 1.96*SE │
@@ -211,7 +225,7 @@ PerformanceEvaluation object with these fields:
per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: TransformedTargetModelDeterministic-399
+Tag: TransformedTargetModelDeterministic-470
Extract:
┌──────────┬───────────┬─────────────┬─────────┐
│ measure │ operation │ measurement │ 1.96*SE │
@@ -254,7 +268,10 @@ evaluations = evaluate(
PerformanceEvaluation("new lambda & normalized target", 18460.0 ± 440.0)
````
-Here's a pretty view of these results:
+(To provide multiple models, tagged with strings, to `evaluate` requires MLJBase 1.12.0
+or higher.)
+
+Here's a pretty view of these results (needs MLJBase 1.13.0 or higher):
````@julia
describe.(evaluations) |> pretty
diff --git a/docs/src/notebooks/UsingMLJ/03_model_tuning/learning_curve.svg b/docs/src/notebooks/UsingMLJ/03_model_tuning/learning_curve.svg
index 1687109..1f9c598 100644
--- a/docs/src/notebooks/UsingMLJ/03_model_tuning/learning_curve.svg
+++ b/docs/src/notebooks/UsingMLJ/03_model_tuning/learning_curve.svg
@@ -1,46 +1,46 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/src/notebooks/UsingMLJ/03_model_tuning/notebook.md b/docs/src/notebooks/UsingMLJ/03_model_tuning/notebook.md
index 286e160..a6661c2 100644
--- a/docs/src/notebooks/UsingMLJ/03_model_tuning/notebook.md
+++ b/docs/src/notebooks/UsingMLJ/03_model_tuning/notebook.md
@@ -11,6 +11,19 @@ Notebook supporting the video series "Using MLJ".
To run the code in this tutorial in a live Julia session, first follow the instructions
given [here](@ref instructions).
+### Video Timings
+
+- 00:00 Introduction
+- 00:06 Goals
+- 01:12 Prerequisites and Getting Help
+- 01:27 Live Coding: Learning Curves
+- 13:27 Tuning as Model Wrapper
+- 18:45 Live Coding: Grid Search
+- 26:17 Live Coding: Random Search
+- 30:48 Nested Resampling
+- 33:11 Final Observations
+- 36:59 End
+
## Part I. Learning Curves
````@julia
@@ -81,7 +94,7 @@ PerformanceEvaluation object with these fields:
per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: ProbabilisticPipeline-908
+Tag: ProbabilisticPipeline-489
Extract:
┌──────────┬──────────────┬─────────────┬─────────┐
│ measure │ operation │ measurement │ 1.96*SE │
@@ -148,7 +161,7 @@ savefig("learning_curve.svg");
````
[ Info: Training machine(DeterministicTunedModel(model = DeterministicPipeline(continuous_encoder = ContinuousEncoder(drop_last = false, …), …), …), …).
[ Info: Attempting to evaluate 30 models.
-
Evaluating over 30 metamodels: 7%[=> ] ETA: 0:00:00[K
Evaluating over 30 metamodels: 10%[==> ] ETA: 0:00:01[K
Evaluating over 30 metamodels: 13%[===> ] ETA: 0:00:01[K
Evaluating over 30 metamodels: 17%[====> ] ETA: 0:00:02[K
Evaluating over 30 metamodels: 20%[=====> ] ETA: 0:00:02[K
Evaluating over 30 metamodels: 23%[=====> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 27%[======> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 30%[=======> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 33%[========> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 37%[=========> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 40%[==========> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 43%[==========> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 47%[===========> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 50%[============> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 53%[=============> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 57%[==============> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 60%[===============> ] ETA: 0:00:05[K
Evaluating over 30 metamodels: 63%[===============> ] ETA: 0:00:05[K
Evaluating over 30 metamodels: 67%[================> ] ETA: 0:00:05[K
Evaluating over 30 metamodels: 70%[=================> ] ETA: 0:00:05[K
Evaluating over 30 metamodels: 73%[==================> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 77%[===================> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 80%[====================> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 83%[====================> ] ETA: 0:00:03[K
Evaluating over 30 metamodels: 87%[=====================> ] ETA: 0:00:03[K
Evaluating over 30 metamodels: 90%[======================> ] ETA: 0:00:02[K
Evaluating over 30 metamodels: 93%[=======================> ] ETA: 0:00:02[K
Evaluating over 30 metamodels: 97%[========================>] ETA: 0:00:01[K
Evaluating over 30 metamodels: 100%[=========================] Time: 0:00:28[K
+
Evaluating over 30 metamodels: 0%[> ] ETA: N/A[K
Evaluating over 30 metamodels: 3%[> ] ETA: 0:00:00[K
Evaluating over 30 metamodels: 7%[=> ] ETA: 0:00:00[K
Evaluating over 30 metamodels: 10%[==> ] ETA: 0:00:01[K
Evaluating over 30 metamodels: 13%[===> ] ETA: 0:00:02[K
Evaluating over 30 metamodels: 17%[====> ] ETA: 0:00:02[K
Evaluating over 30 metamodels: 20%[=====> ] ETA: 0:00:02[K
Evaluating over 30 metamodels: 23%[=====> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 27%[======> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 30%[=======> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 33%[========> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 37%[=========> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 40%[==========> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 43%[==========> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 47%[===========> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 50%[============> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 53%[=============> ] ETA: 0:00:05[K
Evaluating over 30 metamodels: 57%[==============> ] ETA: 0:00:05[K
Evaluating over 30 metamodels: 60%[===============> ] ETA: 0:00:05[K
Evaluating over 30 metamodels: 63%[===============> ] ETA: 0:00:05[K
Evaluating over 30 metamodels: 67%[================> ] ETA: 0:00:05[K
Evaluating over 30 metamodels: 70%[=================> ] ETA: 0:00:05[K
Evaluating over 30 metamodels: 73%[==================> ] ETA: 0:00:05[K
Evaluating over 30 metamodels: 77%[===================> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 80%[====================> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 83%[====================> ] ETA: 0:00:04[K
Evaluating over 30 metamodels: 87%[=====================> ] ETA: 0:00:03[K
Evaluating over 30 metamodels: 90%[======================> ] ETA: 0:00:03[K
Evaluating over 30 metamodels: 93%[=======================> ] ETA: 0:00:02[K
Evaluating over 30 metamodels: 97%[========================>] ETA: 0:00:01[K
Evaluating over 30 metamodels: 100%[=========================] Time: 0:00:35[K
````
@@ -225,7 +238,7 @@ savefig("tuned_model_grid.svg");
````
[ Info: Training machine(DeterministicTunedModel(model = DeterministicPipeline(continuous_encoder = ContinuousEncoder(drop_last = false, …), …), …), …).
[ Info: Attempting to evaluate 36 models.
-
Evaluating over 36 metamodels: 6%[=> ] ETA: 0:00:43[K
Evaluating over 36 metamodels: 8%[==> ] ETA: 0:00:50[K
Evaluating over 36 metamodels: 11%[==> ] ETA: 0:00:43[K
Evaluating over 36 metamodels: 14%[===> ] ETA: 0:00:44[K
Evaluating over 36 metamodels: 17%[====> ] ETA: 0:00:42[K
Evaluating over 36 metamodels: 19%[====> ] ETA: 0:00:40[K
Evaluating over 36 metamodels: 22%[=====> ] ETA: 0:00:39[K
Evaluating over 36 metamodels: 25%[======> ] ETA: 0:00:35[K
Evaluating over 36 metamodels: 28%[======> ] ETA: 0:00:34[K
Evaluating over 36 metamodels: 31%[=======> ] ETA: 0:00:34[K
Evaluating over 36 metamodels: 33%[========> ] ETA: 0:00:33[K
Evaluating over 36 metamodels: 36%[=========> ] ETA: 0:00:30[K
Evaluating over 36 metamodels: 39%[=========> ] ETA: 0:00:30[K
Evaluating over 36 metamodels: 42%[==========> ] ETA: 0:00:29[K
Evaluating over 36 metamodels: 44%[===========> ] ETA: 0:00:28[K
Evaluating over 36 metamodels: 47%[===========> ] ETA: 0:00:26[K
Evaluating over 36 metamodels: 50%[============> ] ETA: 0:00:25[K
Evaluating over 36 metamodels: 53%[=============> ] ETA: 0:00:23[K
Evaluating over 36 metamodels: 56%[=============> ] ETA: 0:00:22[K
Evaluating over 36 metamodels: 58%[==============> ] ETA: 0:00:20[K
Evaluating over 36 metamodels: 61%[===============> ] ETA: 0:00:19[K
Evaluating over 36 metamodels: 64%[===============> ] ETA: 0:00:17[K
Evaluating over 36 metamodels: 67%[================> ] ETA: 0:00:16[K
Evaluating over 36 metamodels: 69%[=================> ] ETA: 0:00:15[K
Evaluating over 36 metamodels: 72%[==================> ] ETA: 0:00:13[K
Evaluating over 36 metamodels: 75%[==================> ] ETA: 0:00:12[K
Evaluating over 36 metamodels: 78%[===================> ] ETA: 0:00:11[K
Evaluating over 36 metamodels: 81%[====================> ] ETA: 0:00:09[K
Evaluating over 36 metamodels: 83%[====================> ] ETA: 0:00:08[K
Evaluating over 36 metamodels: 86%[=====================> ] ETA: 0:00:07[K
Evaluating over 36 metamodels: 89%[======================> ] ETA: 0:00:05[K
Evaluating over 36 metamodels: 92%[======================> ] ETA: 0:00:04[K
Evaluating over 36 metamodels: 94%[=======================> ] ETA: 0:00:03[K
Evaluating over 36 metamodels: 97%[========================>] ETA: 0:00:01[K
Evaluating over 36 metamodels: 100%[=========================] Time: 0:00:48[K
+
Evaluating over 36 metamodels: 6%[=> ] ETA: 0:00:55[K
Evaluating over 36 metamodels: 8%[==> ] ETA: 0:00:51[K
Evaluating over 36 metamodels: 11%[==> ] ETA: 0:00:43[K
Evaluating over 36 metamodels: 14%[===> ] ETA: 0:00:37[K
Evaluating over 36 metamodels: 17%[====> ] ETA: 0:00:34[K
Evaluating over 36 metamodels: 19%[====> ] ETA: 0:00:36[K
Evaluating over 36 metamodels: 22%[=====> ] ETA: 0:00:35[K
Evaluating over 36 metamodels: 25%[======> ] ETA: 0:00:34[K
Evaluating over 36 metamodels: 28%[======> ] ETA: 0:00:35[K
Evaluating over 36 metamodels: 31%[=======> ] ETA: 0:00:34[K
Evaluating over 36 metamodels: 33%[========> ] ETA: 0:00:32[K
Evaluating over 36 metamodels: 36%[=========> ] ETA: 0:00:31[K
Evaluating over 36 metamodels: 39%[=========> ] ETA: 0:00:29[K
Evaluating over 36 metamodels: 42%[==========> ] ETA: 0:00:27[K
Evaluating over 36 metamodels: 44%[===========> ] ETA: 0:00:25[K
Evaluating over 36 metamodels: 47%[===========> ] ETA: 0:00:24[K
Evaluating over 36 metamodels: 50%[============> ] ETA: 0:00:22[K
Evaluating over 36 metamodels: 53%[=============> ] ETA: 0:00:22[K
Evaluating over 36 metamodels: 56%[=============> ] ETA: 0:00:20[K
Evaluating over 36 metamodels: 58%[==============> ] ETA: 0:00:19[K
Evaluating over 36 metamodels: 61%[===============> ] ETA: 0:00:17[K
Evaluating over 36 metamodels: 64%[===============> ] ETA: 0:00:17[K
Evaluating over 36 metamodels: 67%[================> ] ETA: 0:00:15[K
Evaluating over 36 metamodels: 69%[=================> ] ETA: 0:00:14[K
Evaluating over 36 metamodels: 72%[==================> ] ETA: 0:00:12[K
Evaluating over 36 metamodels: 75%[==================> ] ETA: 0:00:11[K
Evaluating over 36 metamodels: 78%[===================> ] ETA: 0:00:10[K
Evaluating over 36 metamodels: 81%[====================> ] ETA: 0:00:09[K
Evaluating over 36 metamodels: 83%[====================> ] ETA: 0:00:07[K
Evaluating over 36 metamodels: 86%[=====================> ] ETA: 0:00:06[K
Evaluating over 36 metamodels: 89%[======================> ] ETA: 0:00:05[K
Evaluating over 36 metamodels: 92%[======================> ] ETA: 0:00:04[K
Evaluating over 36 metamodels: 94%[=======================> ] ETA: 0:00:02[K
Evaluating over 36 metamodels: 97%[========================>] ETA: 0:00:01[K
Evaluating over 36 metamodels: 100%[=========================] Time: 0:00:42[K
````
@@ -307,7 +320,7 @@ savefig("tuned_model_random.svg");
````
[ Info: Training machine(DeterministicTunedModel(model = DeterministicPipeline(continuous_encoder = ContinuousEncoder(drop_last = false, …), …), …), …).
[ Info: Attempting to evaluate 40 models.
-
Evaluating over 40 metamodels: 5%[=> ] ETA: 0:00:56[K
Evaluating over 40 metamodels: 8%[=> ] ETA: 0:00:48[K
Evaluating over 40 metamodels: 10%[==> ] ETA: 0:00:48[K
Evaluating over 40 metamodels: 12%[===> ] ETA: 0:00:48[K
Evaluating over 40 metamodels: 15%[===> ] ETA: 0:00:44[K
Evaluating over 40 metamodels: 18%[====> ] ETA: 0:00:46[K
Evaluating over 40 metamodels: 20%[=====> ] ETA: 0:00:44[K
Evaluating over 40 metamodels: 22%[=====> ] ETA: 0:00:42[K
Evaluating over 40 metamodels: 25%[======> ] ETA: 0:00:41[K
Evaluating over 40 metamodels: 28%[======> ] ETA: 0:00:38[K
Evaluating over 40 metamodels: 30%[=======> ] ETA: 0:00:39[K
Evaluating over 40 metamodels: 32%[========> ] ETA: 0:00:37[K
Evaluating over 40 metamodels: 35%[========> ] ETA: 0:00:36[K
Evaluating over 40 metamodels: 38%[=========> ] ETA: 0:00:35[K
Evaluating over 40 metamodels: 40%[==========> ] ETA: 0:00:33[K
Evaluating over 40 metamodels: 42%[==========> ] ETA: 0:00:33[K
Evaluating over 40 metamodels: 45%[===========> ] ETA: 0:00:31[K
Evaluating over 40 metamodels: 48%[===========> ] ETA: 0:00:29[K
Evaluating over 40 metamodels: 50%[============> ] ETA: 0:00:28[K
Evaluating over 40 metamodels: 52%[=============> ] ETA: 0:00:27[K
Evaluating over 40 metamodels: 55%[=============> ] ETA: 0:00:26[K
Evaluating over 40 metamodels: 58%[==============> ] ETA: 0:00:24[K
Evaluating over 40 metamodels: 60%[===============> ] ETA: 0:00:23[K
Evaluating over 40 metamodels: 62%[===============> ] ETA: 0:00:22[K
Evaluating over 40 metamodels: 65%[================> ] ETA: 0:00:20[K
Evaluating over 40 metamodels: 68%[================> ] ETA: 0:00:19[K
Evaluating over 40 metamodels: 70%[=================> ] ETA: 0:00:17[K
Evaluating over 40 metamodels: 72%[==================> ] ETA: 0:00:15[K
Evaluating over 40 metamodels: 75%[==================> ] ETA: 0:00:14[K
Evaluating over 40 metamodels: 78%[===================> ] ETA: 0:00:12[K
Evaluating over 40 metamodels: 80%[====================> ] ETA: 0:00:11[K
Evaluating over 40 metamodels: 82%[====================> ] ETA: 0:00:10[K
Evaluating over 40 metamodels: 85%[=====================> ] ETA: 0:00:08[K
Evaluating over 40 metamodels: 88%[=====================> ] ETA: 0:00:07[K
Evaluating over 40 metamodels: 90%[======================> ] ETA: 0:00:06[K
Evaluating over 40 metamodels: 92%[=======================> ] ETA: 0:00:04[K
Evaluating over 40 metamodels: 95%[=======================> ] ETA: 0:00:03[K
Evaluating over 40 metamodels: 98%[========================>] ETA: 0:00:01[K
Evaluating over 40 metamodels: 100%[=========================] Time: 0:00:55[K
+
Evaluating over 40 metamodels: 0%[> ] ETA: N/A[K
Evaluating over 40 metamodels: 2%[> ] ETA: 0:00:52[K
Evaluating over 40 metamodels: 5%[=> ] ETA: 0:00:50[K
Evaluating over 40 metamodels: 8%[=> ] ETA: 0:00:43[K
Evaluating over 40 metamodels: 10%[==> ] ETA: 0:00:42[K
Evaluating over 40 metamodels: 12%[===> ] ETA: 0:00:42[K
Evaluating over 40 metamodels: 15%[===> ] ETA: 0:00:43[K
Evaluating over 40 metamodels: 18%[====> ] ETA: 0:00:41[K
Evaluating over 40 metamodels: 20%[=====> ] ETA: 0:00:40[K
Evaluating over 40 metamodels: 22%[=====> ] ETA: 0:00:37[K
Evaluating over 40 metamodels: 25%[======> ] ETA: 0:00:39[K
Evaluating over 40 metamodels: 28%[======> ] ETA: 0:00:36[K
Evaluating over 40 metamodels: 30%[=======> ] ETA: 0:00:35[K
Evaluating over 40 metamodels: 32%[========> ] ETA: 0:00:33[K
Evaluating over 40 metamodels: 35%[========> ] ETA: 0:00:32[K
Evaluating over 40 metamodels: 38%[=========> ] ETA: 0:00:32[K
Evaluating over 40 metamodels: 40%[==========> ] ETA: 0:00:30[K
Evaluating over 40 metamodels: 42%[==========> ] ETA: 0:00:30[K
Evaluating over 40 metamodels: 45%[===========> ] ETA: 0:00:28[K
Evaluating over 40 metamodels: 48%[===========> ] ETA: 0:00:26[K
Evaluating over 40 metamodels: 50%[============> ] ETA: 0:00:26[K
Evaluating over 40 metamodels: 52%[=============> ] ETA: 0:00:24[K
Evaluating over 40 metamodels: 55%[=============> ] ETA: 0:00:23[K
Evaluating over 40 metamodels: 58%[==============> ] ETA: 0:00:22[K
Evaluating over 40 metamodels: 60%[===============> ] ETA: 0:00:20[K
Evaluating over 40 metamodels: 62%[===============> ] ETA: 0:00:20[K
Evaluating over 40 metamodels: 65%[================> ] ETA: 0:00:18[K
Evaluating over 40 metamodels: 68%[================> ] ETA: 0:00:17[K
Evaluating over 40 metamodels: 70%[=================> ] ETA: 0:00:15[K
Evaluating over 40 metamodels: 72%[==================> ] ETA: 0:00:14[K
Evaluating over 40 metamodels: 75%[==================> ] ETA: 0:00:13[K
Evaluating over 40 metamodels: 78%[===================> ] ETA: 0:00:11[K
Evaluating over 40 metamodels: 80%[====================> ] ETA: 0:00:10[K
Evaluating over 40 metamodels: 82%[====================> ] ETA: 0:00:09[K
Evaluating over 40 metamodels: 85%[=====================> ] ETA: 0:00:08[K
Evaluating over 40 metamodels: 88%[=====================> ] ETA: 0:00:06[K
Evaluating over 40 metamodels: 90%[======================> ] ETA: 0:00:05[K
Evaluating over 40 metamodels: 92%[=======================> ] ETA: 0:00:04[K
Evaluating over 40 metamodels: 95%[=======================> ] ETA: 0:00:02[K
Evaluating over 40 metamodels: 98%[========================>] ETA: 0:00:01[K
Evaluating over 40 metamodels: 100%[=========================] Time: 0:00:50[K
````
@@ -341,7 +354,7 @@ PerformanceEvaluation object with these fields:
per_fold, per_observation,
fitted_params_per_fold, report_per_fold,
train_test_rows, resampling, repeats
-Tag: DeterministicTunedModel-569
+Tag: DeterministicTunedModel-587
Extract:
┌──────────┬───────────┬─────────────┬─────────┐
│ measure │ operation │ measurement │ 1.96*SE │
@@ -364,13 +377,13 @@ Comparing with the baseline computed earlier:
````
````
-ebase = PerformanceEvaluation("ProbabilisticPipeline-908", 0.568 ± 0.033)
-e0 = PerformanceEvaluation("DeterministicPipeline-823", 0.184 ± 0.028)
-e1 = PerformanceEvaluation("DeterministicTunedModel-569", 0.175 ± 0.021)
+ebase = PerformanceEvaluation("ProbabilisticPipeline-489", 0.568 ± 0.033)
+e0 = PerformanceEvaluation("DeterministicPipeline-381", 0.184 ± 0.028)
+e1 = PerformanceEvaluation("DeterministicTunedModel-587", 0.175 ± 0.021)
````
-Or, we can do this:
+Or, we can do this (needs MLJBase 1.13.0 or higher):
````@julia
describe.([e0, e1]) |> pretty
@@ -382,8 +395,8 @@ describe.([e0, e1]) |> pretty
│ String │ Measurement{Float64} │
│ Textual │ Continuous │
├─────────────────────────────┼──────────────────────┤
-│ DeterministicPipeline-823 │ 0.184±0.028 │
-│ DeterministicTunedModel-569 │ 0.175±0.021 │
+│ DeterministicPipeline-381 │ 0.184±0.028 │
+│ DeterministicTunedModel-587 │ 0.175±0.021 │
└─────────────────────────────┴──────────────────────┘
````
diff --git a/docs/src/notebooks/UsingMLJ/03_model_tuning/tuned_model_grid.svg b/docs/src/notebooks/UsingMLJ/03_model_tuning/tuned_model_grid.svg
index 94ad1ac..b284d62 100644
--- a/docs/src/notebooks/UsingMLJ/03_model_tuning/tuned_model_grid.svg
+++ b/docs/src/notebooks/UsingMLJ/03_model_tuning/tuned_model_grid.svg
@@ -1,294 +1,294 @@
diff --git a/docs/src/notebooks/UsingMLJ/03_model_tuning/tuned_model_random.svg b/docs/src/notebooks/UsingMLJ/03_model_tuning/tuned_model_random.svg
index 33a2310..fdc05dd 100644
--- a/docs/src/notebooks/UsingMLJ/03_model_tuning/tuned_model_random.svg
+++ b/docs/src/notebooks/UsingMLJ/03_model_tuning/tuned_model_random.svg
@@ -1,269 +1,269 @@
diff --git a/learning_curve.svg b/learning_curve.svg
index 42b059d..177bf45 100644
--- a/learning_curve.svg
+++ b/learning_curve.svg
@@ -1,46 +1,46 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/tuned_model_grid.svg b/tuned_model_grid.svg
index e3ed916..94c8ce2 100644
--- a/tuned_model_grid.svg
+++ b/tuned_model_grid.svg
@@ -1,294 +1,294 @@
diff --git a/tuned_model_random.svg b/tuned_model_random.svg
index 7f4925e..5f65b2d 100644
--- a/tuned_model_random.svg
+++ b/tuned_model_random.svg
@@ -1,269 +1,269 @@