diff --git a/docs/src/notebooks/MLJTutorial/01_data_representation/notebook.md b/docs/src/notebooks/MLJTutorial/01_data_representation/notebook.md index 63dd2b0..6d04a3a 100644 --- a/docs/src/notebooks/MLJTutorial/01_data_representation/notebook.md +++ b/docs/src/notebooks/MLJTutorial/01_data_representation/notebook.md @@ -290,7 +290,7 @@ csv_file = Downloads.download(url) ```` ```` -"/tmp/jl_ruz0kP/horse.csv" +"/tmp/jl_kOThHT/horse.csv" ```` Entering these lines of code downloads the data to a temporary file at the location @@ -531,8 +531,8 @@ A = rand(2, 3) ```` 2×3 Matrix{Float64}: - 0.113155 0.33251 0.397839 - 0.171581 0.90293 0.436722 + 0.832781 0.999535 0.163212 + 0.685279 0.998338 0.966773 ```` ````@julia @@ -550,8 +550,8 @@ Asparse = sparse(A) ```` 2×3 SparseArrays.SparseMatrixCSC{Float64, Int64} with 6 stored entries: - 0.113155 0.33251 0.397839 - 0.171581 0.90293 0.436722 + 0.832781 0.999535 0.163212 + 0.685279 0.998338 0.966773 ```` ````@julia diff --git a/docs/src/notebooks/MLJTutorial/02_models/learning_curve.png b/docs/src/notebooks/MLJTutorial/02_models/learning_curve.png index 7234cf4..e848c6b 100755 Binary files a/docs/src/notebooks/MLJTutorial/02_models/learning_curve.png and b/docs/src/notebooks/MLJTutorial/02_models/learning_curve.png differ diff --git a/docs/src/notebooks/MLJTutorial/02_models/notebook.md b/docs/src/notebooks/MLJTutorial/02_models/notebook.md index c2bde2b..d9e9949 100644 --- a/docs/src/notebooks/MLJTutorial/02_models/notebook.md +++ b/docs/src/notebooks/MLJTutorial/02_models/notebook.md @@ -625,8 +625,8 @@ mach = machine(model, X, y) untrained Machine; caches model-specific representations of data model: NeuralNetworkClassifier(builder = Short(n_hidden = 0, …), …) args: - 1: Source @909 ⏎ ScientificTypesBase.Table{AbstractVector{ScientificTypesBase.Continuous}} - 2: Source @399 ⏎ AbstractVector{ScientificTypesBase.Multiclass{3}} + 1: Source @381 ⏎ ScientificTypesBase.Table{AbstractVector{ScientificTypesBase.Continuous}} + 2: Source @900 ⏎ AbstractVector{ScientificTypesBase.Multiclass{3}} ```` @@ -652,18 +652,18 @@ fit!(mach, rows=train, verbosity=2); ```` [ Info: Training machine(NeuralNetworkClassifier(builder = Short(n_hidden = 0, …), …), …). [ Info: MLJFlux: converting input data to Float32 -[ Info: Loss is 2.023 -[ Info: Loss is 1.592 -[ Info: Loss is 1.491 -[ Info: Loss is 1.205 -[ Info: Loss is 1.227 -[ Info: Loss is 1.177 -[ Info: Loss is 1.19 -[ Info: Loss is 1.128 -[ Info: Loss is 1.09 -[ Info: Loss is 1.058 -[ Info: Loss is 1.059 -[ Info: Loss is 1.056 +[ Info: Loss is 1.843 +[ Info: Loss is 1.666 +[ Info: Loss is 1.449 +[ Info: Loss is 1.333 +[ Info: Loss is 1.242 +[ Info: Loss is 1.188 +[ Info: Loss is 1.105 +[ Info: Loss is 1.003 +[ Info: Loss is 0.9239 +[ Info: Loss is 0.9508 +[ Info: Loss is 0.8857 +[ Info: Loss is 0.94 ```` @@ -676,9 +676,9 @@ yhat[1:3] ```` 3-element CategoricalDistributions.UnivariateFiniteVector{ScientificTypesBase.Multiclass{3}, String, UInt32, Float32}: - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.38, Iris-versicolor=>0.293, Iris-virginica=>0.327) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.449, Iris-versicolor=>0.271, Iris-virginica=>0.281) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.454, Iris-versicolor=>0.27, Iris-virginica=>0.276) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.293, Iris-versicolor=>0.352, Iris-virginica=>0.355) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.618, Iris-versicolor=>0.198, Iris-virginica=>0.184) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.595, Iris-versicolor=>0.211, Iris-virginica=>0.194) ```` We'll have more to say on the form of this prediction shortly. @@ -701,7 +701,7 @@ report(mach) ```` ```` -(training_losses = Float32[1.7850709, 2.023422, 1.5918993, 1.4905419, 1.2054352, 1.2269526, 1.1768, 1.1895036, 1.128155, 1.0898077, 1.0582713, 1.0594574, 1.056259],) +(training_losses = Float32[1.9923493, 1.8433424, 1.6657014, 1.4489316, 1.3331685, 1.2423297, 1.187512, 1.1045707, 1.003318, 0.9239003, 0.95083165, 0.88573194, 0.939968],) ```` You save a machine like this: @@ -720,9 +720,9 @@ yhat[1:3] ```` 3-element CategoricalDistributions.UnivariateFiniteVector{ScientificTypesBase.Multiclass{3}, String, UInt32, Float32}: - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.379, Iris-versicolor=>0.304, Iris-virginica=>0.317) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.454, Iris-versicolor=>0.269, Iris-virginica=>0.277) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.377, Iris-versicolor=>0.292, Iris-virginica=>0.331) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.28, Iris-versicolor=>0.36, Iris-virginica=>0.36) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.605, Iris-versicolor=>0.205, Iris-virginica=>0.19) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.288, Iris-versicolor=>0.353, Iris-virginica=>0.359) ```` Machines remember the last set of hyperparameters used during fit, @@ -737,10 +737,10 @@ fit!(mach, rows=train, verbosity=2); ```` [ Info: Updating machine(NeuralNetworkClassifier(builder = Short(n_hidden = 0, …), …), …) (with warm restart if possible). -[ Info: Loss is 1.067 -[ Info: Loss is 1.038 -[ Info: Loss is 1.034 -[ Info: Loss is 1.038 +[ Info: Loss is 0.9071 +[ Info: Loss is 0.8656 +[ Info: Loss is 0.9126 +[ Info: Loss is 0.8793 ```` @@ -771,10 +771,10 @@ fit!(mach, rows=train, verbosity=2); ```` [ Info: Updating machine(NeuralNetworkClassifier(builder = Short(n_hidden = 0, …), …), …) (with warm restart if possible). -[ Info: Loss is 0.9739 -[ Info: Loss is 0.9423 -[ Info: Loss is 0.9133 -[ Info: Loss is 0.8122 +[ Info: Loss is 0.8646 +[ Info: Loss is 0.7723 +[ Info: Loss is 0.7511 +[ Info: Loss is 0.7037 ```` @@ -789,26 +789,26 @@ fit!(mach, rows=train, verbosity=2); ```` [ Info: Updating machine(NeuralNetworkClassifier(builder = Short(n_hidden = 0, …), …), …) (with warm restart if possible). [ Info: MLJFlux: converting input data to Float32 -[ Info: Loss is 1.285 -[ Info: Loss is 0.9493 -[ Info: Loss is 0.9647 -[ Info: Loss is 0.8598 -[ Info: Loss is 0.8699 -[ Info: Loss is 0.859 -[ Info: Loss is 0.843 -[ Info: Loss is 0.8556 -[ Info: Loss is 0.8016 -[ Info: Loss is 0.7762 -[ Info: Loss is 0.7672 -[ Info: Loss is 0.6868 -[ Info: Loss is 0.7414 -[ Info: Loss is 0.7573 -[ Info: Loss is 0.7062 -[ Info: Loss is 0.7683 -[ Info: Loss is 0.725 -[ Info: Loss is 0.8516 -[ Info: Loss is 0.7587 -[ Info: Loss is 0.7132 +[ Info: Loss is 1.152 +[ Info: Loss is 0.9846 +[ Info: Loss is 0.9392 +[ Info: Loss is 0.8941 +[ Info: Loss is 0.8051 +[ Info: Loss is 0.7857 +[ Info: Loss is 0.6737 +[ Info: Loss is 0.6225 +[ Info: Loss is 0.6503 +[ Info: Loss is 0.642 +[ Info: Loss is 0.6935 +[ Info: Loss is 0.5629 +[ Info: Loss is 0.5992 +[ Info: Loss is 0.5452 +[ Info: Loss is 0.6846 +[ Info: Loss is 0.6737 +[ Info: Loss is 0.649 +[ Info: Loss is 0.679 +[ Info: Loss is 0.7019 +[ Info: Loss is 0.6518 ```` @@ -827,7 +827,7 @@ yhat[1] ```` ```` -UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.156, Iris-versicolor=>0.519, Iris-virginica=>0.326) +UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0677, Iris-versicolor=>0.624, Iris-virginica=>0.308) ```` What's going on here? @@ -858,7 +858,7 @@ pdf(yhat[1], "Iris-virginica") ```` ```` -0.32571402f0 +0.30793336f0 ```` To get the most likely observation, we do @@ -879,10 +879,10 @@ broadcast(pdf, yhat[1:4], "Iris-versicolor") ```` 4-element Vector{Float32}: - 0.5187301 - 0.0025167037 - 0.00241053 - 0.0027305766 + 0.6243697 + 0.02210315 + 0.023541868 + 0.027490756 ```` ````@julia @@ -921,10 +921,10 @@ pdf(yhat, L)[1:4, :] ```` 4×3 Matrix{Float32}: - 0.155556 0.51873 0.325714 - 0.997162 0.0025167 0.000321372 - 0.997287 0.00241053 0.000302146 - 0.996915 0.00273058 0.000354911 + 0.067697 0.62437 0.307933 + 0.975022 0.0221031 0.00287438 + 0.973352 0.0235419 0.00310646 + 0.968749 0.0274908 0.00376043 ```` However, in a typical MLJ workflow, this is not as useful as you might imagine. In @@ -936,7 +936,7 @@ log_loss(yhat, y[test]) ```` ```` -0.41664216811287624 +0.3023546542336178 ```` To apply a deterministic measure, we first need to obtain point-estimates: @@ -946,7 +946,7 @@ misclassification_rate(mode.(yhat), y[test]) ```` ```` -0.08888888888888889 +0.044444444444444446 ```` For more on metrics provided by MLJ, see the [StatisticalMeasures.jl @@ -974,15 +974,15 @@ PerformanceEvaluation object with these fields: per_fold, per_observation, fitted_params_per_fold, report_per_fold, train_test_rows, resampling, repeats -Tag: NeuralNetworkClassifier-433 +Tag: NeuralNetworkClassifier-246 Extract: ┌─────────────────────────┬──────────────┬─────────────┐ │ measure │ operation │ measurement │ ├─────────────────────────┼──────────────┼─────────────┤ -│ LogLoss( │ predict │ 0.417 │ +│ LogLoss( │ predict │ 0.302 │ │ tol = 2.22045e-16) │ │ │ -│ MisclassificationRate() │ predict_mode │ 0.0889 │ -│ BrierScore() │ predict │ -0.236 │ +│ MisclassificationRate() │ predict_mode │ 0.0444 │ +│ BrierScore() │ predict │ -0.159 │ └─────────────────────────┴──────────────┴─────────────┘ Apply `describe` to this result for a named tuple summary. ```` @@ -1004,23 +1004,23 @@ PerformanceEvaluation object with these fields: per_fold, per_observation, fitted_params_per_fold, report_per_fold, train_test_rows, resampling, repeats -Tag: NeuralNetworkClassifier-136 +Tag: NeuralNetworkClassifier-510 Extract: ┌───┬─────────────────────────┬──────────────┬─────────────┬─────────┐ │ │ measure │ operation │ measurement │ 1.96*SE │ ├───┼─────────────────────────┼──────────────┼─────────────┼─────────┤ -│ A │ LogLoss( │ predict │ 0.276 │ 0.035 │ +│ A │ LogLoss( │ predict │ 0.311 │ 0.036 │ │ │ tol = 2.22045e-16) │ │ │ │ │ B │ MisclassificationRate() │ predict_mode │ 0.04 │ 0.022 │ -│ C │ BrierScore() │ predict │ -0.143 │ 0.026 │ +│ C │ BrierScore() │ predict │ -0.164 │ 0.027 │ └───┴─────────────────────────┴──────────────┴─────────────┴─────────┘ -┌───┬──────────────────────────────────────────────────────────┐ -│ │ per_fold │ -├───┼──────────────────────────────────────────────────────────┤ -│ A │ [0.312, 0.256, 0.219, 0.263, 0.277, 0.33] │ -│ B │ [0.04, 0.04, 0.0, 0.04, 0.08, 0.04] │ -│ C │ Float32[-0.181, -0.127, -0.0987, -0.137, -0.142, -0.172] │ -└───┴──────────────────────────────────────────────────────────┘ +┌───┬─────────────────────────────────────────────────────────┐ +│ │ per_fold │ +├───┼─────────────────────────────────────────────────────────┤ +│ A │ [0.339, 0.268, 0.278, 0.358, 0.275, 0.347] │ +│ B │ [0.04, 0.04, 0.0, 0.08, 0.04, 0.04] │ +│ C │ Float32[-0.184, -0.137, -0.149, -0.204, -0.128, -0.183] │ +└───┴─────────────────────────────────────────────────────────┘ Apply `describe` to this result for a named tuple summary. ```` @@ -1043,22 +1043,22 @@ PerformanceEvaluation object with these fields: per_fold, per_observation, fitted_params_per_fold, report_per_fold, train_test_rows, resampling, repeats -Tag: NeuralNetworkClassifier-762 +Tag: NeuralNetworkClassifier-417 Extract: ┌───┬─────────────────────────┬──────────────┬─────────────┬─────────┐ │ │ measure │ operation │ measurement │ 1.96*SE │ ├───┼─────────────────────────┼──────────────┼─────────────┼─────────┤ -│ A │ LogLoss( │ predict │ 0.318 │ 0.035 │ +│ A │ LogLoss( │ predict │ 0.298 │ 0.034 │ │ │ tol = 2.22045e-16) │ │ │ │ -│ B │ MisclassificationRate() │ predict_mode │ 0.042 │ 0.023 │ -│ C │ BrierScore() │ predict │ -0.171 │ 0.023 │ +│ B │ MisclassificationRate() │ predict_mode │ 0.047 │ 0.025 │ +│ C │ BrierScore() │ predict │ -0.154 │ 0.025 │ └───┴─────────────────────────┴──────────────┴─────────────┴─────────┘ ┌───┬─────────────────────────────────────────────────────────────────────────── │ │ per_fold ⋯ ├───┼─────────────────────────────────────────────────────────────────────────── -│ A │ [0.263, 0.309, 0.294, 0.313, 0.317, 0.216, 0.388, 0.297, 0.427, 0.275, 0 ⋯ -│ B │ [0.0, 0.04, 0.04, 0.0, 0.04, 0.04, 0.2, 0.0, 0.0, 0.04, 0.04, 0.08, 0.08 ⋯ -│ C │ Float32[-0.133, -0.164, -0.153, -0.156, -0.175, -0.117, -0.246, -0.158, ⋯ +│ A │ [0.299, 0.369, 0.239, 0.341, 0.317, 0.173, 0.359, 0.363, 0.328, 0.296, 0 ⋯ +│ B │ [0.0, 0.12, 0.04, 0.0, 0.08, 0.04, 0.04, 0.08, 0.0, 0.0, 0.04, 0.08, 0.0 ⋯ +│ C │ Float32[-0.155, -0.208, -0.105, -0.176, -0.171, -0.0781, -0.191, -0.202, ⋯ └───┴─────────────────────────────────────────────────────────────────────────── 1 column omitted Apply `describe` to this result for a named tuple summary. @@ -1085,51 +1085,51 @@ predict(mach, rows=test) # and predict missing targets ```` 45-element CategoricalDistributions.UnivariateFiniteVector{ScientificTypesBase.Multiclass{3}, String, UInt32, Float32}: - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0903, Iris-versicolor=>0.528, Iris-virginica=>0.382) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.979, Iris-versicolor=>0.0153, Iris-virginica=>0.00547) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.978, Iris-versicolor=>0.0163, Iris-virginica=>0.00586) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.975, Iris-versicolor=>0.0185, Iris-virginica=>0.00674) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.134, Iris-versicolor=>0.524, Iris-virginica=>0.342) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>8.56e-5, Iris-versicolor=>0.228, Iris-virginica=>0.772) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.983, Iris-versicolor=>0.0127, Iris-virginica=>0.00446) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.98, Iris-versicolor=>0.0149, Iris-virginica=>0.00533) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00347, Iris-versicolor=>0.395, Iris-virginica=>0.602) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.135, Iris-versicolor=>0.521, Iris-virginica=>0.343) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>2.86e-5, Iris-versicolor=>0.19, Iris-virginica=>0.81) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>9.2e-5, Iris-versicolor=>0.231, Iris-virginica=>0.769) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>3.58e-5, Iris-versicolor=>0.198, Iris-virginica=>0.802) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0527, Iris-versicolor=>0.52, Iris-virginica=>0.427) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.978, Iris-versicolor=>0.0164, Iris-virginica=>0.0059) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.144, Iris-versicolor=>0.518, Iris-virginica=>0.338) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>8.25e-5, Iris-versicolor=>0.227, Iris-virginica=>0.773) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.000144, Iris-versicolor=>0.248, Iris-virginica=>0.752) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00414, Iris-versicolor=>0.404, Iris-virginica=>0.592) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0957, Iris-versicolor=>0.528, Iris-virginica=>0.376) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.215, Iris-versicolor=>0.494, Iris-virginica=>0.291) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.045, Iris-versicolor=>0.515, Iris-virginica=>0.44) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0742, Iris-versicolor=>0.526, Iris-virginica=>0.4) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>5.45e-5, Iris-versicolor=>0.212, Iris-virginica=>0.788) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.964, Iris-versicolor=>0.0261, Iris-virginica=>0.0098) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>4.02e-5, Iris-versicolor=>0.202, Iris-virginica=>0.798) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.131, Iris-versicolor=>0.523, Iris-virginica=>0.346) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00176, Iris-versicolor=>0.361, Iris-virginica=>0.637) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.000152, Iris-versicolor=>0.25, Iris-virginica=>0.749) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.000158, Iris-versicolor=>0.252, Iris-virginica=>0.748) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0393, Iris-versicolor=>0.51, Iris-virginica=>0.451) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.969, Iris-versicolor=>0.023, Iris-virginica=>0.00853) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0425, Iris-versicolor=>0.513, Iris-virginica=>0.445) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.097, Iris-versicolor=>0.527, Iris-virginica=>0.376) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>4.55e-5, Iris-versicolor=>0.206, Iris-virginica=>0.794) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00276, Iris-versicolor=>0.383, Iris-virginica=>0.614) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>9.93e-5, Iris-versicolor=>0.234, Iris-virginica=>0.766) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.138, Iris-versicolor=>0.521, Iris-virginica=>0.341) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.977, Iris-versicolor=>0.0165, Iris-virginica=>0.00596) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.977, Iris-versicolor=>0.0171, Iris-virginica=>0.00619) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.000143, Iris-versicolor=>0.248, Iris-virginica=>0.752) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.97, Iris-versicolor=>0.0217, Iris-virginica=>0.00804) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00164, Iris-versicolor=>0.357, Iris-virginica=>0.641) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.124, Iris-versicolor=>0.525, Iris-virginica=>0.351) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.00131, Iris-versicolor=>0.346, Iris-virginica=>0.652) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.117, Iris-versicolor=>0.646, Iris-virginica=>0.237) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.997, Iris-versicolor=>0.00257, Iris-virginica=>7.17e-11) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.997, Iris-versicolor=>0.00255, Iris-virginica=>7.19e-11) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.997, Iris-versicolor=>0.00283, Iris-virginica=>8.77e-11) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.142, Iris-versicolor=>0.69, Iris-virginica=>0.168) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0346, Iris-versicolor=>0.311, Iris-virginica=>0.654) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.998, Iris-versicolor=>0.00218, Iris-virginica=>5.37e-11) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.998, Iris-versicolor=>0.00242, Iris-virginica=>6.45e-11) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0467, Iris-versicolor=>0.382, Iris-virginica=>0.571) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.223, Iris-versicolor=>0.732, Iris-virginica=>0.0447) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0322, Iris-versicolor=>0.295, Iris-virginica=>0.673) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.035, Iris-versicolor=>0.313, Iris-virginica=>0.652) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0323, Iris-versicolor=>0.296, Iris-virginica=>0.672) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0765, Iris-versicolor=>0.517, Iris-virginica=>0.407) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.997, Iris-versicolor=>0.0026, Iris-virginica=>7.3700004e-11) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.261, Iris-versicolor=>0.716, Iris-virginica=>0.0236) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0329, Iris-versicolor=>0.3, Iris-virginica=>0.667) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0339, Iris-versicolor=>0.306, Iris-virginica=>0.66) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0469, Iris-versicolor=>0.38, Iris-virginica=>0.573) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.118, Iris-versicolor=>0.645, Iris-virginica=>0.236) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.362, Iris-versicolor=>0.634, Iris-virginica=>0.00444) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0913, Iris-versicolor=>0.575, Iris-virginica=>0.334) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.135, Iris-versicolor=>0.685, Iris-virginica=>0.18) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0323, Iris-versicolor=>0.296, Iris-virginica=>0.672) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.996, Iris-versicolor=>0.00382, Iris-virginica=>1.45e-10) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0322, Iris-versicolor=>0.295, Iris-virginica=>0.672) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.175, Iris-versicolor=>0.723, Iris-virginica=>0.101) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0368, Iris-versicolor=>0.324, Iris-virginica=>0.639) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0343, Iris-versicolor=>0.309, Iris-virginica=>0.656) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0341, Iris-versicolor=>0.307, Iris-virginica=>0.659) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0992, Iris-versicolor=>0.596, Iris-virginica=>0.304) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.996, Iris-versicolor=>0.00359, Iris-virginica=>1.36e-10) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.103, Iris-versicolor=>0.611, Iris-virginica=>0.286) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.152, Iris-versicolor=>0.703, Iris-virginica=>0.145) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0327, Iris-versicolor=>0.299, Iris-virginica=>0.668) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0445, Iris-versicolor=>0.368, Iris-virginica=>0.587) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0337, Iris-versicolor=>0.305, Iris-virginica=>0.661) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.222, Iris-versicolor=>0.736, Iris-virginica=>0.0425) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.997, Iris-versicolor=>0.00283, Iris-virginica=>8.7000004e-11) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.997, Iris-versicolor=>0.00269, Iris-virginica=>7.92e-11) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0346, Iris-versicolor=>0.311, Iris-virginica=>0.655) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.997, Iris-versicolor=>0.00341, Iris-virginica=>1.2e-10) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0418, Iris-versicolor=>0.353, Iris-virginica=>0.605) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.156, Iris-versicolor=>0.707, Iris-virginica=>0.137) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(Iris-setosa=>0.0395, Iris-versicolor=>0.341, Iris-virginica=>0.62) ```` ### On learning curves @@ -1158,7 +1158,7 @@ curve = learning_curve( ```` ```` -(parameter_name = "epochs", parameter_scale = :log10, parameter_values = [1, 2, 3, 4, 5, 7, 9, 11, 14, 17, 22, 28, 36, 45, 57, 73, 92, 117, 149, 189, 240, 304, 386, 489, 621, 788, 1000], measurements = [0.9792367389813159, 0.861137424251525, 0.795194663958999, 0.7178450923341597, 0.6672852524240639, 0.6298719732484978, 0.5722341032114211, 0.5468304476118365, 0.5176573197034343, 0.49085053804968687, 0.4656446207911134, 0.4495242488591051, 0.4099614796325163, 0.42257271926898354, 0.31717065032020536, 0.3321625563432286, 0.2845664781923225, 0.22536592494730678, 0.1853560701976897, 0.24943364437705945, 0.24673236950764432, 0.20234975793565377, 0.20691858169733499, 0.20911653361243762, 0.22259957311318937, 0.21965174517509872, 0.21279597941067163]) +(parameter_name = "epochs", parameter_scale = :log10, parameter_values = [1, 2, 3, 4, 5, 7, 9, 11, 14, 17, 22, 28, 36, 45, 57, 73, 92, 117, 149, 189, 240, 304, 386, 489, 621, 788, 1000], measurements = [1.1101430890270785, 1.0523844064782457, 0.9552739719860082, 0.8252618878652416, 0.7407029729629422, 0.7117540611099686, 0.6473811832752417, 0.6051996950232142, 0.5927533935853231, 0.6014127857471417, 0.520697202909161, 0.46281622944694667, 0.408509822448981, 0.3938544280811988, 0.28662257565458665, 0.3400356434034648, 0.3170479733060074, 0.3172615464977853, 0.3085572772007371, 0.29101642775211295, 0.3078519856587645, 0.31229959524410067, 0.2664894730764057, 0.3094692929634764, 0.2821135795288816, 0.31090260857700913, 0.2931820005629499]) ```` ````@julia @@ -1221,16 +1221,16 @@ y4 = [n_devices(row.salary) for row in eachrow(X4)] ```` 10-element Vector{Int64}: - 3 + 2 + 2 + 2 + 1 1 - 3 - 0 2 + 1 6 - 0 - 0 2 - 1 + 5 ```` (b) What models can be applied if you coerce the salary to a @@ -1256,10 +1256,10 @@ pretty(data) │ Int64 │ Float64 │ Float64 │ CategoricalValue{String, UInt32} │ │ Count │ Continuous │ Continuous │ OrderedFactor{2} │ ├───────┼────────────┼────────────┼──────────────────────────────────┤ -│ 1 │ 0.754755 │ 0.208445 │ male │ -│ 2 │ 0.874161 │ 0.399919 │ female │ -│ 3 │ 0.906202 │ 0.19441 │ female │ -│ 4 │ 0.718781 │ 0.818259 │ male │ +│ 1 │ 0.834465 │ 0.356333 │ male │ +│ 2 │ 0.0335365 │ 0.292738 │ female │ +│ 3 │ 0.40159 │ 0.334114 │ female │ +│ 4 │ 0.745999 │ 0.11353 │ male │ └───────┴────────────┴────────────┴──────────────────────────────────┘ ```` diff --git a/docs/src/notebooks/MLJTutorial/03_pipelines/notebook.md b/docs/src/notebooks/MLJTutorial/03_pipelines/notebook.md index f8049ef..9383c41 100644 --- a/docs/src/notebooks/MLJTutorial/03_pipelines/notebook.md +++ b/docs/src/notebooks/MLJTutorial/03_pipelines/notebook.md @@ -31,8 +31,8 @@ x = rand(100); ```` ```` -mean(x) = 0.47696749924138343 -std(x) = 0.29728842148044776 +mean(x) = 0.488231598311445 +std(x) = 0.29947481462620856 ```` @@ -46,8 +46,8 @@ xhat = transform(mach, x); ```` [ Info: Training machine(Standardizer(features = Symbol[], …), …). -mean(xhat) = 2.042810365310288e-16 -std(xhat) = 1.0 +mean(xhat) = -9.769962616701378e-17 +std(xhat) = 0.9999999999999999 ```` @@ -501,7 +501,7 @@ PerformanceEvaluation object with these fields: per_fold, per_observation, fitted_params_per_fold, report_per_fold, train_test_rows, resampling, repeats -Tag: DeterministicPipeline-190 +Tag: DeterministicPipeline-761 Extract: ┌──────────┬───────────┬─────────────┐ │ measure │ operation │ measurement │ @@ -631,7 +631,7 @@ PerformanceEvaluation object with these fields: per_fold, per_observation, fitted_params_per_fold, report_per_fold, train_test_rows, resampling, repeats -Tag: DeterministicPipeline-398 +Tag: DeterministicPipeline-289 Extract: ┌──────────┬───────────┬─────────────┬─────────┐ │ measure │ operation │ measurement │ 1.96*SE │ @@ -672,7 +672,7 @@ PerformanceEvaluation object with these fields: per_fold, per_observation, fitted_params_per_fold, report_per_fold, train_test_rows, resampling, repeats -Tag: DeterministicPipeline-611 +Tag: DeterministicPipeline-526 Extract: ┌──────────┬───────────┬─────────────┬──────────┐ │ measure │ operation │ measurement │ 1.96*SE │ @@ -700,7 +700,7 @@ PerformanceEvaluation object with these fields: per_fold, per_observation, fitted_params_per_fold, report_per_fold, train_test_rows, resampling, repeats -Tag: DeterministicPipeline-592 +Tag: DeterministicPipeline-221 Extract: ┌──────────┬───────────┬─────────────┬─────────┐ │ measure │ operation │ measurement │ 1.96*SE │ diff --git a/docs/src/notebooks/MLJTutorial/04_tuning/gamma_sampler.png b/docs/src/notebooks/MLJTutorial/04_tuning/gamma_sampler.png index 8ddb87e..8314019 100644 Binary files a/docs/src/notebooks/MLJTutorial/04_tuning/gamma_sampler.png and b/docs/src/notebooks/MLJTutorial/04_tuning/gamma_sampler.png differ diff --git a/docs/src/notebooks/MLJTutorial/04_tuning/notebook.md b/docs/src/notebooks/MLJTutorial/04_tuning/notebook.md index 9882163..ac2c41f 100644 --- a/docs/src/notebooks/MLJTutorial/04_tuning/notebook.md +++ b/docs/src/notebooks/MLJTutorial/04_tuning/notebook.md @@ -367,7 +367,7 @@ PerformanceEvaluation object with these fields: per_fold, per_observation, fitted_params_per_fold, report_per_fold, train_test_rows, resampling, repeats -Tag: ProbabilisticPipeline-307 +Tag: ProbabilisticPipeline-620 Extract: ┌──────────────────────┬───────────┬─────────────┬─────────┐ │ measure │ operation │ measurement │ 1.96*SE │ @@ -394,7 +394,7 @@ PerformanceEvaluation object with these fields: per_fold, per_observation, fitted_params_per_fold, report_per_fold, train_test_rows, resampling, repeats -Tag: ProbabilisticTunedModel-779 +Tag: ProbabilisticTunedModel-280 Extract: ┌──────────────────────┬───────────┬─────────────┬─────────┐ │ measure │ operation │ measurement │ 1.96*SE │ diff --git a/docs/src/notebooks/MLJTutorial/04_tuning/tuning.png b/docs/src/notebooks/MLJTutorial/04_tuning/tuning.png index 9be33e5..8e4a675 100644 Binary files a/docs/src/notebooks/MLJTutorial/04_tuning/tuning.png and b/docs/src/notebooks/MLJTutorial/04_tuning/tuning.png differ diff --git a/docs/src/notebooks/MLJTutorial/05_composition/notebook.md b/docs/src/notebooks/MLJTutorial/05_composition/notebook.md index 12dbe0a..eaa3626 100644 --- a/docs/src/notebooks/MLJTutorial/05_composition/notebook.md +++ b/docs/src/notebooks/MLJTutorial/05_composition/notebook.md @@ -64,11 +64,11 @@ pretty(X) │ Float64 │ Float64 │ Float64 │ │ Continuous │ Continuous │ Continuous │ ├────────────┼────────────┼────────────┤ -│ 14.9681 │ -1.47137 │ 12.1949 │ -│ -3.26626 │ -0.710617 │ 7.36507 │ -│ -4.17429 │ -0.0882215 │ 9.87907 │ -│ 6.00034 │ -2.41859 │ 14.9897 │ -│ -4.38537 │ -0.187352 │ 7.22233 │ +│ -9.85801 │ 0.948133 │ 1.5543 │ +│ -9.05017 │ 0.116089 │ 0.3031 │ +│ 13.9369 │ -0.530196 │ -2.03829 │ +│ -0.617015 │ 2.30274 │ -1.08639 │ +│ -8.65166 │ 2.36956 │ 1.34361 │ └────────────┴────────────┴────────────┘ ```` @@ -91,11 +91,11 @@ yhat = predict(mach2, Xstand) ```` 5-element CategoricalDistributions.UnivariateFiniteVector{ScientificTypesBase.Multiclass{3}, Int64, UInt32, Float64}: - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.993, 2=>0.00372, 3=>0.00319) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.000405, 2=>0.000195, 3=>0.999) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.000309, 2=>0.000249, 3=>0.999) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.00752, 2=>0.992, 3=>0.000947) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.00012, 2=>1.96e-5, 3=>1.0) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>8.09e-5, 2=>8.48e-5, 3=>1.0) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.00351, 2=>0.000732, 3=>0.996) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.995, 2=>0.00474, 3=>0.00011) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.00327, 2=>0.989, 3=>0.00818) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>6.88e-5, 2=>0.00843, 3=>0.991) ```` **Step 1** - Edit your code as follows: @@ -119,15 +119,15 @@ yhat = predict(mach2, Xstand) ```` ```` -Node @440 → LogisticClassifier(…) +Node @623 → LogisticClassifier(…) args: - 1: Node @909 → Standardizer(…) + 1: Node @703 → Standardizer(…) formula: predict( machine(LogisticClassifier(lambda = 0.001, …), …), transform( machine(Standardizer(features = Symbol[], …), …), - Source @128, + Source @747, ), ) ```` @@ -150,11 +150,11 @@ Xstand() |> pretty │ Float64 │ Float64 │ Float64 │ │ Continuous │ Continuous │ Continuous │ ├────────────┼────────────┼────────────┤ -│ 1.54143 │ -0.508564 │ 0.563239 │ -│ -0.597675 │ 0.271238 │ -0.895631 │ -│ -0.704197 │ 0.909215 │ -0.136268 │ -│ 0.489404 │ -1.47949 │ 1.40741 │ -│ -0.728959 │ 0.807603 │ -0.938747 │ +│ -0.694096 │ -0.0720163 │ 0.991034 │ +│ -0.614108 │ -0.715398 │ 0.185347 │ +│ 1.66195 │ -1.21514 │ -1.32235 │ +│ 0.220901 │ 0.975443 │ -0.709393 │ +│ -0.574649 │ 1.02711 │ 0.855364 │ └────────────┴────────────┴────────────┘ ```` @@ -178,11 +178,11 @@ yhat() ```` 5-element CategoricalDistributions.UnivariateFiniteVector{ScientificTypesBase.Multiclass{3}, Int64, UInt32, Float64}: - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.993, 2=>0.00372, 3=>0.00319) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.000405, 2=>0.000195, 3=>0.999) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.000309, 2=>0.000249, 3=>0.999) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.00752, 2=>0.992, 3=>0.000947) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.00012, 2=>1.96e-5, 3=>1.0) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>8.09e-5, 2=>8.48e-5, 3=>1.0) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.00351, 2=>0.000732, 3=>0.996) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.995, 2=>0.00474, 3=>0.00011) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.00327, 2=>0.989, 3=>0.00818) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>6.88e-5, 2=>0.00843, 3=>0.991) ```` The node `yhat` is the "descendant" (in an associated DAG we have @@ -194,7 +194,7 @@ origins(yhat) ```` 1-element Vector{MLJBase.Source}: - Source @128 ⏎ `ScientificTypesBase.Table{AbstractVector{ScientificTypesBase.Continuous}}` + Source @747 ⏎ `ScientificTypesBase.Table{AbstractVector{ScientificTypesBase.Continuous}}` ```` The data at the source node is replaced by `Xnew` to obtain a @@ -207,8 +207,8 @@ yhat(Xnew) ```` 2-element CategoricalDistributions.UnivariateFiniteVector{ScientificTypesBase.Multiclass{3}, Int64, UInt32, Float64}: - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>1.13e-8, 2=>1.0, 3=>6.09e-8) - UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>2.89e-16, 2=>1.0, 3=>4.36e-13) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>0.0461, 2=>0.954, 3=>1.47e-10) + UnivariateFinite{ScientificTypesBase.Multiclass{3}}(1=>1.0, 2=>8.32e-5, 3=>1.38e-9) ```` **Step 2** - Export the learning network as a new stand-alone model type @@ -238,15 +238,15 @@ yhat = predict(mach2, Xstand) ```` ```` -Node @530 → :classifier +Node @708 → :classifier args: - 1: Node @804 → :standardizer + 1: Node @363 → :standardizer formula: predict( machine(:classifier, …), transform( machine(:standardizer, …), - Source @128, + Source @747, ), ) ```` @@ -380,7 +380,7 @@ y = source(y) ```` ```` -Source @555 ⏎ `AbstractVector{ScientificTypesBase.Continuous}` +Source @704 ⏎ `AbstractVector{ScientificTypesBase.Continuous}` ```` **First layer and target transformation:** @@ -396,13 +396,13 @@ z = MLJ.transform(mach2, y) ```` ```` -Node @022 → UnivariateBoxCoxTransformer(…) +Node @116 → UnivariateBoxCoxTransformer(…) args: - 1: Source @555 + 1: Source @704 formula: transform( machine(UnivariateBoxCoxTransformer(n = 171, …), …), - Source @555, + Source @704, ) ```` @@ -419,10 +419,10 @@ zhat = 0.5*predict(mach3, W) + 0.5*predict(mach4, W) ```` ```` -Node @135 +Node @464 args: - 1: Node @083 - 2: Node @797 + 1: Node @919 + 2: Node @316 formula: +( var"#*##0#*##1"( @@ -430,7 +430,7 @@ Node @135 machine(RidgeRegressor(lambda = 0.1, …), …), transform( machine(Standardizer(features = Symbol[], …), …), - Source @100, + Source @991, ), ), ), @@ -439,7 +439,7 @@ Node @135 machine(RandomForestRegressor(max_depth = -1, …), …), transform( machine(Standardizer(features = Symbol[], …), …), - Source @100, + Source @991, ), ), ), @@ -453,9 +453,9 @@ yhat = inverse_transform(mach2, zhat) ```` ```` -Node @539 → UnivariateBoxCoxTransformer(…) +Node @061 → UnivariateBoxCoxTransformer(…) args: - 1: Node @135 + 1: Node @464 formula: inverse_transform( machine(UnivariateBoxCoxTransformer(n = 171, …), …), @@ -465,7 +465,7 @@ Node @539 → UnivariateBoxCoxTransformer(…) machine(RidgeRegressor(lambda = 0.1, …), …), transform( machine(Standardizer(features = Symbol[], …), …), - Source @100, + Source @991, ), ), ), @@ -474,7 +474,7 @@ Node @539 → UnivariateBoxCoxTransformer(…) machine(RandomForestRegressor(max_depth = -1, …), …), transform( machine(Standardizer(features = Symbol[], …), …), - Source @100, + Source @991, ), ), ), @@ -491,9 +491,9 @@ yhat(rows=1:3) ```` 3-element Vector{Float64}: - 4.1049434111127105 - 4.674958202525163 - 4.112097582032496 + 1.5198312953553146 + 0.8824006167264453 + 1.57930874294795 ```` Now for the new model type: @@ -547,20 +547,20 @@ PerformanceEvaluation object with these fields: per_fold, per_observation, fitted_params_per_fold, report_per_fold, train_test_rows, resampling, repeats -Tag: CompositeModel-983 +Tag: CompositeModel-665 Extract: ┌───┬────────────────────────┬───────────┬─────────────┬─────────┐ │ │ measure │ operation │ measurement │ 1.96*SE │ ├───┼────────────────────────┼───────────┼─────────────┼─────────┤ -│ A │ RootMeanSquaredError() │ predict │ 3.87 │ 0.46 │ -│ B │ LPLoss( │ predict │ 2.454 │ 0.04 │ +│ A │ RootMeanSquaredError() │ predict │ 3.95 │ 0.68 │ +│ B │ LPLoss( │ predict │ 2.49 │ 0.26 │ │ │ p = 1) │ │ │ │ └───┴────────────────────────┴───────────┴─────────────┴─────────┘ ┌───┬──────────────────────────────────────┐ │ │ per_fold │ ├───┼──────────────────────────────────────┤ -│ A │ [4.45, 4.28, 3.41, 3.3, 4.21, 3.37] │ -│ B │ [2.45, 2.46, 2.46, 2.38, 2.53, 2.45] │ +│ A │ [3.09, 5.37, 3.61, 3.81, 3.63, 3.82] │ +│ B │ [2.17, 2.98, 2.62, 2.24, 2.35, 2.57] │ └───┴──────────────────────────────────────┘ Apply `describe` to this result for a named tuple summary. ```` diff --git a/docs/src/notebooks/MLJTutorial/99_solution_to_exercises/exercise_7c.png b/docs/src/notebooks/MLJTutorial/99_solution_to_exercises/exercise_7c.png index 08ac21b..c1bf480 100644 Binary files a/docs/src/notebooks/MLJTutorial/99_solution_to_exercises/exercise_7c.png and b/docs/src/notebooks/MLJTutorial/99_solution_to_exercises/exercise_7c.png differ diff --git a/docs/src/notebooks/MLJTutorial/99_solution_to_exercises/notebook.md b/docs/src/notebooks/MLJTutorial/99_solution_to_exercises/notebook.md index 5cca734..4831a1b 100644 --- a/docs/src/notebooks/MLJTutorial/99_solution_to_exercises/notebook.md +++ b/docs/src/notebooks/MLJTutorial/99_solution_to_exercises/notebook.md @@ -50,8 +50,8 @@ A = rand(2, 3) ```` 2×3 Matrix{Float64}: - 0.301055 0.691879 0.504642 - 0.572222 0.788816 0.738401 + 0.52628 0.0908301 0.313487 + 0.798614 0.326598 0.49897 ```` ````@julia @@ -77,8 +77,8 @@ Asparse = sparse(A) ```` 2×3 SparseArrays.SparseMatrixCSC{Float64, Int64} with 6 stored entries: - 0.301055 0.691879 0.504642 - 0.572222 0.788816 0.738401 + 0.52628 0.0908301 0.313487 + 0.798614 0.326598 0.49897 ```` ````@julia @@ -95,8 +95,8 @@ C = coerce(A, Multiclass) ```` 2×3 CategoricalArrays.CategoricalArray{Float64,2,UInt32}: - 0.301055 0.691879 0.504642 - 0.572222 0.788816 0.738401 + 0.52628 0.0908301 0.313487 + 0.798614 0.326598 0.49897 ```` ````@julia @@ -326,16 +326,16 @@ y4 = [n_devices(row.salary) for row in eachrow(X4)] ```` 10-element Vector{Int64}: + 1 + 6 2 3 - 2 3 3 - 0 - 2 - 1 - 1 + 3 + 4 2 + 5 ```` 4(a) @@ -411,10 +411,10 @@ pretty(X) │ Float64 │ Float64 │ │ Continuous │ Continuous │ ├────────────┼────────────┤ -│ 0.11433 │ 0.538742 │ -│ 0.0704153 │ 0.19053 │ -│ 0.130344 │ 0.0186654 │ -│ 0.760926 │ 0.454443 │ +│ 0.350305 │ 0.767919 │ +│ 0.274291 │ 0.289089 │ +│ 0.184154 │ 0.321137 │ +│ 0.0530712 │ 0.558397 │ └────────────┴────────────┘ ```` @@ -590,19 +590,19 @@ PerformanceEvaluation object with these fields: per_fold, per_observation, fitted_params_per_fold, report_per_fold, train_test_rows, resampling, repeats -Tag: RandomForestClassifier-355 +Tag: RandomForestClassifier-111 Extract: ┌──────────────────────┬───────────┬─────────────┬─────────┐ │ measure │ operation │ measurement │ 1.96*SE │ ├──────────────────────┼───────────┼─────────────┼─────────┤ -│ LogLoss( │ predict │ 1.27 │ 0.33 │ +│ LogLoss( │ predict │ 1.09 │ 0.4 │ │ tol = 2.22045e-16) │ │ │ │ └──────────────────────┴───────────┴─────────────┴─────────┘ -┌──────────────────────────────────────┐ -│ per_fold │ -├──────────────────────────────────────┤ -│ [1.27, 1.4, 1.79, 1.27, 1.26, 0.616] │ -└──────────────────────────────────────┘ +┌─────────────────────────────────────────┐ +│ per_fold │ +├─────────────────────────────────────────┤ +│ [1.32, 1.35, 1.78, 0.727, 0.729, 0.639] │ +└─────────────────────────────────────────┘ Apply `describe` to this result for a named tuple summary. ```` @@ -661,7 +661,7 @@ err_forest = ```` ```` -1.290842672700936 +0.9752533003598175 ```` #### Exercise 7 @@ -727,19 +727,19 @@ PerformanceEvaluation object with these fields: per_fold, per_observation, fitted_params_per_fold, report_per_fold, train_test_rows, resampling, repeats -Tag: ProbabilisticPipeline-524 +Tag: ProbabilisticPipeline-258 Extract: ┌──────────────────────┬───────────┬─────────────┬─────────┐ │ measure │ operation │ measurement │ 1.96*SE │ ├──────────────────────┼───────────┼─────────────┼─────────┤ -│ LogLoss( │ predict │ 0.85 │ 0.11 │ +│ LogLoss( │ predict │ 0.89 │ 0.11 │ │ tol = 2.22045e-16) │ │ │ │ └──────────────────────┴───────────┴─────────────┴─────────┘ -┌──────────────────────────────────────────┐ -│ per_fold │ -├──────────────────────────────────────────┤ -│ [0.927, 1.02, 0.773, 0.85, 0.875, 0.665] │ -└──────────────────────────────────────────┘ +┌───────────────────────────────────────────┐ +│ per_fold │ +├───────────────────────────────────────────┤ +│ [0.827, 1.11, 0.828, 0.948, 0.842, 0.787] │ +└───────────────────────────────────────────┘ Apply `describe` to this result for a named tuple summary. ```` @@ -831,7 +831,7 @@ PerformanceEvaluation object with these fields: per_fold, per_observation, fitted_params_per_fold, report_per_fold, train_test_rows, resampling, repeats -Tag: DeterministicPipeline-491 +Tag: DeterministicPipeline-391 Extract: ┌──────────┬───────────┬─────────────┬─────────┐ │ measure │ operation │ measurement │ 1.96*SE │ @@ -858,7 +858,7 @@ PerformanceEvaluation object with these fields: per_fold, per_observation, fitted_params_per_fold, report_per_fold, train_test_rows, resampling, repeats -Tag: DeterministicTunedModel-859 +Tag: DeterministicTunedModel-238 Extract: ┌──────────┬───────────┬─────────────┬─────────┐ │ measure │ operation │ measurement │ 1.96*SE │ diff --git a/docs/src/notebooks/UsingMLJ/01_basics/notebook.md b/docs/src/notebooks/UsingMLJ/01_basics/notebook.md index 662e437..d9efcfe 100644 --- a/docs/src/notebooks/UsingMLJ/01_basics/notebook.md +++ b/docs/src/notebooks/UsingMLJ/01_basics/notebook.md @@ -14,14 +14,13 @@ given [here](@ref instructions). ### Video Timings - 00:00 Introduction -- 01:05 Goals -- 01:48 Prerequisites and Getting Help -- 02:21 Supervised Learning Recap -- 05:31 Models and Machines -- 09:28 **Live Coding:** Regression -- 28:55 Scientific Types -- 32:29 **Live Coding:** Scitypes and Classification -- 55:45 End +- 01:10 Goals +- 01:57 Prerequisites and Getting Help +- 02:33 Supervised Learning Recap +- 05:48 Models and Machines +- 09:49 **Live Coding:** Regression +- 29:20 Scientific Types +- 32:52 **Live Coding:** Scitypes and Classification We start by inspecting the packages, and their exact versions, in the currently active package environment: @@ -199,8 +198,8 @@ fitted_params(mach) ```` (forest = Ensemble of Decision Trees Trees: 100 -Avg Leaves: 146.21 -Avg Depth: 14.61,) +Avg Leaves: 146.19 +Avg Depth: 14.53,) ```` ````@julia @@ -219,9 +218,9 @@ predict(mach, X)[1:3] ```` 3-element Vector{Float64}: - 27.011 - 22.410999999999994 - 34.27 + 26.528000000000002 + 22.439 + 34.312 ```` Predict in the `test` rows: @@ -237,7 +236,7 @@ mae(ypred, y[test]) ```` ```` -4.574237623762377 +4.458702970297029 ```` `mae` is actually just an alias: @@ -284,14 +283,14 @@ PerformanceEvaluation object with these fields: per_fold, per_observation, fitted_params_per_fold, report_per_fold, train_test_rows, resampling, repeats -Tag: RandomForestRegressor-336 +Tag: RandomForestRegressor-330 Extract: ┌────────────┬───────────┬─────────────┐ │ measure │ operation │ measurement │ ├────────────┼───────────┼─────────────┤ -│ LPLoss( │ predict │ 4.57 │ +│ LPLoss( │ predict │ 4.46 │ │ p = 1) │ │ │ -│ RSquared() │ predict │ 0.33 │ +│ RSquared() │ predict │ 0.356 │ └────────────┴───────────┴─────────────┘ Apply `describe` to this result for a named tuple summary. ```` @@ -313,21 +312,21 @@ PerformanceEvaluation object with these fields: per_fold, per_observation, fitted_params_per_fold, report_per_fold, train_test_rows, resampling, repeats -Tag: RandomForestRegressor-817 +Tag: RandomForestRegressor-275 Extract: ┌───┬────────────┬───────────┬─────────────┬─────────┐ │ │ measure │ operation │ measurement │ 1.96*SE │ ├───┼────────────┼───────────┼─────────────┼─────────┤ -│ A │ LPLoss( │ predict │ 3.05 │ 0.89 │ +│ A │ LPLoss( │ predict │ 3.03 │ 0.87 │ │ │ p = 1) │ │ │ │ -│ B │ RSquared() │ predict │ 0.65 │ 0.16 │ +│ B │ RSquared() │ predict │ 0.66 │ 0.15 │ └───┴────────────┴───────────┴─────────────┴─────────┘ -┌───┬─────────────────────────────────────────┐ -│ │ per_fold │ -├───┼─────────────────────────────────────────┤ -│ A │ [2.27, 2.36, 3.25, 2.48, 4.97, 2.96] │ -│ B │ [0.723, 0.81, 0.742, 0.806, 0.441, 0.4] │ -└───┴─────────────────────────────────────────┘ +┌───┬────────────────────────────────────────────┐ +│ │ per_fold │ +├───┼────────────────────────────────────────────┤ +│ A │ [2.23, 2.43, 3.37, 2.47, 4.89, 2.83] │ +│ B │ [0.726, 0.811, 0.728, 0.827, 0.438, 0.447] │ +└───┴────────────────────────────────────────────┘ Apply `describe` to this result for a named tuple summary. ```` @@ -350,21 +349,21 @@ PerformanceEvaluation object with these fields: per_fold, per_observation, fitted_params_per_fold, report_per_fold, train_test_rows, resampling, repeats -Tag: RandomForestRegressor-621 +Tag: RandomForestRegressor-131 Extract: ┌───┬────────────┬───────────┬─────────────┬─────────┐ │ │ measure │ operation │ measurement │ 1.96*SE │ ├───┼────────────┼───────────┼─────────────┼─────────┤ -│ A │ LPLoss( │ predict │ 2.48 │ 0.1 │ +│ A │ LPLoss( │ predict │ 2.429 │ 0.095 │ │ │ p = 1) │ │ │ │ -│ B │ RSquared() │ predict │ 0.817 │ 0.038 │ +│ B │ RSquared() │ predict │ 0.84 │ 0.012 │ └───┴────────────┴───────────┴─────────────┴─────────┘ -┌───┬──────────────────────────────────────────────────────────────────────┐ -│ │ per_fold │ -├───┼──────────────────────────────────────────────────────────────────────┤ -│ A │ [2.37, 2.74, 2.27, 2.42, 2.75, 2.54, 2.36, 2.48, 2.41, 2.5] │ -│ B │ [0.87, 0.702, 0.857, 0.848, 0.824, 0.72, 0.843, 0.834, 0.839, 0.834] │ -└───┴──────────────────────────────────────────────────────────────────────┘ +┌───┬───────────────────────────────────────────────────────────────────────┐ +│ │ per_fold │ +├───┼───────────────────────────────────────────────────────────────────────┤ +│ A │ [2.29, 2.52, 2.51, 2.33, 2.41, 2.52, 2.52, 2.25, 2.26, 2.68] │ +│ B │ [0.825, 0.853, 0.841, 0.851, 0.83, 0.846, 0.853, 0.858, 0.851, 0.795] │ +└───┴───────────────────────────────────────────────────────────────────────┘ Apply `describe` to this result for a named tuple summary. ```` @@ -374,8 +373,8 @@ e.uncertainty_radius_95 ```` 2-element Vector{Float64}: - 0.10349121844149421 - 0.03773282475420102 + 0.09488695112138384 + 0.012422766062865595 ```` # Interlude on scientific types @@ -620,8 +619,8 @@ first(yprob, 5) 5-element CategoricalDistributions.UnivariateFiniteVector{ScientificTypesBase.Multiclass{2}, InlineStrings.String7, UInt32, Float64}: UnivariateFinite{ScientificTypesBase.Multiclass{2}}(<=50K=>1.0, >50K=>0.0) UnivariateFinite{ScientificTypesBase.Multiclass{2}}(<=50K=>1.0, >50K=>0.0) - UnivariateFinite{ScientificTypesBase.Multiclass{2}}(<=50K=>0.89, >50K=>0.11) - UnivariateFinite{ScientificTypesBase.Multiclass{2}}(<=50K=>0.28, >50K=>0.72) + UnivariateFinite{ScientificTypesBase.Multiclass{2}}(<=50K=>0.9, >50K=>0.1) + UnivariateFinite{ScientificTypesBase.Multiclass{2}}(<=50K=>0.25, >50K=>0.75) UnivariateFinite{ScientificTypesBase.Multiclass{2}}(<=50K=>1.0, >50K=>0.0) ```` @@ -632,8 +631,8 @@ yprob[3] ```` UnivariateFinite{ScientificTypesBase.Multiclass{2}} ┌ ┐ - <=50K ┤■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 0.89 - >50K ┤■■■■ 0.11 + <=50K ┤■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 0.9 + >50K ┤■■■■ 0.1 └ ┘ ```` @@ -644,7 +643,7 @@ accuracy(ypoint, y[test]) ```` ```` -0.7821057480677689 +0.7817986384808312 ```` ````@julia @@ -652,7 +651,7 @@ log_loss(yprob, y[test]) ```` ```` -1.53133125245395 +1.532945976938586 ```` Evaluate with one command: @@ -672,13 +671,13 @@ PerformanceEvaluation object with these fields: per_fold, per_observation, fitted_params_per_fold, report_per_fold, train_test_rows, resampling, repeats -Tag: RandomForestClassifier-321 +Tag: RandomForestClassifier-494 Extract: ┌──────────────────────┬──────────────┬─────────────┐ │ measure │ operation │ measurement │ ├──────────────────────┼──────────────┼─────────────┤ │ Accuracy() │ predict_mode │ 0.782 │ -│ LogLoss( │ predict │ 1.55 │ +│ LogLoss( │ predict │ 1.53 │ │ tol = 2.22045e-16) │ │ │ └──────────────────────┴──────────────┴─────────────┘ Apply `describe` to this result for a named tuple summary. diff --git a/docs/src/notebooks/UsingMLJ/02_model_composition/notebook.md b/docs/src/notebooks/UsingMLJ/02_model_composition/notebook.md index 7dde1c6..2162e5c 100644 --- a/docs/src/notebooks/UsingMLJ/02_model_composition/notebook.md +++ b/docs/src/notebooks/UsingMLJ/02_model_composition/notebook.md @@ -14,16 +14,15 @@ given [here](@ref instructions). ### Video Timings - 00:00 Introduction -- 00:08 Goals -- 01:19 Prerequisites and Getting Help -- 02:00 Composite Models Defined -- 04:13 Model Pipelines -- 05:59 Data Leakage -- 08:37 Target Transformations -- 12:43 Live Coding: Pipelines -- 18:42 Live Coding: Target Transformations -- 20:35 Other Model Wrappers -- 25:02 End +- 00:13 Goals +- 01:28 Prerequisites and Getting Help +- 02:12 Composite Models Defined +- 04:28 Model Pipelines +- 06:07 Data Leakage +- 09:00 Target Transformations +- 13:10 **Live Coding:** Pipelines +- 19:11 **Live Coding:** Target Transformations +- 21:01 Other Model Wrappers ````@julia using MLJ @@ -160,7 +159,7 @@ PerformanceEvaluation object with these fields: per_fold, per_observation, fitted_params_per_fold, report_per_fold, train_test_rows, resampling, repeats -Tag: DeterministicPipeline-500 +Tag: DeterministicPipeline-776 Extract: ┌──────────┬───────────┬─────────────┬─────────┐ │ measure │ operation │ measurement │ 1.96*SE │ @@ -225,7 +224,7 @@ PerformanceEvaluation object with these fields: per_fold, per_observation, fitted_params_per_fold, report_per_fold, train_test_rows, resampling, repeats -Tag: TransformedTargetModelDeterministic-470 +Tag: TransformedTargetModelDeterministic-712 Extract: ┌──────────┬───────────┬─────────────┬─────────┐ │ measure │ operation │ measurement │ 1.96*SE │ diff --git a/docs/src/notebooks/UsingMLJ/03_model_tuning/learning_curve.svg b/docs/src/notebooks/UsingMLJ/03_model_tuning/learning_curve.svg index 1f9c598..7df8fcb 100644 --- a/docs/src/notebooks/UsingMLJ/03_model_tuning/learning_curve.svg +++ b/docs/src/notebooks/UsingMLJ/03_model_tuning/learning_curve.svg @@ -1,46 +1,46 @@ - + - + - + - + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/src/notebooks/UsingMLJ/03_model_tuning/notebook.md b/docs/src/notebooks/UsingMLJ/03_model_tuning/notebook.md index a6661c2..ea90790 100644 --- a/docs/src/notebooks/UsingMLJ/03_model_tuning/notebook.md +++ b/docs/src/notebooks/UsingMLJ/03_model_tuning/notebook.md @@ -14,15 +14,14 @@ given [here](@ref instructions). ### Video Timings - 00:00 Introduction -- 00:06 Goals -- 01:12 Prerequisites and Getting Help -- 01:27 Live Coding: Learning Curves -- 13:27 Tuning as Model Wrapper -- 18:45 Live Coding: Grid Search -- 26:17 Live Coding: Random Search -- 30:48 Nested Resampling -- 33:11 Final Observations -- 36:59 End +- 00:11 Goals +- 01:21 Prerequisites and Getting Help +- 01:42 **Live Coding:** Learning Curves +- 13:57 Tuning as Model Wrapper +- 18:53 **Live Coding:** Grid Search +- 26:32 **Live Coding:** Random Search +- 31:07 Nested Resampling +- 33:31 Final Observations ## Part I. Learning Curves @@ -94,7 +93,7 @@ PerformanceEvaluation object with these fields: per_fold, per_observation, fitted_params_per_fold, report_per_fold, train_test_rows, resampling, repeats -Tag: ProbabilisticPipeline-489 +Tag: ProbabilisticPipeline-772 Extract: ┌──────────┬──────────────┬─────────────┬─────────┐ │ measure │ operation │ measurement │ 1.96*SE │ @@ -161,7 +160,7 @@ savefig("learning_curve.svg"); ```` [ Info: Training machine(DeterministicTunedModel(model = DeterministicPipeline(continuous_encoder = ContinuousEncoder(drop_last = false, …), …), …), …). [ Info: Attempting to evaluate 30 models. - Evaluating over 30 metamodels: 0%[> ] ETA: N/A Evaluating over 30 metamodels: 3%[> ] ETA: 0:00:00 Evaluating over 30 metamodels: 7%[=> ] ETA: 0:00:00 Evaluating over 30 metamodels: 10%[==> ] ETA: 0:00:01 Evaluating over 30 metamodels: 13%[===> ] ETA: 0:00:02 Evaluating over 30 metamodels: 17%[====> ] ETA: 0:00:02 Evaluating over 30 metamodels: 20%[=====> ] ETA: 0:00:02 Evaluating over 30 metamodels: 23%[=====> ] ETA: 0:00:04 Evaluating over 30 metamodels: 27%[======> ] ETA: 0:00:04 Evaluating over 30 metamodels: 30%[=======> ] ETA: 0:00:04 Evaluating over 30 metamodels: 33%[========> ] ETA: 0:00:04 Evaluating over 30 metamodels: 37%[=========> ] ETA: 0:00:04 Evaluating over 30 metamodels: 40%[==========> ] ETA: 0:00:04 Evaluating over 30 metamodels: 43%[==========> ] ETA: 0:00:04 Evaluating over 30 metamodels: 47%[===========> ] ETA: 0:00:04 Evaluating over 30 metamodels: 50%[============> ] ETA: 0:00:04 Evaluating over 30 metamodels: 53%[=============> ] ETA: 0:00:05 Evaluating over 30 metamodels: 57%[==============> ] ETA: 0:00:05 Evaluating over 30 metamodels: 60%[===============> ] ETA: 0:00:05 Evaluating over 30 metamodels: 63%[===============> ] ETA: 0:00:05 Evaluating over 30 metamodels: 67%[================> ] ETA: 0:00:05 Evaluating over 30 metamodels: 70%[=================> ] ETA: 0:00:05 Evaluating over 30 metamodels: 73%[==================> ] ETA: 0:00:05 Evaluating over 30 metamodels: 77%[===================> ] ETA: 0:00:04 Evaluating over 30 metamodels: 80%[====================> ] ETA: 0:00:04 Evaluating over 30 metamodels: 83%[====================> ] ETA: 0:00:04 Evaluating over 30 metamodels: 87%[=====================> ] ETA: 0:00:03 Evaluating over 30 metamodels: 90%[======================> ] ETA: 0:00:03 Evaluating over 30 metamodels: 93%[=======================> ] ETA: 0:00:02 Evaluating over 30 metamodels: 97%[========================>] ETA: 0:00:01 Evaluating over 30 metamodels: 100%[=========================] Time: 0:00:35 + Evaluating over 30 metamodels: 0%[> ] ETA: N/A Evaluating over 30 metamodels: 3%[> ] ETA: 0:00:00 Evaluating over 30 metamodels: 7%[=> ] ETA: 0:00:01 Evaluating over 30 metamodels: 10%[==> ] ETA: 0:00:02 Evaluating over 30 metamodels: 13%[===> ] ETA: 0:00:02 Evaluating over 30 metamodels: 17%[====> ] ETA: 0:00:02 Evaluating over 30 metamodels: 20%[=====> ] ETA: 0:00:05 Evaluating over 30 metamodels: 23%[=====> ] ETA: 0:00:04 Evaluating over 30 metamodels: 27%[======> ] ETA: 0:00:04 Evaluating over 30 metamodels: 30%[=======> ] ETA: 0:00:04 Evaluating over 30 metamodels: 33%[========> ] ETA: 0:00:04 Evaluating over 30 metamodels: 37%[=========> ] ETA: 0:00:04 Evaluating over 30 metamodels: 40%[==========> ] ETA: 0:00:04 Evaluating over 30 metamodels: 43%[==========> ] ETA: 0:00:04 Evaluating over 30 metamodels: 47%[===========> ] ETA: 0:00:04 Evaluating over 30 metamodels: 50%[============> ] ETA: 0:00:04 Evaluating over 30 metamodels: 53%[=============> ] ETA: 0:00:04 Evaluating over 30 metamodels: 57%[==============> ] ETA: 0:00:05 Evaluating over 30 metamodels: 60%[===============> ] ETA: 0:00:05 Evaluating over 30 metamodels: 63%[===============> ] ETA: 0:00:05 Evaluating over 30 metamodels: 67%[================> ] ETA: 0:00:05 Evaluating over 30 metamodels: 70%[=================> ] ETA: 0:00:05 Evaluating over 30 metamodels: 73%[==================> ] ETA: 0:00:04 Evaluating over 30 metamodels: 77%[===================> ] ETA: 0:00:04 Evaluating over 30 metamodels: 80%[====================> ] ETA: 0:00:04 Evaluating over 30 metamodels: 83%[====================> ] ETA: 0:00:04 Evaluating over 30 metamodels: 87%[=====================> ] ETA: 0:00:03 Evaluating over 30 metamodels: 90%[======================> ] ETA: 0:00:03 Evaluating over 30 metamodels: 93%[=======================> ] ETA: 0:00:02 Evaluating over 30 metamodels: 97%[========================>] ETA: 0:00:01 Evaluating over 30 metamodels: 100%[=========================] Time: 0:00:32 ```` @@ -238,7 +237,7 @@ savefig("tuned_model_grid.svg"); ```` [ Info: Training machine(DeterministicTunedModel(model = DeterministicPipeline(continuous_encoder = ContinuousEncoder(drop_last = false, …), …), …), …). [ Info: Attempting to evaluate 36 models. - Evaluating over 36 metamodels: 6%[=> ] ETA: 0:00:55 Evaluating over 36 metamodels: 8%[==> ] ETA: 0:00:51 Evaluating over 36 metamodels: 11%[==> ] ETA: 0:00:43 Evaluating over 36 metamodels: 14%[===> ] ETA: 0:00:37 Evaluating over 36 metamodels: 17%[====> ] ETA: 0:00:34 Evaluating over 36 metamodels: 19%[====> ] ETA: 0:00:36 Evaluating over 36 metamodels: 22%[=====> ] ETA: 0:00:35 Evaluating over 36 metamodels: 25%[======> ] ETA: 0:00:34 Evaluating over 36 metamodels: 28%[======> ] ETA: 0:00:35 Evaluating over 36 metamodels: 31%[=======> ] ETA: 0:00:34 Evaluating over 36 metamodels: 33%[========> ] ETA: 0:00:32 Evaluating over 36 metamodels: 36%[=========> ] ETA: 0:00:31 Evaluating over 36 metamodels: 39%[=========> ] ETA: 0:00:29 Evaluating over 36 metamodels: 42%[==========> ] ETA: 0:00:27 Evaluating over 36 metamodels: 44%[===========> ] ETA: 0:00:25 Evaluating over 36 metamodels: 47%[===========> ] ETA: 0:00:24 Evaluating over 36 metamodels: 50%[============> ] ETA: 0:00:22 Evaluating over 36 metamodels: 53%[=============> ] ETA: 0:00:22 Evaluating over 36 metamodels: 56%[=============> ] ETA: 0:00:20 Evaluating over 36 metamodels: 58%[==============> ] ETA: 0:00:19 Evaluating over 36 metamodels: 61%[===============> ] ETA: 0:00:17 Evaluating over 36 metamodels: 64%[===============> ] ETA: 0:00:17 Evaluating over 36 metamodels: 67%[================> ] ETA: 0:00:15 Evaluating over 36 metamodels: 69%[=================> ] ETA: 0:00:14 Evaluating over 36 metamodels: 72%[==================> ] ETA: 0:00:12 Evaluating over 36 metamodels: 75%[==================> ] ETA: 0:00:11 Evaluating over 36 metamodels: 78%[===================> ] ETA: 0:00:10 Evaluating over 36 metamodels: 81%[====================> ] ETA: 0:00:09 Evaluating over 36 metamodels: 83%[====================> ] ETA: 0:00:07 Evaluating over 36 metamodels: 86%[=====================> ] ETA: 0:00:06 Evaluating over 36 metamodels: 89%[======================> ] ETA: 0:00:05 Evaluating over 36 metamodels: 92%[======================> ] ETA: 0:00:04 Evaluating over 36 metamodels: 94%[=======================> ] ETA: 0:00:02 Evaluating over 36 metamodels: 97%[========================>] ETA: 0:00:01 Evaluating over 36 metamodels: 100%[=========================] Time: 0:00:42 + Evaluating over 36 metamodels: 6%[=> ] ETA: 0:00:47 Evaluating over 36 metamodels: 8%[==> ] ETA: 0:00:47 Evaluating over 36 metamodels: 11%[==> ] ETA: 0:00:47 Evaluating over 36 metamodels: 14%[===> ] ETA: 0:00:41 Evaluating over 36 metamodels: 17%[====> ] ETA: 0:00:36 Evaluating over 36 metamodels: 19%[====> ] ETA: 0:00:37 Evaluating over 36 metamodels: 22%[=====> ] ETA: 0:00:40 Evaluating over 36 metamodels: 25%[======> ] ETA: 0:00:37 Evaluating over 36 metamodels: 28%[======> ] ETA: 0:00:35 Evaluating over 36 metamodels: 31%[=======> ] ETA: 0:00:33 Evaluating over 36 metamodels: 33%[========> ] ETA: 0:00:30 Evaluating over 36 metamodels: 36%[=========> ] ETA: 0:00:29 Evaluating over 36 metamodels: 39%[=========> ] ETA: 0:00:28 Evaluating over 36 metamodels: 42%[==========> ] ETA: 0:00:26 Evaluating over 36 metamodels: 44%[===========> ] ETA: 0:00:25 Evaluating over 36 metamodels: 47%[===========> ] ETA: 0:00:23 Evaluating over 36 metamodels: 50%[============> ] ETA: 0:00:23 Evaluating over 36 metamodels: 53%[=============> ] ETA: 0:00:22 Evaluating over 36 metamodels: 56%[=============> ] ETA: 0:00:20 Evaluating over 36 metamodels: 58%[==============> ] ETA: 0:00:19 Evaluating over 36 metamodels: 61%[===============> ] ETA: 0:00:18 Evaluating over 36 metamodels: 64%[===============> ] ETA: 0:00:17 Evaluating over 36 metamodels: 67%[================> ] ETA: 0:00:15 Evaluating over 36 metamodels: 69%[=================> ] ETA: 0:00:14 Evaluating over 36 metamodels: 72%[==================> ] ETA: 0:00:13 Evaluating over 36 metamodels: 75%[==================> ] ETA: 0:00:12 Evaluating over 36 metamodels: 78%[===================> ] ETA: 0:00:11 Evaluating over 36 metamodels: 81%[====================> ] ETA: 0:00:09 Evaluating over 36 metamodels: 83%[====================> ] ETA: 0:00:08 Evaluating over 36 metamodels: 86%[=====================> ] ETA: 0:00:07 Evaluating over 36 metamodels: 89%[======================> ] ETA: 0:00:05 Evaluating over 36 metamodels: 92%[======================> ] ETA: 0:00:04 Evaluating over 36 metamodels: 94%[=======================> ] ETA: 0:00:03 Evaluating over 36 metamodels: 97%[========================>] ETA: 0:00:01 Evaluating over 36 metamodels: 100%[=========================] Time: 0:00:49 ```` @@ -320,7 +319,7 @@ savefig("tuned_model_random.svg"); ```` [ Info: Training machine(DeterministicTunedModel(model = DeterministicPipeline(continuous_encoder = ContinuousEncoder(drop_last = false, …), …), …), …). [ Info: Attempting to evaluate 40 models. - Evaluating over 40 metamodels: 0%[> ] ETA: N/A Evaluating over 40 metamodels: 2%[> ] ETA: 0:00:52 Evaluating over 40 metamodels: 5%[=> ] ETA: 0:00:50 Evaluating over 40 metamodels: 8%[=> ] ETA: 0:00:43 Evaluating over 40 metamodels: 10%[==> ] ETA: 0:00:42 Evaluating over 40 metamodels: 12%[===> ] ETA: 0:00:42 Evaluating over 40 metamodels: 15%[===> ] ETA: 0:00:43 Evaluating over 40 metamodels: 18%[====> ] ETA: 0:00:41 Evaluating over 40 metamodels: 20%[=====> ] ETA: 0:00:40 Evaluating over 40 metamodels: 22%[=====> ] ETA: 0:00:37 Evaluating over 40 metamodels: 25%[======> ] ETA: 0:00:39 Evaluating over 40 metamodels: 28%[======> ] ETA: 0:00:36 Evaluating over 40 metamodels: 30%[=======> ] ETA: 0:00:35 Evaluating over 40 metamodels: 32%[========> ] ETA: 0:00:33 Evaluating over 40 metamodels: 35%[========> ] ETA: 0:00:32 Evaluating over 40 metamodels: 38%[=========> ] ETA: 0:00:32 Evaluating over 40 metamodels: 40%[==========> ] ETA: 0:00:30 Evaluating over 40 metamodels: 42%[==========> ] ETA: 0:00:30 Evaluating over 40 metamodels: 45%[===========> ] ETA: 0:00:28 Evaluating over 40 metamodels: 48%[===========> ] ETA: 0:00:26 Evaluating over 40 metamodels: 50%[============> ] ETA: 0:00:26 Evaluating over 40 metamodels: 52%[=============> ] ETA: 0:00:24 Evaluating over 40 metamodels: 55%[=============> ] ETA: 0:00:23 Evaluating over 40 metamodels: 58%[==============> ] ETA: 0:00:22 Evaluating over 40 metamodels: 60%[===============> ] ETA: 0:00:20 Evaluating over 40 metamodels: 62%[===============> ] ETA: 0:00:20 Evaluating over 40 metamodels: 65%[================> ] ETA: 0:00:18 Evaluating over 40 metamodels: 68%[================> ] ETA: 0:00:17 Evaluating over 40 metamodels: 70%[=================> ] ETA: 0:00:15 Evaluating over 40 metamodels: 72%[==================> ] ETA: 0:00:14 Evaluating over 40 metamodels: 75%[==================> ] ETA: 0:00:13 Evaluating over 40 metamodels: 78%[===================> ] ETA: 0:00:11 Evaluating over 40 metamodels: 80%[====================> ] ETA: 0:00:10 Evaluating over 40 metamodels: 82%[====================> ] ETA: 0:00:09 Evaluating over 40 metamodels: 85%[=====================> ] ETA: 0:00:08 Evaluating over 40 metamodels: 88%[=====================> ] ETA: 0:00:06 Evaluating over 40 metamodels: 90%[======================> ] ETA: 0:00:05 Evaluating over 40 metamodels: 92%[=======================> ] ETA: 0:00:04 Evaluating over 40 metamodels: 95%[=======================> ] ETA: 0:00:02 Evaluating over 40 metamodels: 98%[========================>] ETA: 0:00:01 Evaluating over 40 metamodels: 100%[=========================] Time: 0:00:50 + Evaluating over 40 metamodels: 5%[=> ] ETA: 0:00:57 Evaluating over 40 metamodels: 8%[=> ] ETA: 0:00:49 Evaluating over 40 metamodels: 10%[==> ] ETA: 0:00:49 Evaluating over 40 metamodels: 12%[===> ] ETA: 0:00:48 Evaluating over 40 metamodels: 15%[===> ] ETA: 0:00:48 Evaluating over 40 metamodels: 18%[====> ] ETA: 0:00:47 Evaluating over 40 metamodels: 20%[=====> ] ETA: 0:00:45 Evaluating over 40 metamodels: 22%[=====> ] ETA: 0:00:43 Evaluating over 40 metamodels: 25%[======> ] ETA: 0:00:44 Evaluating over 40 metamodels: 28%[======> ] ETA: 0:00:41 Evaluating over 40 metamodels: 30%[=======> ] ETA: 0:00:40 Evaluating over 40 metamodels: 32%[========> ] ETA: 0:00:38 Evaluating over 40 metamodels: 35%[========> ] ETA: 0:00:38 Evaluating over 40 metamodels: 38%[=========> ] ETA: 0:00:36 Evaluating over 40 metamodels: 40%[==========> ] ETA: 0:00:35 Evaluating over 40 metamodels: 42%[==========> ] ETA: 0:00:34 Evaluating over 40 metamodels: 45%[===========> ] ETA: 0:00:33 Evaluating over 40 metamodels: 48%[===========> ] ETA: 0:00:31 Evaluating over 40 metamodels: 50%[============> ] ETA: 0:00:30 Evaluating over 40 metamodels: 52%[=============> ] ETA: 0:00:28 Evaluating over 40 metamodels: 55%[=============> ] ETA: 0:00:26 Evaluating over 40 metamodels: 58%[==============> ] ETA: 0:00:25 Evaluating over 40 metamodels: 60%[===============> ] ETA: 0:00:24 Evaluating over 40 metamodels: 62%[===============> ] ETA: 0:00:22 Evaluating over 40 metamodels: 65%[================> ] ETA: 0:00:21 Evaluating over 40 metamodels: 68%[================> ] ETA: 0:00:19 Evaluating over 40 metamodels: 70%[=================> ] ETA: 0:00:18 Evaluating over 40 metamodels: 72%[==================> ] ETA: 0:00:16 Evaluating over 40 metamodels: 75%[==================> ] ETA: 0:00:14 Evaluating over 40 metamodels: 78%[===================> ] ETA: 0:00:13 Evaluating over 40 metamodels: 80%[====================> ] ETA: 0:00:12 Evaluating over 40 metamodels: 82%[====================> ] ETA: 0:00:10 Evaluating over 40 metamodels: 85%[=====================> ] ETA: 0:00:09 Evaluating over 40 metamodels: 88%[=====================> ] ETA: 0:00:07 Evaluating over 40 metamodels: 90%[======================> ] ETA: 0:00:06 Evaluating over 40 metamodels: 92%[=======================> ] ETA: 0:00:04 Evaluating over 40 metamodels: 95%[=======================> ] ETA: 0:00:03 Evaluating over 40 metamodels: 98%[========================>] ETA: 0:00:01 Evaluating over 40 metamodels: 100%[=========================] Time: 0:00:57 ```` @@ -354,7 +353,7 @@ PerformanceEvaluation object with these fields: per_fold, per_observation, fitted_params_per_fold, report_per_fold, train_test_rows, resampling, repeats -Tag: DeterministicTunedModel-587 +Tag: DeterministicTunedModel-147 Extract: ┌──────────┬───────────┬─────────────┬─────────┐ │ measure │ operation │ measurement │ 1.96*SE │ @@ -377,9 +376,9 @@ Comparing with the baseline computed earlier: ```` ```` -ebase = PerformanceEvaluation("ProbabilisticPipeline-489", 0.568 ± 0.033) -e0 = PerformanceEvaluation("DeterministicPipeline-381", 0.184 ± 0.028) -e1 = PerformanceEvaluation("DeterministicTunedModel-587", 0.175 ± 0.021) +ebase = PerformanceEvaluation("ProbabilisticPipeline-772", 0.568 ± 0.033) +e0 = PerformanceEvaluation("DeterministicPipeline-751", 0.184 ± 0.028) +e1 = PerformanceEvaluation("DeterministicTunedModel-147", 0.175 ± 0.021) ```` @@ -395,8 +394,8 @@ describe.([e0, e1]) |> pretty │ String │ Measurement{Float64} │ │ Textual │ Continuous │ ├─────────────────────────────┼──────────────────────┤ -│ DeterministicPipeline-381 │ 0.184±0.028 │ -│ DeterministicTunedModel-587 │ 0.175±0.021 │ +│ DeterministicPipeline-751 │ 0.184±0.028 │ +│ DeterministicTunedModel-147 │ 0.175±0.021 │ └─────────────────────────────┴──────────────────────┘ ```` diff --git a/docs/src/notebooks/UsingMLJ/03_model_tuning/tuned_model_grid.svg b/docs/src/notebooks/UsingMLJ/03_model_tuning/tuned_model_grid.svg index b284d62..61ea8e7 100644 --- a/docs/src/notebooks/UsingMLJ/03_model_tuning/tuned_model_grid.svg +++ b/docs/src/notebooks/UsingMLJ/03_model_tuning/tuned_model_grid.svg @@ -1,294 +1,294 @@ - + - + - + - + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + - + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/src/notebooks/UsingMLJ/03_model_tuning/tuned_model_random.svg b/docs/src/notebooks/UsingMLJ/03_model_tuning/tuned_model_random.svg index fdc05dd..eb5cc66 100644 --- a/docs/src/notebooks/UsingMLJ/03_model_tuning/tuned_model_random.svg +++ b/docs/src/notebooks/UsingMLJ/03_model_tuning/tuned_model_random.svg @@ -1,269 +1,269 @@ - + - + - + - + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + - + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/src/notebooks/lightning_tour/notebook.md b/docs/src/notebooks/lightning_tour/notebook.md index c912761..5b4c245 100644 --- a/docs/src/notebooks/lightning_tour/notebook.md +++ b/docs/src/notebooks/lightning_tour/notebook.md @@ -246,8 +246,8 @@ mach = machine(self_tuning_pipe, X, y) untrained Machine; does not cache data model: DeterministicTunedModel(model = DeterministicPipeline(continuous_encoder = ContinuousEncoder(drop_last = false, …), …), …) args: - 1: Source @634 ⏎ ScientificTypesBase.Table{AbstractVector{ScientificTypesBase.Continuous}} - 2: Source @471 ⏎ AbstractVector{ScientificTypesBase.Continuous} + 1: Source @585 ⏎ ScientificTypesBase.Table{AbstractVector{ScientificTypesBase.Continuous}} + 2: Source @899 ⏎ AbstractVector{ScientificTypesBase.Continuous} ```` @@ -261,9 +261,9 @@ first(yhat, 3) ```` 3-element Vector{Float32}: - -2.8776388 - -0.45805833 - -0.8760577 + -1.4080157 + -0.40302494 + -1.5811985 ```` Evaluating the "self-tuning" pipeline model's performance using all data and 5-fold @@ -285,21 +285,21 @@ PerformanceEvaluation object with these fields: per_fold, per_observation, fitted_params_per_fold, report_per_fold, train_test_rows, resampling, repeats -Tag: DeterministicTunedModel-157 +Tag: DeterministicTunedModel-336 Extract: ┌───┬────────────┬───────────┬─────────────┬─────────┐ │ │ measure │ operation │ measurement │ 1.96*SE │ ├───┼────────────┼───────────┼─────────────┼─────────┤ -│ A │ LPLoss( │ predict │ 0.17 │ 0.023 │ +│ A │ LPLoss( │ predict │ 0.138 │ 0.043 │ │ │ p = 1) │ │ │ │ -│ B │ RSquared() │ predict │ 0.9699 │ 0.009 │ +│ B │ RSquared() │ predict │ 0.89 │ 0.071 │ └───┴────────────┴───────────┴─────────────┴─────────┘ -┌───┬────────────────────────────────────┐ -│ │ per_fold │ -├───┼────────────────────────────────────┤ -│ A │ [0.155, 0.159, 0.2, 0.19, 0.146] │ -│ B │ [0.96, 0.981, 0.962, 0.969, 0.977] │ -└───┴────────────────────────────────────┘ +┌───┬─────────────────────────────────────┐ +│ │ per_fold │ +├───┼─────────────────────────────────────┤ +│ A │ [0.175, 0.179, 0.144, 0.0735, 0.12] │ +│ B │ [0.85, 0.788, 0.897, 0.971, 0.942] │ +└───┴─────────────────────────────────────┘ Apply `describe` to this result for a named tuple summary. ```` @@ -320,7 +320,7 @@ describe.(evaluations) |> pretty ```` [ Info: Performing evaluations using 1 thread. - Evaluating over 5 folds: 40%[==========> ] ETA: 0:00:11 Evaluating over 5 folds: 60%[===============> ] ETA: 0:00:07 Evaluating over 5 folds: 80%[====================> ] ETA: 0:00:04 Evaluating over 5 folds: 100%[=========================] Time: 0:00:18 + Evaluating over 5 folds: 40%[==========> ] ETA: 0:00:07 Evaluating over 5 folds: 60%[===============> ] ETA: 0:00:06 Evaluating over 5 folds: 80%[====================> ] ETA: 0:00:03 Evaluating over 5 folds: 100%[=========================] Time: 0:00:16 [ Info: Performing evaluations using 1 thread. Evaluating over 5 folds: 40%[==========> ] ETA: 0:00:01 Evaluating over 5 folds: 100%[=========================] Time: 0:00:00 ┌─────────┬──────────────────────┬──────────────────────┐ @@ -328,8 +328,8 @@ describe.(evaluations) |> pretty │ String │ Measurement{Float64} │ Measurement{Float64} │ │ Textual │ Continuous │ Continuous │ ├─────────┼──────────────────────┼──────────────────────┤ -│ booster │ 0.17±0.023 │ 0.9699±0.009 │ -│ dummy │ 1.13±0.15 │ -0.0044±0.0051 │ +│ booster │ 0.138±0.043 │ 0.89±0.071 │ +│ dummy │ 0.497±0.068 │ -0.093±0.14 │ └─────────┴──────────────────────┴──────────────────────┘ ```` diff --git a/learning_curve.svg b/learning_curve.svg index 177bf45..6ef948d 100644 --- a/learning_curve.svg +++ b/learning_curve.svg @@ -1,46 +1,46 @@ - + - + - + - + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/tuned_model_grid.svg b/tuned_model_grid.svg index 94c8ce2..b4fab86 100644 --- a/tuned_model_grid.svg +++ b/tuned_model_grid.svg @@ -1,294 +1,294 @@ - + - + - + - + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + - + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/tuned_model_random.svg b/tuned_model_random.svg index 5f65b2d..3233ede 100644 --- a/tuned_model_random.svg +++ b/tuned_model_random.svg @@ -1,269 +1,269 @@ - + - + - + - + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + - + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +