fit binary decision tree for regression

syntax

tree = fitrtree(tbl,responsevarname)

tree = fitrtree(tbl,formula)

tree = fitrtree(tbl,y)

tree = fitrtree(x,y)

tree = fitrtree(___,name,value)

description

tree = fitrtree(tbl,responsevarname) returns a regression tree based on the input variables (also known as predictors, features, or attributes) in the table tbl and the output (response) contained in tbl.responsevarname. the returned tree is a binary tree where each branching node is split based on the values of a column of tbl.

tree = fitrtree(tbl,formula) returns a regression tree based on the input variables contained in the table tbl. the input formula is an explanatory model of the response and a subset of predictor variables in tbl used to fit tree.

tree = fitrtree(tbl,y) returns a regression tree based on the input variables contained in the table tbl and the output in vector y.

example

tree = fitrtree(x,y) returns a regression tree based on the input variables x and the output y. the returned tree is a binary tree where each branching node is split based on the values of a column of x.

example

tree = fitrtree(___,name,value) specifies options using one or more name-value pair arguments in addition to any of the input argument combinations in previous syntaxes. for example, you can specify observation weights or train a cross-validated model.

examples

construct regression tree

load the sample data.

load carsmall

construct a regression tree using the sample data. the response variable is miles per gallon, mpg.

tree = fitrtree([weight, cylinders],mpg,...
                'categoricalpredictors',2,'minparentsize',20,...
                'predictornames',{'w','c'})

tree = 
  regressiontree
           predictornames: {'w'  'c'}
             responsename: 'y'
    categoricalpredictors: 2
        responsetransform: 'none'
          numobservations: 94
  properties, methods

predict the mileage of 4,000-pound cars with 4, 6, and 8 cylinders.

mpg4kpred = predict(tree,[4000 4; 4000 6; 4000 8])

mpg4kpred = 3×1
   19.2778
   19.2778
   14.3889

control regression tree depth

fitrtree grows deep decision trees by default. you can grow shallower trees to reduce model complexity or computation time. to control the depth of trees, use the 'maxnumsplits', 'minleafsize', or 'minparentsize' name-value pair arguments.

load the carsmall data set. consider displacement, horsepower, and weight as predictors of the response mpg.

load carsmall
x = [displacement horsepower weight];

the default values of the tree-depth controllers for growing regression trees are:

n - 1 for maxnumsplits. n is the training sample size.
1 for minleafsize.
10 for minparentsize.

these default values tend to grow deep trees for large training sample sizes.

train a regression tree using the default values for tree-depth control. cross-validate the model using 10-fold cross-validation.

rng(1); % for reproducibility
mdldefault = fitrtree(x,mpg,'crossval','on');

draw a histogram of the number of imposed splits on the trees. the number of imposed splits is one less than the number of leaves. also, view one of the trees.

numbranches = @(x)sum(x.isbranch);
mdldefaultnumsplits = cellfun(numbranches, mdldefault.trained);
figure;
histogram(mdldefaultnumsplits)

figure contains an axes object. the axes object contains an object of type histogram.

view(mdldefault.trained{1},'mode','graph')

figure regression tree viewer contains an axes object and other objects of type uimenu, uicontrol. the axes object contains 51 objects of type line, text. one or more of the lines displays its values using only markers

the average number of splits is between 14 and 15.

suppose that you want a regression tree that is not as complex (deep) as the ones trained using the default number of splits. train another regression tree, but set the maximum number of splits at 7, which is about half the mean number of splits from the default regression tree. cross-validate the model using 10-fold cross-validation.

mdl7 = fitrtree(x,mpg,'maxnumsplits',7,'crossval','on');
view(mdl7.trained{1},'mode','graph')

figure regression tree viewer contains an axes object and other objects of type uimenu, uicontrol. the axes object contains 27 objects of type line, text. one or more of the lines displays its values using only markers

compare the cross-validation mean squared errors (mses) of the models.

msedefault = kfoldloss(mdldefault)

msedefault = 25.7383

mse7 = kfoldloss(mdl7)

mse7 = 26.5748

mdl7 is much less complex and performs only slightly worse than mdldefault.

optimize regression tree

optimize hyperparameters automatically using fitrtree.

load the carsmall data set.

load carsmall

use weight and horsepower as predictors for mpg. find hyperparameters that minimize five-fold cross-validation loss by using automatic hyperparameter optimization.

for reproducibility, set the random seed and use the 'expected-improvement-plus' acquisition function.

x = [weight,horsepower];
y = mpg;
rng default
mdl = fitrtree(x,y,'optimizehyperparameters','auto',...
    'hyperparameteroptimizationoptions',struct('acquisitionfunctionname',...
    'expected-improvement-plus'))

|======================================================================================|
| iter | eval   | objective:  | objective   | bestsofar   | bestsofar   |  minleafsize |
|      | result | log(1 loss) | runtime     | (observed)  | (estim.)    |              |
|======================================================================================|
|    1 | best   |      3.2818 |     0.29455 |      3.2818 |      3.2818 |           28 |
|    2 | accept |      3.4183 |     0.12915 |      3.2818 |      3.2888 |            1 |
|    3 | best   |      3.1457 |     0.07859 |      3.1457 |      3.1628 |            4 |
|    4 | best   |      2.9885 |    0.069395 |      2.9885 |      2.9885 |            9 |
|    5 | accept |      2.9978 |    0.077333 |      2.9885 |      2.9885 |            7 |
|    6 | accept |      3.0203 |    0.089233 |      2.9885 |      3.0013 |            8 |
|    7 | accept |      2.9885 |    0.053796 |      2.9885 |      2.9981 |            9 |
|    8 | best   |      2.9589 |    0.063884 |      2.9589 |      2.9589 |           10 |
|    9 | accept |       3.078 |     0.11994 |      2.9589 |      2.9888 |           13 |
|   10 | accept |      4.1881 |    0.079741 |      2.9589 |      2.9592 |           50 |
|   11 | accept |      3.4182 |    0.077421 |      2.9589 |      2.9592 |            2 |
|   12 | accept |      3.0376 |     0.14607 |      2.9589 |      2.9591 |            6 |
|   13 | accept |      3.1453 |     0.11396 |      2.9589 |      2.9591 |           20 |
|   14 | accept |      2.9589 |    0.081747 |      2.9589 |       2.959 |           10 |
|   15 | accept |      3.0123 |     0.11077 |      2.9589 |      2.9728 |           11 |
|   16 | accept |      2.9589 |    0.072013 |      2.9589 |      2.9593 |           10 |
|   17 | accept |      3.3055 |    0.085531 |      2.9589 |      2.9593 |            3 |
|   18 | accept |      2.9589 |    0.060048 |      2.9589 |      2.9592 |           10 |
|   19 | accept |      3.4577 |    0.056087 |      2.9589 |      2.9591 |           37 |
|   20 | accept |      3.2166 |     0.16203 |      2.9589 |       2.959 |           16 |
|======================================================================================|
| iter | eval   | objective:  | objective   | bestsofar   | bestsofar   |  minleafsize |
|      | result | log(1 loss) | runtime     | (observed)  | (estim.)    |              |
|======================================================================================|
|   21 | accept |       3.107 |     0.11253 |      2.9589 |      2.9591 |            5 |
|   22 | accept |      3.2818 |    0.061186 |      2.9589 |       2.959 |           24 |
|   23 | accept |      3.3226 |    0.057868 |      2.9589 |       2.959 |           32 |
|   24 | accept |      4.1881 |     0.05985 |      2.9589 |      2.9589 |           43 |
|   25 | accept |      3.1789 |    0.079785 |      2.9589 |      2.9589 |           18 |
|   26 | accept |      3.0992 |    0.071655 |      2.9589 |      2.9589 |           14 |
|   27 | accept |      3.0556 |    0.072465 |      2.9589 |      2.9589 |           22 |
|   28 | accept |      3.0459 |    0.076194 |      2.9589 |      2.9589 |           12 |
|   29 | accept |      3.2818 |    0.091907 |      2.9589 |      2.9589 |           26 |
|   30 | accept |      3.4361 |    0.086409 |      2.9589 |      2.9589 |           34 |

figure contains an axes object. the axes object with title min objective vs. number of function evaluations, xlabel function evaluations, ylabel min objective contains 2 objects of type line. these objects represent min observed objective, estimated min objective.

figure contains an axes object. the axes object with title objective function model, xlabel minleafsize, ylabel estimated objective function value contains 8 objects of type line. one or more of the lines displays its values using only markers these objects represent observed points, model mean, model error bars, noise error bars, next point, model minimum feasible.

__________________________________________________________
optimization completed.
maxobjectiveevaluations of 30 reached.
total function evaluations: 30
total elapsed time: 29.4765 seconds
total objective function evaluation time: 2.7911
best observed feasible point:
    minleafsize
    ___________
        10     
observed objective function value = 2.9589
estimated objective function value = 2.9589
function evaluation time = 0.063884
best estimated feasible point (according to models):
    minleafsize
    ___________
        10     
estimated objective function value = 2.9589
estimated function evaluation time = 0.086051

mdl = 
  regressiontree
                         responsename: 'y'
                categoricalpredictors: []
                    responsetransform: 'none'
                      numobservations: 94
    hyperparameteroptimizationresults: [1x1 bayesianoptimization]
  properties, methods

unbiased predictor importance estimates

load the carsmall data set. consider a model that predicts the mean fuel economy of a car given its acceleration, number of cylinders, engine displacement, horsepower, manufacturer, model year, and weight. consider cylinders, mfg, and model_year as categorical variables.

load carsmall
cylinders = categorical(cylinders);
mfg = categorical(cellstr(mfg));
model_year = categorical(model_year);
x = table(acceleration,cylinders,displacement,horsepower,mfg,...
    model_year,weight,mpg);

display the number of categories represented in the categorical variables.

numcylinders = numel(categories(cylinders))

numcylinders = 3

nummfg = numel(categories(mfg))

nummfg = 28

nummodelyear = numel(categories(model_year))

nummodelyear = 3

because there are 3 categories only in cylinders and model_year, the standard cart, predictor-splitting algorithm prefers splitting a continuous predictor over these two variables.

train a regression tree using the entire data set. to grow unbiased trees, specify usage of the curvature test for splitting predictors. because there are missing values in the data, specify usage of surrogate splits.

mdl = fitrtree(x,'mpg','predictorselection','curvature','surrogate','on');

estimate predictor importance values by summing changes in the risk due to splits on every predictor and dividing the sum by the number of branch nodes. compare the estimates using a bar graph.

imp = predictorimportance(mdl);
figure;
bar(imp);
title('predictor importance estimates');
ylabel('estimates');
xlabel('predictors');
h = gca;
h.xticklabel = mdl.predictornames;
h.xticklabelrotation = 45;
h.ticklabelinterpreter = 'none';

figure contains an axes object. the axes object with title predictor importance estimates, xlabel predictors, ylabel estimates contains an object of type bar.

in this case, displacement is the most important predictor, followed by horsepower.

control maximum tree depth on tall array

fitrtree grows deep decision trees by default. build a shallower tree that requires fewer passes through a tall array. use the 'maxdepth' name-value pair argument to control the maximum tree depth.

when you perform calculations on tall arrays, matlab® uses either a parallel pool (default if you have parallel computing toolbox™) or the local matlab session. if you want to run the example using the local matlab session when you have parallel computing toolbox, you can change the global execution environment by using the function.

load the carsmall data set. consider displacement, horsepower, and weight as predictors of the response mpg.

load carsmall
x = [displacement horsepower weight];

convert the in-memory arrays x and mpg to tall arrays.

tx = tall(x);

starting parallel pool (parpool) using the 'local' profile ...
connected to the parallel pool (number of workers: 6).

ty = tall(mpg);

grow a regression tree using all observations. allow the tree to grow to the maximum possible depth.

for reproducibility, set the seeds of the random number generators using rng and tallrng. the results can vary depending on the number of workers and the execution environment for the tall arrays. for details, see .

rng('default') 
tallrng('default')
mdl = fitrtree(tx,ty);

evaluating tall expression using the parallel pool 'local':
- pass 1 of 2: completed in 4.1 sec
- pass 2 of 2: completed in 0.71 sec
evaluation completed in 6.7 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 7: completed in 1.4 sec
- pass 2 of 7: completed in 0.29 sec
- pass 3 of 7: completed in 1.5 sec
- pass 4 of 7: completed in 3.3 sec
- pass 5 of 7: completed in 0.63 sec
- pass 6 of 7: completed in 1.2 sec
- pass 7 of 7: completed in 2.6 sec
evaluation completed in 12 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 7: completed in 0.36 sec
- pass 2 of 7: completed in 0.27 sec
- pass 3 of 7: completed in 0.85 sec
- pass 4 of 7: completed in 2 sec
- pass 5 of 7: completed in 0.55 sec
- pass 6 of 7: completed in 0.92 sec
- pass 7 of 7: completed in 1.6 sec
evaluation completed in 7.4 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 7: completed in 0.32 sec
- pass 2 of 7: completed in 0.29 sec
- pass 3 of 7: completed in 0.89 sec
- pass 4 of 7: completed in 1.9 sec
- pass 5 of 7: completed in 0.83 sec
- pass 6 of 7: completed in 1.2 sec
- pass 7 of 7: completed in 2.4 sec
evaluation completed in 9 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 7: completed in 0.33 sec
- pass 2 of 7: completed in 0.28 sec
- pass 3 of 7: completed in 0.89 sec
- pass 4 of 7: completed in 2.4 sec
- pass 5 of 7: completed in 0.76 sec
- pass 6 of 7: completed in 1 sec
- pass 7 of 7: completed in 1.7 sec
evaluation completed in 8.3 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 7: completed in 0.34 sec
- pass 2 of 7: completed in 0.26 sec
- pass 3 of 7: completed in 0.81 sec
- pass 4 of 7: completed in 1.7 sec
- pass 5 of 7: completed in 0.56 sec
- pass 6 of 7: completed in 1 sec
- pass 7 of 7: completed in 1.9 sec
evaluation completed in 7.4 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 7: completed in 0.35 sec
- pass 2 of 7: completed in 0.28 sec
- pass 3 of 7: completed in 0.81 sec
- pass 4 of 7: completed in 1.8 sec
- pass 5 of 7: completed in 0.76 sec
- pass 6 of 7: completed in 0.96 sec
- pass 7 of 7: completed in 2.2 sec
evaluation completed in 8 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 7: completed in 0.35 sec
- pass 2 of 7: completed in 0.32 sec
- pass 3 of 7: completed in 0.92 sec
- pass 4 of 7: completed in 1.9 sec
- pass 5 of 7: completed in 1 sec
- pass 6 of 7: completed in 1.5 sec
- pass 7 of 7: completed in 2.1 sec
evaluation completed in 9.2 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 7: completed in 0.33 sec
- pass 2 of 7: completed in 0.28 sec
- pass 3 of 7: completed in 0.82 sec
- pass 4 of 7: completed in 1.4 sec
- pass 5 of 7: completed in 0.61 sec
- pass 6 of 7: completed in 0.93 sec
- pass 7 of 7: completed in 1.5 sec
evaluation completed in 6.6 sec

view the trained tree mdl.

view(mdl,'mode','graph')

mdl is a tree of depth 8.

estimate the in-sample mean squared error.

mse_mdl = gather(loss(mdl,tx,ty))

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 1.6 sec
evaluation completed in 1.9 sec

mse_mdl = 4.9078

grow a regression tree using all observations. limit the tree depth by specifying a maximum tree depth of 4.

mdl2 = fitrtree(tx,ty,'maxdepth',4);

evaluating tall expression using the parallel pool 'local':
- pass 1 of 2: completed in 0.27 sec
- pass 2 of 2: completed in 0.28 sec
evaluation completed in 0.84 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 7: completed in 0.36 sec
- pass 2 of 7: completed in 0.3 sec
- pass 3 of 7: completed in 0.95 sec
- pass 4 of 7: completed in 1.6 sec
- pass 5 of 7: completed in 0.55 sec
- pass 6 of 7: completed in 0.93 sec
- pass 7 of 7: completed in 1.5 sec
evaluation completed in 7 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 7: completed in 0.34 sec
- pass 2 of 7: completed in 0.3 sec
- pass 3 of 7: completed in 0.95 sec
- pass 4 of 7: completed in 1.7 sec
- pass 5 of 7: completed in 0.57 sec
- pass 6 of 7: completed in 0.94 sec
- pass 7 of 7: completed in 1.8 sec
evaluation completed in 7.7 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 7: completed in 0.34 sec
- pass 2 of 7: completed in 0.3 sec
- pass 3 of 7: completed in 0.87 sec
- pass 4 of 7: completed in 1.5 sec
- pass 5 of 7: completed in 0.57 sec
- pass 6 of 7: completed in 0.81 sec
- pass 7 of 7: completed in 1.7 sec
evaluation completed in 6.9 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 7: completed in 0.32 sec
- pass 2 of 7: completed in 0.27 sec
- pass 3 of 7: completed in 0.85 sec
- pass 4 of 7: completed in 1.6 sec
- pass 5 of 7: completed in 0.63 sec
- pass 6 of 7: completed in 0.9 sec
- pass 7 of 7: completed in 1.6 sec
evaluation completed in 7 sec

view the trained tree mdl2.

view(mdl2,'mode','graph')

estimate the in-sample mean squared error.

mse_mdl2 = gather(loss(mdl2,tx,ty))

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.73 sec
evaluation completed in 1 sec

mse_mdl2 = 9.3903

mdl2 is a less complex tree with a depth of 4 and an in-sample mean squared error that is higher than the mean squared error of mdl.

optimize regression tree on tall array

optimize hyperparameters of a regression tree automatically using a tall array. the sample data set is the carsmall data set. this example converts the data set to a tall array and uses it to run the optimization procedure.

load the carsmall data set. consider displacement, horsepower, and weight as predictors of the response mpg.

load carsmall
x = [displacement horsepower weight];

convert the in-memory arrays x and mpg to tall arrays.

tx = tall(x);

starting parallel pool (parpool) using the 'local' profile ...
connected to the parallel pool (number of workers: 6).

ty = tall(mpg);

optimize hyperparameters automatically using the 'optimizehyperparameters' name-value pair argument. find the optimal 'minleafsize' value that minimizes holdout cross-validation loss. (specifying 'auto' uses 'minleafsize'.) for reproducibility, use the 'expected-improvement-plus' acquisition function and set the seeds of the random number generators using rng and tallrng. the results can vary depending on the number of workers and the execution environment for the tall arrays. for details, see .

rng('default') 
tallrng('default')
[mdl,fitinfo,hyperparameteroptimizationresults] = fitrtree(tx,ty,...
    'optimizehyperparameters','auto',...
    'hyperparameteroptimizationoptions',struct('holdout',0.3,...
    'acquisitionfunctionname','expected-improvement-plus'))

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 4.4 sec
evaluation completed in 6.2 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.97 sec
- pass 2 of 4: completed in 1.6 sec
- pass 3 of 4: completed in 3.6 sec
- pass 4 of 4: completed in 2.4 sec
evaluation completed in 9.8 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.55 sec
- pass 2 of 4: completed in 1.3 sec
- pass 3 of 4: completed in 2.7 sec
- pass 4 of 4: completed in 1.9 sec
evaluation completed in 7.3 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.52 sec
- pass 2 of 4: completed in 1.3 sec
- pass 3 of 4: completed in 3 sec
- pass 4 of 4: completed in 2 sec
evaluation completed in 8.1 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.55 sec
- pass 2 of 4: completed in 1.4 sec
- pass 3 of 4: completed in 2.6 sec
- pass 4 of 4: completed in 2 sec
evaluation completed in 7.3 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.61 sec
- pass 2 of 4: completed in 1.2 sec
- pass 3 of 4: completed in 2.1 sec
- pass 4 of 4: completed in 1.7 sec
evaluation completed in 6.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.53 sec
- pass 2 of 4: completed in 1.2 sec
- pass 3 of 4: completed in 2.4 sec
- pass 4 of 4: completed in 1.6 sec
evaluation completed in 6.6 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 1.4 sec
evaluation completed in 1.7 sec
|======================================================================================|
| iter | eval   | objective:  | objective   | bestsofar   | bestsofar   |  minleafsize |
|      | result | log(1 loss) | runtime     | (observed)  | (estim.)    |              |
|======================================================================================|
|    1 | best   |      3.2007 |      69.013 |      3.2007 |      3.2007 |            2 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.52 sec
evaluation completed in 0.83 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.65 sec
- pass 2 of 4: completed in 1.2 sec
- pass 3 of 4: completed in 3 sec
- pass 4 of 4: completed in 2 sec
evaluation completed in 8.3 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.79 sec
evaluation completed in 1 sec
|    2 | error  |         nan |      13.772 |         nan |      3.2007 |           46 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.52 sec
evaluation completed in 0.81 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.57 sec
- pass 2 of 4: completed in 1.3 sec
- pass 3 of 4: completed in 2.2 sec
- pass 4 of 4: completed in 1.7 sec
evaluation completed in 6.6 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.5 sec
- pass 2 of 4: completed in 1.2 sec
- pass 3 of 4: completed in 2.7 sec
- pass 4 of 4: completed in 1.7 sec
evaluation completed in 6.9 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.47 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 2.1 sec
- pass 4 of 4: completed in 1.9 sec
evaluation completed in 6.4 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.72 sec
evaluation completed in 0.99 sec
|    3 | best   |      3.1876 |      29.091 |      3.1876 |      3.1884 |           18 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.48 sec
evaluation completed in 0.76 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.5 sec
- pass 2 of 4: completed in 1.2 sec
- pass 3 of 4: completed in 1.9 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.8 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.48 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 2 sec
- pass 4 of 4: completed in 1.5 sec
evaluation completed in 5.8 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.54 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.9 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.7 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.46 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.64 sec
evaluation completed in 0.92 sec
|    4 | best   |      2.9048 |      33.465 |      2.9048 |      2.9537 |            6 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.44 sec
evaluation completed in 0.71 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.46 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 2 sec
- pass 4 of 4: completed in 1.5 sec
evaluation completed in 5.9 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.47 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.9 sec
- pass 4 of 4: completed in 1.5 sec
evaluation completed in 5.7 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.44 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.9 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.6 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.66 sec
evaluation completed in 0.92 sec
|    5 | accept |      3.2895 |      25.902 |      2.9048 |      2.9048 |           15 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.54 sec
evaluation completed in 0.82 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.53 sec
- pass 2 of 4: completed in 1.2 sec
- pass 3 of 4: completed in 2 sec
- pass 4 of 4: completed in 1.5 sec
evaluation completed in 6 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.5 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 2.1 sec
- pass 4 of 4: completed in 1.9 sec
evaluation completed in 6.4 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.49 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.9 sec
- pass 4 of 4: completed in 2 sec
evaluation completed in 6.6 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.45 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 2 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.8 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.68 sec
evaluation completed in 0.99 sec
|    6 | accept |      3.1641 |      35.522 |      2.9048 |      3.1493 |            5 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.51 sec
evaluation completed in 0.79 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.67 sec
- pass 2 of 4: completed in 1.3 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 6.2 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.45 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.9 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.7 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.48 sec
- pass 2 of 4: completed in 1.4 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.8 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.46 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.6 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.63 sec
evaluation completed in 0.89 sec
|    7 | accept |      2.9048 |      33.755 |      2.9048 |      2.9048 |            6 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.45 sec
evaluation completed in 0.75 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.51 sec
- pass 2 of 4: completed in 1.2 sec
- pass 3 of 4: completed in 2.2 sec
- pass 4 of 4: completed in 1.5 sec
evaluation completed in 6.1 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.49 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.9 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.6 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.46 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.6 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.45 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.4 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.68 sec
evaluation completed in 0.97 sec
|    8 | accept |      2.9522 |      33.362 |      2.9048 |      2.9048 |            7 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.42 sec
evaluation completed in 0.71 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.48 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.45 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.5 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.9 sec
- pass 4 of 4: completed in 1.5 sec
evaluation completed in 5.7 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.49 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.64 sec
evaluation completed in 0.9 sec
|    9 | accept |      2.9985 |      32.674 |      2.9048 |      2.9048 |            8 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.43 sec
evaluation completed in 0.7 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.47 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.56 sec
- pass 2 of 4: completed in 1.2 sec
- pass 3 of 4: completed in 2 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 6 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.45 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.5 sec
evaluation completed in 5.7 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.47 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.6 sec
evaluation completed in 5.8 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.88 sec
evaluation completed in 1.2 sec
|   10 | accept |      3.0185 |      33.922 |      2.9048 |      2.9048 |           10 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.44 sec
evaluation completed in 0.74 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.46 sec
- pass 2 of 4: completed in 1.2 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.6 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.48 sec
- pass 2 of 4: completed in 1.2 sec
- pass 3 of 4: completed in 2 sec
- pass 4 of 4: completed in 1.6 sec
evaluation completed in 6.2 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.73 sec
- pass 2 of 4: completed in 1.2 sec
- pass 3 of 4: completed in 2 sec
- pass 4 of 4: completed in 1.5 sec
evaluation completed in 6.2 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.63 sec
evaluation completed in 0.88 sec
|   11 | accept |      3.2895 |      26.625 |      2.9048 |      2.9048 |           14 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.48 sec
evaluation completed in 0.78 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.51 sec
- pass 2 of 4: completed in 1.2 sec
- pass 3 of 4: completed in 1.9 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.7 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.48 sec
- pass 2 of 4: completed in 1.2 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.65 sec
evaluation completed in 0.9 sec
|   12 | accept |      3.4798 |      18.111 |      2.9048 |      2.9049 |           31 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.44 sec
evaluation completed in 0.71 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.45 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.4 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.5 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.5 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.48 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.7 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.44 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.4 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.43 sec
- pass 2 of 4: completed in 1.2 sec
- pass 3 of 4: completed in 2 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.7 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.64 sec
evaluation completed in 0.91 sec
|   13 | accept |      3.2248 |      47.436 |      2.9048 |      2.9048 |            1 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.46 sec
evaluation completed in 0.74 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.6 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.45 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.6 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.57 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 2.6 sec
- pass 4 of 4: completed in 1.6 sec
evaluation completed in 6.6 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.62 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.7 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.5 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.6 sec
evaluation completed in 6.1 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.61 sec
evaluation completed in 0.88 sec
|   14 | accept |      3.1498 |      42.062 |      2.9048 |      2.9048 |            3 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.46 sec
evaluation completed in 0.76 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.48 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.5 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.67 sec
- pass 2 of 4: completed in 1.3 sec
- pass 3 of 4: completed in 2.3 sec
- pass 4 of 4: completed in 2.2 sec
evaluation completed in 7.4 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.45 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.4 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.6 sec
evaluation completed in 0.86 sec
|   15 | accept |      2.9048 |        34.3 |      2.9048 |      2.9048 |            6 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.48 sec
evaluation completed in 0.78 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.44 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.44 sec
- pass 2 of 4: completed in 1.2 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.6 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.43 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.4 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.44 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 2 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.7 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.62 sec
evaluation completed in 0.88 sec
|   16 | accept |      2.9048 |       32.97 |      2.9048 |      2.9048 |            6 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.43 sec
evaluation completed in 0.73 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.47 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.43 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.62 sec
evaluation completed in 0.9 sec
|   17 | accept |      3.1847 |       17.47 |      2.9048 |      2.9048 |           23 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.43 sec
evaluation completed in 0.72 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.44 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.7 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.68 sec
- pass 2 of 4: completed in 1.4 sec
- pass 3 of 4: completed in 1.9 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 6.3 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.45 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.4 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.44 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.4 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.62 sec
evaluation completed in 0.93 sec
|   18 | accept |      3.1817 |      33.346 |      2.9048 |      2.9048 |            4 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.43 sec
evaluation completed in 0.72 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.44 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.4 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.62 sec
evaluation completed in 0.86 sec
|   19 | error  |         nan |      10.235 |      2.9048 |      2.9048 |           38 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.47 sec
evaluation completed in 0.76 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.44 sec
- pass 2 of 4: completed in 1.2 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.44 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.44 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.9 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.43 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.63 sec
evaluation completed in 0.89 sec
|   20 | accept |      3.0628 |      32.459 |      2.9048 |      2.9048 |           12 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.46 sec
evaluation completed in 0.76 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.48 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.68 sec
- pass 2 of 4: completed in 1.7 sec
- pass 3 of 4: completed in 2.1 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 6.8 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.64 sec
evaluation completed in 0.9 sec
|======================================================================================|
| iter | eval   | objective:  | objective   | bestsofar   | bestsofar   |  minleafsize |
|      | result | log(1 loss) | runtime     | (observed)  | (estim.)    |              |
|======================================================================================|
|   21 | accept |      3.1847 |       19.02 |      2.9048 |      2.9048 |           27 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.45 sec
evaluation completed in 0.75 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.47 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.6 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.45 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.5 sec
- pass 2 of 4: completed in 1.6 sec
- pass 3 of 4: completed in 2.4 sec
- pass 4 of 4: completed in 1.5 sec
evaluation completed in 6.8 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.44 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.5 sec
evaluation completed in 5.6 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.63 sec
evaluation completed in 0.89 sec
|   22 | accept |      3.0185 |      33.933 |      2.9048 |      2.9048 |            9 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.46 sec
evaluation completed in 0.76 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.45 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.45 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.43 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.4 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.64 sec
evaluation completed in 0.89 sec
|   23 | accept |      3.0749 |      25.147 |      2.9048 |      2.9048 |           20 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.44 sec
evaluation completed in 0.73 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.42 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.43 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.4 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.53 sec
- pass 2 of 4: completed in 1.4 sec
- pass 3 of 4: completed in 1.9 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.9 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.44 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.62 sec
evaluation completed in 0.88 sec
|   24 | accept |      3.0628 |      32.764 |      2.9048 |      2.9048 |           11 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.44 sec
evaluation completed in 0.73 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.44 sec
- pass 2 of 4: completed in 1.2 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.61 sec
evaluation completed in 0.87 sec
|   25 | error  |         nan |      10.294 |      2.9048 |      2.9048 |           34 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.44 sec
evaluation completed in 0.73 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.45 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.4 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.43 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.4 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.62 sec
evaluation completed in 0.87 sec
|   26 | accept |      3.1847 |      17.587 |      2.9048 |      2.9048 |           25 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.45 sec
evaluation completed in 0.73 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.45 sec
- pass 2 of 4: completed in 1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.4 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.44 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.43 sec
- pass 2 of 4: completed in 1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.3 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.66 sec
evaluation completed in 0.96 sec
|   27 | accept |      3.2895 |      24.867 |      2.9048 |      2.9048 |           16 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.44 sec
evaluation completed in 0.74 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.45 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.4 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.43 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.44 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.4 sec
evaluation completed in 5.4 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.6 sec
evaluation completed in 0.88 sec
|   28 | accept |      3.2135 |      24.928 |      2.9048 |      2.9048 |           13 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.47 sec
evaluation completed in 0.76 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.45 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.4 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.46 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.62 sec
evaluation completed in 0.87 sec
|   29 | accept |      3.1847 |      17.582 |      2.9048 |      2.9048 |           21 |

evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.53 sec
evaluation completed in 0.81 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.44 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.4 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 4: completed in 0.43 sec
- pass 2 of 4: completed in 1.1 sec
- pass 3 of 4: completed in 1.8 sec
- pass 4 of 4: completed in 1.3 sec
evaluation completed in 5.4 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 1: completed in 0.63 sec
evaluation completed in 0.88 sec
|   30 | accept |      3.1827 |      17.597 |      2.9048 |      2.9122 |           29 |

__________________________________________________________
optimization completed.
maxobjectiveevaluations of 30 reached.
total function evaluations: 30
total elapsed time: 882.5668 seconds.
total objective function evaluation time: 859.2122
best observed feasible point:
    minleafsize
    ___________
         6     
observed objective function value = 2.9048
estimated objective function value = 2.9122
function evaluation time = 33.4655
best estimated feasible point (according to models):
    minleafsize
    ___________
         6     
estimated objective function value = 2.9122
estimated function evaluation time = 33.6594
evaluating tall expression using the parallel pool 'local':
- pass 1 of 2: completed in 0.26 sec
- pass 2 of 2: completed in 0.26 sec
evaluation completed in 0.84 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 7: completed in 0.31 sec
- pass 2 of 7: completed in 0.25 sec
- pass 3 of 7: completed in 0.75 sec
- pass 4 of 7: completed in 1.2 sec
- pass 5 of 7: completed in 0.45 sec
- pass 6 of 7: completed in 0.69 sec
- pass 7 of 7: completed in 1.2 sec
evaluation completed in 5.7 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 7: completed in 0.28 sec
- pass 2 of 7: completed in 0.24 sec
- pass 3 of 7: completed in 0.75 sec
- pass 4 of 7: completed in 1.2 sec
- pass 5 of 7: completed in 0.46 sec
- pass 6 of 7: completed in 0.67 sec
- pass 7 of 7: completed in 1.2 sec
evaluation completed in 5.6 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 7: completed in 0.32 sec
- pass 2 of 7: completed in 0.25 sec
- pass 3 of 7: completed in 0.71 sec
- pass 4 of 7: completed in 1.2 sec
- pass 5 of 7: completed in 0.47 sec
- pass 6 of 7: completed in 0.66 sec
- pass 7 of 7: completed in 1.2 sec
evaluation completed in 5.6 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 7: completed in 0.29 sec
- pass 2 of 7: completed in 0.25 sec
- pass 3 of 7: completed in 0.73 sec
- pass 4 of 7: completed in 1.2 sec
- pass 5 of 7: completed in 0.46 sec
- pass 6 of 7: completed in 0.68 sec
- pass 7 of 7: completed in 1.2 sec
evaluation completed in 5.5 sec
evaluating tall expression using the parallel pool 'local':
- pass 1 of 7: completed in 0.27 sec
- pass 2 of 7: completed in 0.25 sec
- pass 3 of 7: completed in 0.75 sec
- pass 4 of 7: completed in 1.2 sec
- pass 5 of 7: completed in 0.47 sec
- pass 6 of 7: completed in 0.69 sec
- pass 7 of 7: completed in 1.2 sec
evaluation completed in 5.6 sec

mdl = 
  compactregressiontree
             responsename: 'y'
    categoricalpredictors: []
        responsetransform: 'none'
  properties, methods

fitinfo = struct with no fields.

hyperparameteroptimizationresults = 
  bayesianoptimization with properties:
                      objectivefcn: @createobjfcn/tallobjfcn
              variabledescriptions: [3×1 optimizablevariable]
                           options: [1×1 struct]
                      minobjective: 2.9048
                   xatminobjective: [1×1 table]
             minestimatedobjective: 2.9122
          xatminestimatedobjective: [1×1 table]
           numobjectiveevaluations: 30
                  totalelapsedtime: 882.5668
                         nextpoint: [1×1 table]
                            xtrace: [30×1 table]
                    objectivetrace: [30×1 double]
                  constraintstrace: []
                     userdatatrace: {30×1 cell}
      objectiveevaluationtimetrace: [30×1 double]
                iterationtimetrace: [30×1 double]
                        errortrace: [30×1 double]
                  feasibilitytrace: [30×1 logical]
       feasibilityprobabilitytrace: [30×1 double]
               indexofminimumtrace: [30×1 double]
             objectiveminimumtrace: [30×1 double]
    estimatedobjectiveminimumtrace: [30×1 double]

input arguments

`tbl` — sample data
table

sample data used to train the model, specified as a table. each row of tbl corresponds to one observation, and each column corresponds to one predictor variable. optionally, tbl can contain one additional column for the response variable. multicolumn variables and cell arrays other than cell arrays of character vectors are not allowed.

if tbl contains the response variable, and you want to use all remaining variables in tbl as predictors, then specify the response variable by using responsevarname.
if tbl contains the response variable, and you want to use only a subset of the remaining variables in tbl as predictors, then specify a formula by using formula.
if tbl does not contain the response variable, then specify a response variable by using y. the length of the response variable and the number of rows in tbl must be equal.

`responsevarname` — response variable name
name of variable in `tbl`

response variable name, specified as the name of a variable in tbl. the response variable must be a numeric vector.

you must specify responsevarname as a character vector or string scalar. for example, if tbl stores the response variable y as tbl.y, then specify it as 'y'. otherwise, the software treats all columns of tbl, including y, as predictors when training the model.

data types: char | string

`formula` — explanatory model of response variable and subset of predictor variables
character vector | string scalar

explanatory model of the response variable and a subset of the predictor variables, specified as a character vector or string scalar in the form "y~x1 x2 x3". in this form, y represents the response variable, and x1, x2, and x3 represent the predictor variables.

to specify a subset of variables in tbl as predictors for training the model, use a formula. if you specify a formula, then the software does not use any variables in tbl that do not appear in formula.

the variable names in the formula must be both variable names in tbl (tbl.properties.variablenames) and valid matlab^® identifiers. you can verify the variable names in tbl by using the isvarname function. if the variable names are not valid, then you can convert them by using the matlab.lang.makevalidname function.

data types: char | string

`y` — response data
numeric column vector

response data, specified as a numeric column vector with the same number of rows as x. each entry in y is the response to the data in the corresponding row of x.

the software considers nan values in y to be missing values. fitrtree does not use observations with missing values for y in the fit.

data types: single | double

`x` — predictor data
numeric matrix

predictor data, specified as a numeric matrix. each column of x represents one variable, and each row represents one observation.

fitrtree considers nan values in x as missing values. fitrtree does not use observations with all missing values for x in the fit. fitrtree uses observations with some missing values for x to find splits on variables for which these observations have valid values.

data types: single | double

name-value arguments

specify optional pairs of arguments as name1=value1,...,namen=valuen, where name is the argument name and value is the corresponding value. name-value arguments must appear after other arguments, but the order of the pairs does not matter.

before r2021a, use commas to separate each name and value, and enclose name in quotes.

example: 'crossval','on','minparentsize',30 specifies a cross-validated regression tree with a minimum of 30 observations per branch node.

note

you cannot use any cross-validation name-value argument together with the 'optimizehyperparameters' name-value argument. you can modify the cross-validation for 'optimizehyperparameters' only by using the 'hyperparameteroptimizationoptions' name-value argument.

model parameters

`categoricalpredictors` — categorical predictors list
vector of positive integers | logical vector | character matrix | string array | cell array of character vectors | `'all'`

categorical predictors list, specified as one of the values in this table.

value	description
vector of positive integers	each entry in the vector is an index value indicating that the corresponding predictor is categorical. the index values are between 1 and `p`, where `p` is the number of predictors used to train the model. if `fitrtree` uses a subset of input variables as predictors, then the function indexes the predictors using only the subset. the `categoricalpredictors` values do not count the response variable, observation weights variable, or any other variables that the function does not use.
logical vector	a `true` entry means that the corresponding predictor is categorical. the length of the vector is `p`.
character matrix	each row of the matrix is the name of a predictor variable. the names must match the entries in `predictornames`. pad the names with extra blanks so each row of the character matrix has the same length.
string array or cell array of character vectors	each element in the array is the name of a predictor variable. the names must match the entries in `predictornames`.
`"all"`	all predictors are categorical.

by default, if the predictor data is a table (tbl), fitrtree assumes that a variable is categorical if it is a logical vector, unordered categorical vector, character array, string array, or cell array of character vectors. if the predictor data is a matrix (x), fitrtree assumes that all predictors are continuous. to identify any other predictors as categorical predictors, specify them by using the categoricalpredictors name-value argument.

example: 'categoricalpredictors','all'

`maxdepth` — maximum tree depth
positive integer

maximum tree depth, specified as the comma-separated pair consisting of 'maxdepth' and a positive integer. specify a value for this argument to return a tree that has fewer levels and requires fewer passes through the tall array to compute. generally, the algorithm of fitrtree takes one pass through the data and an additional pass for each tree level. the function does not set a maximum tree depth, by default.

note

this option applies only when you use fitrtree on tall arrays. see tall arrays for more information.

`mergeleaves` — leaf merge flag
`'on'` (default) | `'off'`

leaf merge flag, specified as the comma-separated pair consisting of 'mergeleaves' and 'on' or 'off'.

if mergeleaves is 'on', then fitrtree:

merges leaves that originate from the same parent node and yield a sum of risk values greater than or equal to the risk associated with the parent node
estimates the optimal sequence of pruned subtrees, but does not prune the regression tree

otherwise, fitrtree does not merge leaves.

example: 'mergeleaves','off'

`minparentsize` — minimum number of branch node observations
`10` (default) | positive integer value

minimum number of branch node observations, specified as the comma-separated pair consisting of 'minparentsize' and a positive integer value. each branch node in the tree has at least minparentsize observations. if you supply both minparentsize and minleafsize, fitrtree uses the setting that gives larger leaves: minparentsize = max(minparentsize,2*minleafsize).

example: 'minparentsize',8

data types: single | double

`numbins` — number of bins for numeric predictors
`[]`(empty) (default) | positive integer scalar

number of bins for numeric predictors, specified as the comma-separated pair consisting of 'numbins' and a positive integer scalar.

if the 'numbins' value is empty (default), then fitrtree does not bin any predictors.
if you specify the 'numbins' value as a positive integer scalar (numbins), then fitrtree bins every numeric predictor into at most numbins equiprobable bins, and then grows trees on the bin indices instead of the original data.
- the number of bins can be less than numbins if a predictor has fewer than numbins unique values.
- fitrtree does not bin categorical predictors.

when you use a large training data set, this binning option speeds up training but might cause a potential decrease in accuracy. you can try 'numbins',50 first, and then change the value depending on the accuracy and training speed.

a trained model stores the bin edges in the binedges property.

example: 'numbins',50

data types: single | double

`predictornames` — predictor variable names
string array of unique names | cell array of unique character vectors

predictor variable names, specified as a string array of unique names or cell array of unique character vectors. the functionality of predictornames depends on the way you supply the training data.

if you supply x and y, then you can use predictornames to assign names to the predictor variables in x.
- the order of the names in predictornames must correspond to the column order of x. that is, predictornames{1} is the name of x(:,1), predictornames{2} is the name of x(:,2), and so on. also, size(x,2) and numel(predictornames) must be equal.
- by default, predictornames is {'x1','x2',...}.
if you supply tbl, then you can use predictornames to choose which predictor variables to use in training. that is, fitrtree uses only the predictor variables in predictornames and the response variable during training.
- predictornames must be a subset of tbl.properties.variablenames and cannot include the name of the response variable.
- by default, predictornames contains the names of all predictor variables.
- a good practice is to specify the predictors for training using either predictornames or formula, but not both.

example: "predictornames",["sepallength","sepalwidth","petallength","petalwidth"]

data types: string | cell

`predictorselection` — algorithm used to select the best split predictor
`'allsplits'` (default) | `'curvature'` | `'interaction-curvature'`

algorithm used to select the best split predictor at each node, specified as the comma-separated pair consisting of 'predictorselection' and a value in this table.

value	description
`'allsplits'`	standard cart — selects the split predictor that maximizes the split-criterion gain over all possible splits of all predictors [1].
`'curvature'`	curvature test — selects the split predictor that minimizes the p-value of chi-square tests of independence between each predictor and the response [2]. training speed is similar to standard cart.
`'interaction-curvature'`	interaction test — chooses the split predictor that minimizes the p-value of chi-square tests of independence between each predictor and the response (that is, conducts curvature tests), and that minimizes the p-value of a chi-square test of independence between each pair of predictors and response [2]. training speed can be slower than standard cart.

for 'curvature' and 'interaction-curvature', if all tests yield p-values greater than 0.05, then fitrtree stops splitting nodes.

tip

standard cart tends to select split predictors containing many distinct values, e.g., continuous variables, over those containing few distinct values, e.g., categorical variables [3]. consider specifying the curvature or interaction test if any of the following are true:
- if there are predictors that have relatively fewer distinct values than other predictors, for example, if the predictor data set is heterogeneous.
- if an analysis of predictor importance is your goal. for more on predictor importance estimation, see and introduction to feature selection.
trees grown using standard cart are not sensitive to predictor variable interactions. also, such trees are less likely to identify important variables in the presence of many irrelevant predictors than the application of the interaction test. therefore, to account for predictor interactions and identify importance variables in the presence of many irrelevant variables, specify the interaction test .
prediction speed is unaffected by the value of 'predictorselection'.

for details on how fitrtree selects split predictors, see node splitting rules and .

example: 'predictorselection','curvature'

`prune` — flag to estimate optimal sequence of pruned subtrees
`'on'` (default) | `'off'`

flag to estimate the optimal sequence of pruned subtrees, specified as the comma-separated pair consisting of 'prune' and 'on' or 'off'.

if prune is 'on', then fitrtree grows the regression tree and estimates the optimal sequence of pruned subtrees, but does not prune the regression tree. otherwise, fitrtree grows the regression tree without estimating the optimal sequence of pruned subtrees.

to prune a trained regression tree, pass the regression tree to .

example: 'prune','off'

`prunecriterion` — pruning criterion
`'mse'` (default)

pruning criterion, specified as the comma-separated pair consisting of 'prunecriterion' and 'mse'.

`quadraticerrortolerance` — quadratic error tolerance
`1e-6` (default) | positive scalar value

quadratic error tolerance per node, specified as the comma-separated pair consisting of 'quadraticerrortolerance' and a positive scalar value. the function stops splitting nodes when the weighted mean squared error per node drops below quadraticerrortolerance*ε, where ε is the weighted mean squared error of all n responses computed before growing the decision tree.

$ε = \sum_{i = 1}^{n} w_{i} {(y_{i} - \bar{y})}^{2} .$

w_i is the weight of observation i, given that the weights of all the observations sum to one ( $\sum_{i = 1}^{n} w_{i} = 1$ ), and

$\bar{y} = \sum_{i = 1}^{n} w_{i} y_{i}$

is the weighted average of all the responses.

for more details on node splitting, see node splitting rules.

example: 'quadraticerrortolerance',1e-4

`reproducible` — flag to enforce reproducibility
`false` (logical `0`) (default) | `true` (logical `1`)

flag to enforce reproducibility over repeated runs of training a model, specified as the comma-separated pair consisting of 'reproducible' and either false or true.

if 'numvariablestosample' is not 'all', then the software selects predictors at random for each split. to reproduce the random selections, you must specify 'reproducible',true and set the seed of the random number generator by using . note that setting 'reproducible' to true can slow down training.

example: 'reproducible',true

data types: logical

`responsename` — response variable name
`"y"` (default) | character vector | string scalar

response variable name, specified as a character vector or string scalar.

if you supply y, then you can use responsename to specify a name for the response variable.
if you supply responsevarname or formula, then you cannot use responsename.

example: "responsename","response"

data types: char | string

`responsetransform` — response transformation
`'none'` (default) | function handle

response transformation, specified as either 'none' or a function handle. the default is 'none', which means @(y)y, or no transformation. for a matlab function or a function you define, use its function handle for the response transformation. the function handle must accept a vector (the original response values) and return a vector of the same size (the transformed response values).

example: suppose you create a function handle that applies an exponential transformation to an input vector by using myfunction = @(y)exp(y). then, you can specify the response transformation as 'responsetransform',myfunction.

data types: char | string | function_handle

`splitcriterion` — split criterion
`'mse'` (default)

split criterion, specified as the comma-separated pair consisting of 'splitcriterion' and 'mse', meaning mean squared error.

example: 'splitcriterion','mse'

`surrogate` — surrogate decision splits flag
`'off'` (default) | `'on'` | `'all'` | positive integer

surrogate decision splits flag, specified as the comma-separated pair consisting of 'surrogate' and 'on', 'off', 'all', or a positive integer.

when 'on', fitrtree finds at most 10 surrogate splits at each branch node.
when set to a positive integer, fitrtree finds at most the specified number of surrogate splits at each branch node.
when set to 'all', fitrtree finds all surrogate splits at each branch node. the 'all' setting can use much time and memory.

use surrogate splits to improve the accuracy of predictions for data with missing values. the setting also enables you to compute measures of predictive association between predictors.

example: 'surrogate','on'

data types: single | double | char | string

`weights` — observation weights
`ones(size(x,1),1)` (default) | vector of scalar values | name of variable in `tbl`

observation weights, specified as the comma-separated pair consisting of 'weights' and a vector of scalar values or the name of a variable in tbl. the software weights the observations in each row of x or tbl with the corresponding value in weights. the size of weights must equal the number of rows in x or tbl.

if you specify the input data as a table tbl, then weights can be the name of a variable in tbl that contains a numeric vector. in this case, you must specify weights as a character vector or string scalar. for example, if weights vector w is stored as tbl.w, then specify it as 'w'. otherwise, the software treats all columns of tbl, including w, as predictors when training the model.

fitrtree normalizes the values of weights to sum to 1.

data types: single | double | char | string

cross-validation

`crossval` — cross-validation flag
`'off'` (default) | `'on'`

cross-validation flag, specified as the comma-separated pair consisting of 'crossval' and either 'on' or 'off'.

if 'on', fitrtree grows a cross-validated decision tree with 10 folds. you can override this cross-validation setting using one of the 'kfold', 'holdout', 'leaveout', or 'cvpartition' name-value pair arguments. you can only use one of these four options ('kfold', 'holdout', 'leaveout', or 'cvpartition') at a time when creating a cross-validated tree.

alternatively, cross-validate tree later using the method.

example: 'crossval','on'

`cvpartition` — partition for cross-validation tree
`cvpartition` object

partition for cross-validated tree, specified as the comma-separated pair consisting of 'cvpartition' and an object created using cvpartition.

if you use 'cvpartition', you cannot use any of the 'kfold', 'holdout', or 'leaveout' name-value pair arguments.

`holdout` — fraction of data for holdout validation
`0` (default) | scalar value in the range `[0,1]`

fraction of data used for holdout validation, specified as the comma-separated pair consisting of 'holdout' and a scalar value in the range [0,1]. holdout validation tests the specified fraction of the data, and uses the rest of the data for training.

if you use 'holdout', you cannot use any of the 'cvpartition', 'kfold', or 'leaveout' name-value pair arguments.

example: 'holdout',0.1

data types: single | double

`kfold` — number of folds
`10` (default) | positive integer greater than 1

number of folds to use in a cross-validated tree, specified as the comma-separated pair consisting of 'kfold' and a positive integer value greater than 1.

if you use 'kfold', you cannot use any of the 'cvpartition', 'holdout', or 'leaveout' name-value pair arguments.

example: 'kfold',8

data types: single | double

`leaveout` — leave-one-out cross-validation flag
`'off'` (default) | `'on'`

leave-one-out cross-validation flag, specified as the comma-separated pair consisting of 'leaveout' and either 'on' or 'off. use leave-one-out cross-validation by setting to 'on'.

if you use 'leaveout', you cannot use any of the 'cvpartition', 'holdout', or 'kfold' name-value pair arguments.

example: 'leaveout','on'

hyperparameters

`maxnumsplits` — maximal number of decision splits
`size(x,1) - 1` (default) | positive integer

maximal number of decision splits (or branch nodes), specified as the comma-separated pair consisting of 'maxnumsplits' and a positive integer. fitrtree splits maxnumsplits or fewer branch nodes. for more details on splitting behavior, see tree depth control.

example: 'maxnumsplits',5

data types: single | double

`minleafsize` — minimum number of leaf node observations
`1` (default) | positive integer value

minimum number of leaf node observations, specified as the comma-separated pair consisting of 'minleafsize' and a positive integer value. each leaf has at least minleafsize observations per tree leaf. if you supply both minparentsize and minleafsize, fitrtree uses the setting that gives larger leaves: minparentsize = max(minparentsize,2*minleafsize).

example: 'minleafsize',3

data types: single | double

`numvariablestosample` — number of predictors to select at random for each split
`'all'` (default) | positive integer value

number of predictors to select at random for each split, specified as the comma-separated pair consisting of 'numvariablestosample' and a positive integer value. alternatively, you can specify 'all' to use all available predictors.

if the training data includes many predictors and you want to analyze predictor importance, then specify 'numvariablestosample' as 'all'. otherwise, the software might not select some predictors, underestimating their importance.

to reproduce the random selections, you must set the seed of the random number generator by using and specify 'reproducible',true.

example: 'numvariablestosample',3

data types: char | string | single | double

hyperparameter optimization

`optimizehyperparameters` — parameters to optimize
`'none'` (default) | `'auto'` | `'all'` | string array or cell array of eligible parameter names | vector of `optimizablevariable` objects

parameters to optimize, specified as the comma-separated pair consisting of 'optimizehyperparameters' and one of the following:

'none' — do not optimize.
'auto' — use {'minleafsize'}.
'all' — optimize all eligible parameters.
string array or cell array of eligible parameter names.
vector of optimizablevariable objects, typically the output of .

the optimization attempts to minimize the cross-validation loss (error) for fitrtree by varying the parameters. to control the cross-validation type and other aspects of the optimization, use the hyperparameteroptimizationoptions name-value pair.

note

the values of 'optimizehyperparameters' override any values you specify using other name-value arguments. for example, setting 'optimizehyperparameters' to 'auto' causes fitrtree to optimize hyperparameters corresponding to the 'auto' option and to ignore any specified values for the hyperparameters.

the eligible parameters for fitrtree are:

maxnumsplits — fitrtree searches among integers, by default log-scaled in the range [1,max(2,numobservations-1)].
minleafsize — fitrtree searches among integers, by default log-scaled in the range [1,max(2,floor(numobservations/2))].
numvariablestosample — fitrtree does not optimize over this hyperparameter. if you pass numvariablestosample as a parameter name, fitrtree simply uses the full number of predictors. however, fitrensemble does optimize over this hyperparameter.

set nondefault parameters by passing a vector of optimizablevariable objects that have nondefault values. for example,

load carsmall
params = hyperparameters('fitrtree',[horsepower,weight],mpg);
params(1).range = [1,30];

pass params as the value of optimizehyperparameters.

by default, the iterative display appears at the command line, and plots appear according to the number of hyperparameters in the optimization. for the optimization and plots, the objective function is log(1 cross-validation loss). to control the iterative display, set the verbose field of the 'hyperparameteroptimizationoptions' name-value argument. to control the plots, set the showplots field of the 'hyperparameteroptimizationoptions' name-value argument.

for an example, see optimize regression tree.

example: 'auto'

`hyperparameteroptimizationoptions` — options for optimization
structure

options for optimization, specified as a structure. this argument modifies the effect of the optimizehyperparameters name-value argument. all fields in the structure are optional.

field name	values	default
`optimizer`	`'bayesopt'` — use bayesian optimization. internally, this setting calls . `'gridsearch'` — use grid search with `numgriddivisions` values per dimension. `'randomsearch'` — search at random among `maxobjectiveevaluations` points. `'gridsearch'` searches in a random order, using uniform sampling without replacement from the grid. after optimization, you can get a table in grid order by using the command `sortrows(mdl.hyperparameteroptimizationresults)`.	`'bayesopt'`
`acquisitionfunctionname`	`'expected-improvement-per-second-plus'` `'expected-improvement'` `'expected-improvement-plus'` `'expected-improvement-per-second'` `'lower-confidence-bound'` `'probability-of-improvement'` acquisition functions whose names include `per-second` do not yield reproducible results because the optimization depends on the runtime of the objective function. acquisition functions whose names include `plus` modify their behavior when they are overexploiting an area. for more details, see acquisition function types.	`'expected-improvement-per-second-plus'`
`maxobjectiveevaluations`	maximum number of objective function evaluations.	`30` for `'bayesopt'` and `'randomsearch'`, and the entire grid for `'gridsearch'`
`maxtime`	time limit, specified as a positive real scalar. the time limit is in seconds, as measured by `tic` and `toc`. the run time can exceed `maxtime` because `maxtime` does not interrupt function evaluations.	`inf`
`numgriddivisions`	for `'gridsearch'`, the number of values in each dimension. the value can be a vector of positive integers giving the number of values for each dimension, or a scalar that applies to all dimensions. this field is ignored for categorical variables.	`10`
`showplots`	logical value indicating whether to show plots. if `true`, this field plots the best observed objective function value against the iteration number. if you use bayesian optimization (`optimizer` is `'bayesopt'`), then this field also plots the best estimated objective function value. the best observed objective function values and best estimated objective function values correspond to the values in the `bestsofar (observed)` and `bestsofar (estim.)` columns of the iterative display, respectively. you can find these values in the properties `objectiveminimumtrace` and `estimatedobjectiveminimumtrace` of `mdl.hyperparameteroptimizationresults`. if the problem includes one or two optimization parameters for bayesian optimization, then `showplots` also plots a model of the objective function against the parameters.	`true`
`saveintermediateresults`	logical value indicating whether to save results when `optimizer` is `'bayesopt'`. if `true`, this field overwrites a workspace variable named `'bayesoptresults'` at each iteration. the variable is a `bayesianoptimization` object.	`false`
`verbose`	display at the command line: `0` — no iterative display `1` — iterative display `2` — iterative display with extra information for details, see the `bayesopt` name-value argument and the example .	`1`
`useparallel`	logical value indicating whether to run bayesian optimization in parallel, which requires parallel computing toolbox™. due to the nonreproducibility of parallel timing, parallel bayesian optimization does not necessarily yield reproducible results. for details, see .	`false`
`repartition`	logical value indicating whether to repartition the cross-validation at every iteration. if this field is `false`, the optimizer uses a single partition for the optimization. the setting `true` usually gives the most robust results because it takes partitioning noise into account. however, for good results, `true` requires at least twice as many function evaluations.	`false`
use no more than one of the following three options.
`cvpartition`	a `cvpartition` object, as created by `cvpartition`	`'kfold',5` if you do not specify a cross-validation field
`holdout`	a scalar in the range `(0,1)` representing the holdout fraction
`kfold`	an integer greater than 1

example: 'hyperparameteroptimizationoptions',struct('maxobjectiveevaluations',60)

data types: struct

output arguments

`tree` — regression tree
regression tree object

regression tree, returned as a regression tree object. using the 'crossval', 'kfold', 'holdout', 'leaveout', or 'cvpartition' options results in a tree of class . you cannot use a partitioned tree for prediction, so this kind of tree does not have a predict method.

otherwise, tree is of class regressiontree, and you can use the method to make predictions.

more about

curvature test

the curvature test is a statistical test assessing the null hypothesis that two variables are unassociated.

the curvature test between predictor variable x and y is conducted using this process.

if x is continuous, then partition it into its quartiles. create a nominal variable that bins observations according to which section of the partition they occupy. if there are missing values, then create an extra bin for them.
for each level in the partitioned predictor j = 1...j and class in the response k = 1,...,k, compute the weighted proportion of observations in class k

${\hat{π}}_{j k} = \sum_{i = 1}^{n} i {y_{i} = k} w_{i} .$
w_i is the weight of observation i, $\sum w_{i} = 1$ , i is the indicator function, and n is the sample size. if all observations have the same weight, then ${\hat{π}}_{j k} = \frac{n_{j k}}{n}$ , where n_jk is the number of observations in level j of the predictor that are in class k.
compute the test statistic

$t = n \sum_{k = 1}^{k} \sum_{j = 1}^{j} \frac{{({\hat{π}}_{j k} - {\hat{π}}_{j} {\hat{π}}_{k})}^{2}}{{\hat{π}}_{j} {\hat{π}}_{k}}$
${\hat{π}}_{j} = \sum_{k} {\hat{π}}_{j k}$ , that is, the marginal probability of observing the predictor at level j. ${\hat{π}}_{k} = \sum_{j} {\hat{π}}_{j k}$ , that is the marginal probability of observing class k. if n is large enough, then t is distributed as a χ² with (k – 1)(j – 1) degrees of freedom.
if the p-value for the test is less than 0.05, then reject the null hypothesis that there is no association between x and y.

when determining the best split predictor at each node, the standard cart algorithm prefers to select continuous predictors that have many levels. sometimes, such a selection can be spurious and can also mask more important predictors that have fewer levels, such as categorical predictors.

the curvature test can be applied instead of standard cart to determine the best split predictor at each node. in that case, the best split predictor variable is the one that minimizes the significant p-values (those less than 0.05) of curvature tests between each predictor and the response variable. such a selection is robust to the number of levels in individual predictors.

for more details on how the curvature test applies to growing regression trees, see node splitting rules and [3].

interaction test

the interaction test is a statistical test that assesses the null hypothesis that there is no interaction between a pair of predictor variables and the response variable.

the interaction test assessing the association between predictor variables x₁ and x₂ with respect to y is conducted using this process.

if x₁ or x₂ is continuous, then partition that variable into its quartiles. create a nominal variable that bins observations according to which section of the partition they occupy. if there are missing values, then create an extra bin for them.
create the nominal variable z with j = j₁j₂ levels that assigns an index to observation i according to which levels of x₁ and x₂ it belongs. remove any levels of z that do not correspond to any observations.
conduct a curvature test between z and y.

when growing decision trees, if there are important interactions between pairs of predictors, but there are also many other less important predictors in the data, then standard cart tends to miss the important interactions. however, conducting curvature and interaction tests for predictor selection instead can improve detection of important interactions, which can yield more accurate decision trees.

for more details on how the interaction test applies to growing decision trees, see curvature test, node splitting rules and [2].

predictive measure of association

the predictive measure of association is a value that indicates the similarity between decision rules that split observations. among all possible decision splits that are compared to the optimal split (found by growing the tree), the best surrogate decision split yields the maximum predictive measure of association. the second-best surrogate split has the second-largest predictive measure of association.

suppose x_j and x_k are predictor variables j and k, respectively, and j ≠ k. at node t, the predictive measure of association between the optimal split x_j < u and a surrogate split x_k < v is

$λ_{j k} = \frac{min (p_{l}, p_{r}) - (1 - p_{l_{j} l_{k}} - p_{r_{j} r_{k}})}{min (p_{l}, p_{r})} .$

p_l is the proportion of observations in node t, such that x_j < u. the subscript l stands for the left child of node t.
p_r is the proportion of observations in node t, such that x_j ≥ u. the subscript r stands for the right child of node t.
$p_{l_{j} l_{k}}$ is the proportion of observations at node t, such that x_j < u and x_k < v.
$p_{r_{j} r_{k}}$ is the proportion of observations at node t, such that x_j ≥ u and x_k ≥ v.
observations with missing values for x_j or x_k do not contribute to the proportion calculations.

λ_jk is a value in (–∞,1]. if λ_jk > 0, then x_k < v is a worthwhile surrogate split for x_j < u.

surrogate decision splits

a surrogate decision split is an alternative to the optimal decision split at a given node in a decision tree. the optimal split is found by growing the tree; the surrogate split uses a similar or correlated predictor variable and split criterion.

when the value of the optimal split predictor for an observation is missing, the observation is sent to the left or right child node using the best surrogate predictor. when the value of the best surrogate split predictor for the observation is also missing, the observation is sent to the left or right child node using the second-best surrogate predictor, and so on. candidate splits are sorted in descending order by their predictive measure of association.

tips

by default, prune is 'on'. however, this specification does not prune the regression tree. to prune a trained regression tree, pass the regression tree to .
after training a model, you can generate c/c code that predicts responses for new data. generating c/c code requires matlab coder™. for details, see introduction to code generation.

algorithms

node splitting rules

fitrtree uses these processes to determine how to split node t.

for standard cart (that is, if predictorselection is 'allpairs') and for all predictors x_i, i = 1,...,p:
1. fitrtree computes the weighted mean squared error (mse) of the responses in node t using
  
  $ε_{t} = \sum_{j \in t} w_{j} {(y_{j} - {\bar{y}}_{t})}^{2} .$
  w_j is the weight of observation j, and t is the set of all observation indices in node t. if you do not specify weights, then w_j = 1/n, where n is the sample size.
2. fitrtree estimates the probability that an observation is in node t using
  
  $p (t) = \sum_{j \in t} w_{j} .$
3. fitrtree sorts x_i in ascending order. each element of the sorted predictor is a splitting candidate or cut point. fitrtree records any indices corresponding to missing values in the set t_u, which is the unsplit set.
4. fitrtree determines the best way to split node t using x_i by maximizing the reduction in mse (δi) over all splitting candidates. that is, for all splitting candidates in x_i:
  1. fitrtree splits the observations in node t into left and right child nodes (t_l and t_r, respectively).
  2. fitrtree computes δi. suppose that for a particular splitting candidate, t_l and t_r contain observation indices in the sets t_l and t_r, respectively.
    
    if x_i does not contain any missing values, then the reduction in mse for the current splitting candidate is
    
    $δ i = p (t) ε_{t} - p (t_{l}) ε_{t_{l}} - p (t_{r}) ε_{t_{r}} .$
    if x_i contains missing values, then, assuming that the observations are missing at random, the reduction in mse is
    
    $δ i_{u} = p (t - t_{u}) ε_{t} - p (t_{l}) ε_{t_{l}} - p (t_{r}) ε_{t_{r}} .$
    t – t_u is the set of all observation indices in node t that are not missing.
    if you use surrogate decision splits, then:
    
    fitrtree computes the predictive measures of association between the decision split x_j < u and all possible decision splits x_k < v, j ≠ k.
    fitrtree sorts the possible alternative decision splits in descending order by their predictive measure of association with the optimal split. the surrogate split is the decision split yielding the largest measure.
    fitrtree decides the child node assignments for observations with a missing value for x_i using the surrogate split. if the surrogate predictor also contains a missing value, then fitrtree uses the decision split with the second largest measure, and so on, until there are no other surrogates. it is possible for fitrtree to split two different observations at node t using two different surrogate splits. for example, suppose the predictors x₁ and x₂ are the best and second best surrogates, respectively, for the predictor x_i, i ∉ {1,2}, at node t. if observation m of predictor x_i is missing (i.e., x_mi is missing), but x_m1 is not missing, then x₁ is the surrogate predictor for observation x_mi. if observations x_{(m
    1),i} and x(m 1),1 are missing, but x_{(m
    1),2} is not missing, then x₂ is the surrogate predictor for observation m 1.
    fitrtree uses the appropriate mse reduction formula. that is, if fitrtree fails to assign all missing observations in node t to children nodes using surrogate splits, then the mse reduction is δi_u. otherwise, fitrtree uses δi for the mse reduction.
  3. fitrtree chooses the candidate that yields the largest mse reduction.
fitrtree splits the predictor variable at the cut point that maximizes the mse reduction.
for the curvature test (that is, if predictorselection is 'curvature'):
1. fitrtree computes the residuals $r_{t i} = y_{t i} - {\bar{y}}_{t}$ for all observations in node t. ${\bar{y}}_{t} = \frac{1}{\sum_{i} w_{i}} \sum_{i} w_{i} y_{t i}$ , which is the weighted average of the responses in node t. the weights are the observation weights in weights.
2. fitrtree assigns observations to one of two bins according to the sign of the corresponding residuals. let z_t be a nominal variable that contains the bin assignments for the observations in node t.
3. fitrtree conducts curvature tests between each predictor and z_t. for regression trees, k = 2.
  - if all p-values are at least 0.05, then fitrtree does not split node t.
  - if there is a minimal p-value, then fitrtree chooses the corresponding predictor to split node t.
  - if more than one p-value is zero due to underflow, then fitrtree applies standard cart to the corresponding predictors to choose the split predictor.
4. if fitrtree chooses a split predictor, then it uses standard cart to choose the cut point (see step 4 in the standard cart process).
for the interaction test (that is, if predictorselection is 'interaction-curvature' ):
1. for observations in node t, fitrtree conducts curvature tests between each predictor and the response and interaction tests between each pair of predictors and the response.
  - if all p-values are at least 0.05, then fitrtree does not split node t.
  - if there is a minimal p-value and it is the result of a curvature test, then fitrtree chooses the corresponding predictor to split node t.
  - if there is a minimal p-value and it is the result of an interaction test, then fitrtree chooses the split predictor using standard cart on the corresponding pair of predictors.
  - if more than one p-value is zero due to underflow, then fitrtree applies standard cart to the corresponding predictors to choose the split predictor.
2. if fitrtree chooses a split predictor, then it uses standard cart to choose the cut point (see step 4 in the standard cart process).

tree depth control

if mergeleaves is 'on' and prunecriterion is 'mse' (which are the default values for these name-value pair arguments), then the software applies pruning only to the leaves and by using mse. this specification amounts to merging leaves coming from the same parent node whose mse is at most the sum of the mse of its two leaves.
to accommodate maxnumsplits, fitrtree splits all nodes in the current layer, and then counts the number of branch nodes. a layer is the set of nodes that are equidistant from the root node. if the number of branch nodes exceeds maxnumsplits, fitrtree follows this procedure:
1. determine how many branch nodes in the current layer must be unsplit so that there are at most maxnumsplits branch nodes.
2. sort the branch nodes by their impurity gains.
3. unsplit the number of least successful branches.
4. return the decision tree grown so far.
this procedure produces maximally balanced trees.
the software splits branch nodes layer by layer until at least one of these events occurs:
- there are maxnumsplits branch nodes.
- a proposed split causes the number of observations in at least one branch node to be fewer than minparentsize.
- a proposed split causes the number of observations in at least one leaf node to be fewer than minleafsize.
- the algorithm cannot find a good split within a layer (i.e., the pruning criterion (see prunecriterion), does not improve for all proposed splits in a layer). a special case is when all nodes are pure (i.e., all observations in the node have the same class).
- for values 'curvature' or 'interaction-curvature' of predictorselection, all tests yield p-values greater than 0.05.
maxnumsplits and minleafsize do not affect splitting at their default values. therefore, if you set 'maxnumsplits', splitting might stop due to the value of minparentsize, before maxnumsplits splits occur.

parallelization

for dual-core systems and above, fitrtree parallelizes training decision trees using intel^® threading building blocks (tbb). for details on intel tbb, see .

references

[1] breiman, l., j. friedman, r. olshen, and c. stone. classification and regression trees. boca raton, fl: crc press, 1984.

[2] loh, w.y. “regression trees with unbiased variable selection and interaction detection.” statistica sinica, vol. 12, 2002, pp. 361–386.

[3] loh, w.y. and y.s. shih. “split selection methods for classification trees.” statistica sinica, vol. 7, 1997, pp. 815–840.

extended capabilities

tall arrays
calculate with arrays that have more rows than fit in memory.

usage notes and limitations:

for more information, see .

automatic parallel support
accelerate code by automatically running computation in parallel using parallel computing toolbox™.

to perform parallel hyperparameter optimization, use the 'hyperparameteroptimizationoptions', struct('useparallel',true) name-value argument in the call to the fitrtree function.

for more information on parallel hyperparameter optimization, see .

for general information about parallel computing, see (parallel computing toolbox).

gpu arrays
accelerate code by running on a graphics processing unit (gpu) using parallel computing toolbox™.

usage notes and limitations:

fitrtree does not support surrogate splits. you can specify the name-value argument surrogate only as "off".
for data with categorical predictors, you can specify the name-value argument numvariablestosample only as "all".
you can specify the name-value argument predictorselection only as "allsplits".
fitrtree fits the model on a gpu if any of the following apply:
- the input argument x is a gpuarray object.
- the input argument y is a gpuarray object.
- the input argument tbl contains gpuarray predictor or response variables.
note that fitrtree might not execute faster on a gpu than a cpu for deeper decision trees.

for more information, see run matlab functions on a gpu (parallel computing toolbox).

version history

introduced in r2014a

fit binary decision tree for regression -凯发k8网页登录

syntax

description

examples

construct regression tree

control regression tree depth

optimize regression tree

unbiased predictor importance estimates

control maximum tree depth on tall array

optimize regression tree on tall array

input arguments

tbl — sample data table

responsevarname — response variable name name of variable in tbl

formula — explanatory model of response variable and subset of predictor variables character vector | string scalar

y — response data numeric column vector

x — predictor data numeric matrix

name-value arguments

categoricalpredictors — categorical predictors list vector of positive integers | logical vector | character matrix | string array | cell array of character vectors | 'all'

maxdepth — maximum tree depth positive integer

mergeleaves — leaf merge flag 'on' (default) | 'off'

minparentsize — minimum number of branch node observations 10 (default) | positive integer value

numbins — number of bins for numeric predictors [](empty) (default) | positive integer scalar

predictornames — predictor variable names string array of unique names | cell array of unique character vectors

predictorselection — algorithm used to select the best split predictor 'allsplits' (default) | 'curvature' | 'interaction-curvature'

prune — flag to estimate optimal sequence of pruned subtrees 'on' (default) | 'off'

prunecriterion — pruning criterion 'mse' (default)

quadraticerrortolerance — quadratic error tolerance 1e-6 (default) | positive scalar value

reproducible — flag to enforce reproducibility false (logical 0) (default) | true (logical 1)

responsename — response variable name "y" (default) | character vector | string scalar

responsetransform — response transformation 'none' (default) | function handle

splitcriterion — split criterion 'mse' (default)

surrogate — surrogate decision splits flag 'off' (default) | 'on' | 'all' | positive integer

weights — observation weights ones(size(x,1),1) (default) | vector of scalar values | name of variable in tbl

crossval — cross-validation flag 'off' (default) | 'on'

cvpartition — partition for cross-validation tree cvpartition object

holdout — fraction of data for holdout validation 0 (default) | scalar value in the range [0,1]

kfold — number of folds 10 (default) | positive integer greater than 1

leaveout — leave-one-out cross-validation flag 'off' (default) | 'on'

maxnumsplits — maximal number of decision splits size(x,1) - 1 (default) | positive integer

minleafsize — minimum number of leaf node observations 1 (default) | positive integer value

numvariablestosample — number of predictors to select at random for each split 'all' (default) | positive integer value

optimizehyperparameters — parameters to optimize 'none' (default) | 'auto' | 'all' | string array or cell array of eligible parameter names | vector of optimizablevariable objects

hyperparameteroptimizationoptions — options for optimization structure

output arguments

tree — regression tree regression tree object

more about

curvature test

interaction test

predictive measure of association

surrogate decision splits

tips

algorithms

node splitting rules

tree depth control

parallelization

references

extended capabilities

tall arrays calculate with arrays that have more rows than fit in memory.

automatic parallel support accelerate code by automatically running computation in parallel using parallel computing toolbox™.

gpu arrays accelerate code by running on a graphics processing unit (gpu) using parallel computing toolbox™.

version history

see also

topics

wechat

`tbl` — sample data
table

`responsevarname` — response variable name
name of variable in `tbl`

`formula` — explanatory model of response variable and subset of predictor variables
character vector | string scalar

`y` — response data
numeric column vector

`x` — predictor data
numeric matrix

`categoricalpredictors` — categorical predictors list
vector of positive integers | logical vector | character matrix | string array | cell array of character vectors | `'all'`

`maxdepth` — maximum tree depth
positive integer

`mergeleaves` — leaf merge flag
`'on'` (default) | `'off'`

`minparentsize` — minimum number of branch node observations
`10` (default) | positive integer value

`numbins` — number of bins for numeric predictors
`[]`(empty) (default) | positive integer scalar

`predictornames` — predictor variable names
string array of unique names | cell array of unique character vectors

`predictorselection` — algorithm used to select the best split predictor
`'allsplits'` (default) | `'curvature'` | `'interaction-curvature'`

`prune` — flag to estimate optimal sequence of pruned subtrees
`'on'` (default) | `'off'`

`prunecriterion` — pruning criterion
`'mse'` (default)

`quadraticerrortolerance` — quadratic error tolerance
`1e-6` (default) | positive scalar value

`reproducible` — flag to enforce reproducibility
`false` (logical `0`) (default) | `true` (logical `1`)

`responsename` — response variable name
`"y"` (default) | character vector | string scalar

`responsetransform` — response transformation
`'none'` (default) | function handle

`splitcriterion` — split criterion
`'mse'` (default)

`surrogate` — surrogate decision splits flag
`'off'` (default) | `'on'` | `'all'` | positive integer

`weights` — observation weights
`ones(size(x,1),1)` (default) | vector of scalar values | name of variable in `tbl`

`crossval` — cross-validation flag
`'off'` (default) | `'on'`

`cvpartition` — partition for cross-validation tree
`cvpartition` object

`holdout` — fraction of data for holdout validation
`0` (default) | scalar value in the range `[0,1]`

`kfold` — number of folds
`10` (default) | positive integer greater than 1

`leaveout` — leave-one-out cross-validation flag
`'off'` (default) | `'on'`

`maxnumsplits` — maximal number of decision splits
`size(x,1) - 1` (default) | positive integer

`minleafsize` — minimum number of leaf node observations
`1` (default) | positive integer value

`numvariablestosample` — number of predictors to select at random for each split
`'all'` (default) | positive integer value

`optimizehyperparameters` — parameters to optimize
`'none'` (default) | `'auto'` | `'all'` | string array or cell array of eligible parameter names | vector of `optimizablevariable` objects

`hyperparameteroptimizationoptions` — options for optimization
structure

`tree` — regression tree
regression tree object

tall arrays
calculate with arrays that have more rows than fit in memory.

automatic parallel support
accelerate code by automatically running computation in parallel using parallel computing toolbox™.

gpu arrays
accelerate code by running on a graphics processing unit (gpu) using parallel computing toolbox™.