train linear regression model -凯发k8网页登录
statistics and machine learning toolbox™ provides several features for training a linear regression model.
for greater accuracy on low-dimensional through medium-dimensional data sets, use . after fitting the model, you can use the object functions to improve, evaluate, and visualize the fitted model. to regularize a regression, use or .
for reduced computation time on high-dimensional data sets, use
fitrlinear. this function offers useful options for cross-validation, regularization, and hyperparameter optimization.
this example shows the typical workflow for linear regression analysis using fitlm. the workflow includes preparing a data set, fitting a linear regression model, evaluating and improving the fitted model, and predicting response values for new predictor data. the example also describes how to fit and evaluate a linear regression model for tall arrays.
prepare data
load the sample data set nychousing2015.
load nychousing2015the data set includes 10 variables with information on the sales of properties in new york city in 2015. this example uses some of these variables to analyze the sale prices.
instead of loading the sample data set nychousing2015, you can download the data from the website and import the data as follows.
folder = 'annualized_rolling_sales_update'; ds = spreadsheetdatastore(folder,"texttype","string","numheaderlines",4); ds.files = ds.files(contains(ds.files,"2015")); ds.selectedvariablenames = ["borough","neighborhood","buildingclasscategory","residentialunits", ... "commercialunits","landsquarefeet","grosssquarefeet","yearbuilt","saleprice","saledate"]; nychousing2015 = readall(ds);
preprocess the data set to choose the predictor variables of interest. first, change the variable names to lowercase for readability.
nychousing2015.properties.variablenames = lower(nychousing2015.properties.variablenames);
next, convert the saledate variable, specified as a datetime array, into two numeric columns mm (month) and dd (day), and remove the saledate variable. ignore the year values because all samples are for the year 2015.
[~,nychousing2015.mm,nychousing2015.dd] = ymd(nychousing2015.saledate); nychousing2015.saledate = [];
the numeric values in the borough variable indicate the names of the boroughs. change the variable to a categorical variable using the names.
nychousing2015.borough = categorical(nychousing2015.borough,1:5, ... ["manhattan","bronx","brooklyn","queens","staten island"]);
the neighborhood variable has 254 categories. remove this variable for simplicity.
nychousing2015.neighborhood = [];
convert the buildingclasscategory variable to a categorical variable, and explore the variable by using the function.
nychousing2015.buildingclasscategory = categorical(nychousing2015.buildingclasscategory); wordcloud(nychousing2015.buildingclasscategory);
assume that you are interested only in one-, two-, and three-family dwellings. find the sample indices for these dwellings and delete the other samples. then, change the data type of the buildingclasscategory variable to double.
idx = ismember(string(nychousing2015.buildingclasscategory), ... ["01 one family dwellings","02 two family dwellings","03 three family dwellings"]); nychousing2015 = nychousing2015(idx,:); nychousing2015.buildingclasscategory = renamecats(nychousing2015.buildingclasscategory, ... ["01 one family dwellings","02 two family dwellings","03 three family dwellings"], ... ["1","2","3"]); nychousing2015.buildingclasscategory = double(nychousing2015.buildingclasscategory);
the buildingclasscategory variable now indicates the number of families in one dwelling.
explore the response variable saleprice using the summary function.
s = summary(nychousing2015); s.saleprice
ans = struct with fields:
size: [37881 1]
type: 'double'
description: ''
units: ''
continuity: []
min: 0
median: 352000
max: 37000000
nummissing: 0
assume that a saleprice less than or equal to $1000 indicates ownership transfer without a cash consideration. remove the samples that have this saleprice.
idx0 = nychousing2015.saleprice <= 1000; nychousing2015(idx0,:) = [];
create a histogram of the saleprice variable.
histogram(nychousing2015.saleprice)
the maximum value of saleprice is , but most values are smaller than . you can identify the outliers of saleprice by using the function.
idx = isoutlier(nychousing2015.saleprice);
remove the identified outliers and create the histogram again.
nychousing2015(idx,:) = []; histogram(nychousing2015.saleprice)
partition the data set into a training set and test set by using cvpartition.
rng('default') % for reproducibility c = cvpartition(height(nychousing2015),"holdout",0.3); traindata = nychousing2015(training(c),:); testdata = nychousing2015(test(c),:);
train model
fit a linear regression model by using the function.
mdl = fitlm(traindata,"predictorvars",["borough","grosssquarefeet", ... "landsquarefeet","buildingclasscategory","yearbuilt","mm","dd"], ... "responsevar","saleprice")
mdl =
linear regression model:
saleprice ~ 1 borough buildingclasscategory landsquarefeet grosssquarefeet yearbuilt mm dd
estimated coefficients:
estimate se tstat pvalue
___________ __________ ________ ___________
(intercept) 2.0345e 05 1.0308e 05 1.9736 0.048441
borough_bronx -3.0165e 05 56676 -5.3224 1.0378e-07
borough_brooklyn -41160 56490 -0.72862 0.46624
borough_queens -91136 56537 -1.612 0.10699
borough_staten island -2.2199e 05 56726 -3.9134 9.1385e-05
buildingclasscategory 3165.7 3510.3 0.90185 0.36715
landsquarefeet 13.149 0.84534 15.555 3.714e-54
grosssquarefeet 112.34 2.9494 38.09 8.0393e-304
yearbuilt 100.07 45.464 2.201 0.02775
mm 3850.5 543.79 7.0808 1.4936e-12
dd -367.19 207.56 -1.7691 0.076896
number of observations: 15848, error degrees of freedom: 15837
root mean squared error: 2.32e 05
r-squared: 0.235, adjusted r-squared: 0.235
f-statistic vs. constant model: 487, p-value = 0
mdl is a object. the model display includes the model formula, estimated coefficients, and summary statistics.
borough is a categorical variable that has five categories: manhattan, bronx, brooklyn, queens, and staten island. the fitted model mdl has four indicator variables. the fitlm function uses the first category manhattan as a reference level, so the model does not include the indicator variable for the reference level. fitlm fixes the coefficient of the indicator variable for the reference level as zero. the coefficient values of the four indicator variables are relative to manhattan. for more details on how the function treats a categorical predictor, see of fitlm.
to learn how to interpret the values in the model display, see .
you can use the properties of a linearmodel object to investigate a fitted linear regression model. the object properties include information about coefficient estimates, summary statistics, fitting method, and input data. for example, you can find the r-squared and adjusted r-squared values in the rsquared property. you can access the property values through the workspace browser or using dot notation.
mdl.rsquared
ans = struct with fields:
ordinary: 0.2352
adjusted: 0.2348
the model display also shows these values. the r-squared value indicates that the model explains approximately 24% of the variability in the response variable. see of a linearmodel object for details about other properties.
evaluate model
the model display shows the p-value of each coefficient. the p-values indicate which variables are significant to the model. for the categorical predictor borough, the model uses four indicator variables and displays four p-values. to examine the categorical variable as a group of indicator variables, use the object function . this function returns analysis of variance (anova) statistics of the model.
anova(mdl)
ans=8×5 table
sumsq df meansq f pvalue
__________ _____ __________ _______ ___________
borough 1.123e 14 4 2.8076e 13 520.96 0
buildingclasscategory 4.3833e 10 1 4.3833e 10 0.81334 0.36715
landsquarefeet 1.3039e 13 1 1.3039e 13 241.95 3.714e-54
grosssquarefeet 7.8189e 13 1 7.8189e 13 1450.8 8.0393e-304
yearbuilt 2.6108e 11 1 2.6108e 11 4.8444 0.02775
mm 2.7021e 12 1 2.7021e 12 50.138 1.4936e-12
dd 1.6867e 11 1 1.6867e 11 3.1297 0.076896
error 8.535e 14 15837 5.3893e 10
the p-values for the indicator variables borough_brooklyn and borough_queens are large, but the p-value of the borough variable as a group of four indicator variables is almost zero, which indicates that the borough variable is statistically significant.
the p-values of buildingclasscategory and dd are larger than 0.05, which indicates that these variables are not significant at the 5% significance level. therefore, you can consider removing these variables.
you can also use coeffci, coeeftest, and dwtest to further evaluate the fitted model.
returns confidence intervals of the coefficient estimates.
performs a linear hypothesis test on the model coefficients.
performs the durbin-watson test. (this test is used for time series data, so
dwtestis not appropriate for the housing data in this example.)
visualize model and summary statistics
a linearmodel object provides multiple plotting functions.
when creating a model, use to understand the effect of adding or removing a predictor variable.
when verifying a model, use to find questionable data and to understand the effect of each observation. also, use to analyze the residuals of the model.
after fitting a model, use ,
plotpartialdependence, and to understand the effect of a particular predictor. use to examine the interaction effect between two predictors. also, use to plot slices through the prediction surface.
in addition, creates an added variable plot for the whole model, except the intercept term, if mdl includes multiple predictor variables.
plot(mdl)
this plot is equivalent to plotadded(mdl). the fitted line represents how the model, as a group of variables, can explain the response variable. the slope of the fitted line is not close to zero, and the confidence bound does not include a horizontal line, indicating that the model fits better than a degenerate model consisting of only a constant term. the test statistic value shown in the model display (f-statistic vs. constant model) also indicates that the model fits better than the degenerate model.
create an added variable plot for the insignificant variables buildingclasscategory and dd. the p-values of these variables are larger than 0.05. first, find the indices of these coefficients in mdl.coefficientnames.
mdl.coefficientnames
ans = 1×11 cell
{'(intercept)'} {'borough_bronx'} {'borough_brooklyn'} {'borough_queens'} {'borough_staten island'} {'buildingclasscategory'} {'landsquarefeet'} {'grosssquarefeet'} {'yearbuilt'} {'mm'} {'dd'}
buildingclasscategory and dd are the 6th and 11th coefficients, respectively. create an added plot for these two variables.
plotadded(mdl,[6,11])
the slope of the fitted line is close to zero, indicating that the information from the two variables does not explain the part of the response values not explained by the other predictors. for more details about an added variable plot, see .
create a histogram of the model residuals. plots a histogram of the raw residuals using probability density function scaling.
plotresiduals(mdl)
the histogram shows that a few residuals are smaller than . identify these outliers.
find(mdl.residuals.raw < -1*10^6)
ans = 4×1
1327
4136
4997
13894
alternatively, you can find the outliers by using isoutlier. specify the 'grubbs' option to apply grubb's test. this option is suitable for a normally distributed data set.
find(isoutlier(mdl.residuals.raw,'grubbs'))ans = 3×1
1327
4136
4997
the isoutlier function does not identify residual 13894 as an outlier. this residual is close to –110. display the residual value.
mdl.residuals.raw(13894)
ans = -1.0720e 06
you can exclude outliers when fitting a linear regression model by using the name-value pair argument. in this case, the example adjusts the fitted model and checks whether the improved model can also explain the outliers.
adjust model
remove the dd and buildingclasscategory variables using .
newmdl1 = removeterms(mdl,"dd buildingclasscategory")newmdl1 =
linear regression model:
saleprice ~ 1 borough landsquarefeet grosssquarefeet yearbuilt mm
estimated coefficients:
estimate se tstat pvalue
___________ __________ ________ __________
(intercept) 2.0529e 05 1.0274e 05 1.9981 0.045726
borough_bronx -3.0038e 05 56675 -5.3 1.1739e-07
borough_brooklyn -39704 56488 -0.70286 0.48215
borough_queens -90231 56537 -1.596 0.11052
borough_staten island -2.2149e 05 56720 -3.9049 9.4652e-05
landsquarefeet 13.04 0.83912 15.54 4.6278e-54
grosssquarefeet 113.85 2.5078 45.396 0
yearbuilt 96.649 45.395 2.1291 0.033265
mm 3875.6 543.49 7.131 1.0396e-12
number of observations: 15848, error degrees of freedom: 15839
root mean squared error: 2.32e 05
r-squared: 0.235, adjusted r-squared: 0.235
f-statistic vs. constant model: 608, p-value = 0
because the two variables are not significant in explaining the response variable, the r-squared and adjusted r-squared values of newmdl1 are close to the values of mdl.
improve the model by adding or removing variables using . the default upper bound of the model is a model containing an intercept term, the linear term for each predictor, and all products of pairs of distinct predictors (no squared terms), and the default lower bound is a model containing an intercept term. specify the maximum number of steps to take as 30. the function stops when no single step improves the model.
newmdl2 = step(newmdl1,'nsteps',30)1. adding borough:grosssquarefeet, fstat = 58.7413, pvalue = 2.63078e-49 2. adding borough:yearbuilt, fstat = 31.5067, pvalue = 3.50645e-26 3. adding borough:landsquarefeet, fstat = 29.5473, pvalue = 1.60885e-24 4. adding grosssquarefeet:yearbuilt, fstat = 69.312, pvalue = 9.08599e-17 5. adding landsquarefeet:grosssquarefeet, fstat = 33.2929, pvalue = 8.07535e-09 6. adding landsquarefeet:yearbuilt, fstat = 45.2756, pvalue = 1.7704e-11 7. adding yearbuilt:mm, fstat = 18.0785, pvalue = 2.13196e-05 8. adding residentialunits, fstat = 16.0491, pvalue = 6.20026e-05 9. adding residentialunits:landsquarefeet, fstat = 160.2601, pvalue = 1.49309e-36 10. adding residentialunits:grosssquarefeet, fstat = 27.351, pvalue = 1.71835e-07 11. adding commercialunits, fstat = 14.1503, pvalue = 0.000169381 12. adding commercialunits:grosssquarefeet, fstat = 25.6942, pvalue = 4.04549e-07 13. adding borough:commercialunits, fstat = 6.1327, pvalue = 6.3015e-05 14. adding buildingclasscategory, fstat = 11.1412, pvalue = 0.00084624 15. adding buildingclasscategory:landsquarefeet, fstat = 66.9205, pvalue = 3.04003e-16 16. adding buildingclasscategory:yearbuilt, fstat = 15.0776, pvalue = 0.0001036 17. adding buildingclasscategory:grosssquarefeet, fstat = 18.3304, pvalue = 1.86812e-05 18. adding residentialunits:yearbuilt, fstat = 15.0732, pvalue = 0.00010384 19. adding buildingclasscategory:residentialunits, fstat = 13.5644, pvalue = 0.00023129 20. adding borough:buildingclasscategory, fstat = 2.8214, pvalue = 0.023567 21. adding landsquarefeet:mm, fstat = 4.9185, pvalue = 0.026585 22. removing grosssquarefeet:yearbuilt, fstat = 1.6052, pvalue = 0.20519
newmdl2 =
linear regression model:
saleprice ~ 1 borough*buildingclasscategory borough*commercialunits borough*landsquarefeet borough*grosssquarefeet borough*yearbuilt buildingclasscategory*residentialunits buildingclasscategory*landsquarefeet buildingclasscategory*grosssquarefeet buildingclasscategory*yearbuilt residentialunits*landsquarefeet residentialunits*grosssquarefeet residentialunits*yearbuilt commercialunits*grosssquarefeet landsquarefeet*grosssquarefeet landsquarefeet*yearbuilt landsquarefeet*mm yearbuilt*mm
estimated coefficients:
estimate se tstat pvalue
___________ __________ ________ __________
(intercept) 2.2152e 07 1.318e 07 1.6808 0.092825
borough_bronx -2.3263e 07 1.3176e 07 -1.7656 0.077486
borough_brooklyn -1.8935e 07 1.3174e 07 -1.4373 0.15064
borough_queens -2.1757e 07 1.3173e 07 -1.6516 0.098636
borough_staten island -2.3471e 07 1.3177e 07 -1.7813 0.074891
buildingclasscategory -7.2403e 05 1.9374e 05 -3.737 0.00018685
residentialunits 6.1912e 05 1.2399e 05 4.9932 6.003e-07
commercialunits 4.2016e 05 1.2815e 05 3.2786 0.0010456
landsquarefeet -390.54 96.349 -4.0535 5.0709e-05
grosssquarefeet 189.33 83.723 2.2614 0.023748
yearbuilt -11556 6958.7 -1.6606 0.096805
mm 95189 31787 2.9946 0.0027521
borough_bronx:buildingclasscategory -1.1972e 05 1.0481e 05 -1.1422 0.25338
borough_brooklyn:buildingclasscategory -1.4154e 05 1.0448e 05 -1.3548 0.17551
borough_queens:buildingclasscategory -1.1597e 05 1.0454e 05 -1.1093 0.2673
borough_staten island:buildingclasscategory -1.1851e 05 1.0513e 05 -1.1273 0.25964
borough_bronx:commercialunits -2.7488e 05 1.3267e 05 -2.0719 0.038293
borough_brooklyn:commercialunits -3.8228e 05 1.2835e 05 -2.9784 0.0029015
borough_queens:commercialunits -3.9818e 05 1.2884e 05 -3.0906 0.0020008
borough_staten island:commercialunits -4.9381e 05 1.353e 05 -3.6496 0.00026348
borough_bronx:landsquarefeet 121.81 77.442 1.573 0.11574
borough_brooklyn:landsquarefeet 113.09 77.413 1.4609 0.14405
borough_queens:landsquarefeet 99.894 77.374 1.2911 0.1967
borough_staten island:landsquarefeet 84.508 77.376 1.0922 0.27477
borough_bronx:grosssquarefeet -55.417 83.412 -0.66437 0.50646
borough_brooklyn:grosssquarefeet 6.4033 83.031 0.077119 0.93853
borough_queens:grosssquarefeet 38.28 83.144 0.46041 0.64523
borough_staten island:grosssquarefeet 12.539 83.459 0.15024 0.88058
borough_bronx:yearbuilt 12121 6956.8 1.7422 0.081485
borough_brooklyn:yearbuilt 9986.5 6955.8 1.4357 0.1511
borough_queens:yearbuilt 11382 6955.3 1.6364 0.10177
borough_staten island:yearbuilt 12237 6957.1 1.7589 0.078613
buildingclasscategory:residentialunits 21392 5465 3.9143 9.1041e-05
buildingclasscategory:landsquarefeet -13.099 2.0014 -6.545 6.1342e-11
buildingclasscategory:grosssquarefeet -30.087 5.2786 -5.6998 1.2209e-08
buildingclasscategory:yearbuilt 462.31 85.912 5.3813 7.5021e-08
residentialunits:landsquarefeet -1.0826 0.13896 -7.7911 7.0554e-15
residentialunits:grosssquarefeet -5.1192 1.7923 -2.8563 0.0042917
residentialunits:yearbuilt -326.69 63.556 -5.1403 2.7762e-07
commercialunits:grosssquarefeet -29.839 5.0231 -5.9403 2.9045e-09
landsquarefeet:grosssquarefeet -0.0055199 0.0010364 -5.3262 1.0165e-07
landsquarefeet:yearbuilt 0.1766 0.030902 5.7151 1.1164e-08
landsquarefeet:mm 0.6595 0.30229 2.1817 0.029145
yearbuilt:mm -47.944 16.392 -2.9248 0.0034512
number of observations: 15848, error degrees of freedom: 15804
root mean squared error: 2.25e 05
r-squared: 0.285, adjusted r-squared: 0.283
f-statistic vs. constant model: 146, p-value = 0
the r-squared and adjusted r-squared values of newmdl2 are larger than the values of newmdl1.
create a histogram of the model residuals by using .
plotresiduals(newmdl2)
the residual histogram of newmdl2 is symmetric, without outliers.
you can also use to add specific terms. alternatively, you can use to specify terms in a starting model and continue improving the model by using stepwise regression.
predict responses to new data
predict responses to the test data set testdata by using the fitted model newmdl2 and the object function to
ypred = predict(newmdl2,testdata);
plot the residual histogram of the test data set.
errs = ypred - testdata.saleprice;
histogram(errs)
title("histogram of residuals - test data")
the residual values have a few outliers.
errs(isoutlier(errs,'grubbs'))ans = 6×1
107 ×
0.1788
-0.4688
-1.2981
0.1019
0.1122
0.1331
analyze using tall arrays
the fitlm function supports tall arrays for out-of-memory data, with some limitations. for tall data, fitlm returns a object that contains most of the same properties as a linearmodel object. the main difference is that the compact object is sensitive to memory requirements. the compact object does not have properties that include the data, or that include an array of the same size as the data. therefore, some linearmodel object functions that require data do not work with a compact model. see for the list of supported object functions. also, see for the usage notes and limitations of fitlm for tall arrays.
when you perform calculations on tall arrays, matlab® uses either a parallel pool (default if you have parallel computing toolbox™) or the local matlab session. if you want to run the example using the local matlab session when you have parallel computing toolbox, you can change the global execution environment by using the function.
assume that all the data in the datastore ds does not fit in memory. you can use instead of readall to read ds.
nychousing2015 = tall(ds);
for this example, convert the in-memory table nychousing2015 to a tall table by using the tall function.
nychousing2015_t = tall(nychousing2015);
starting parallel pool (parpool) using the 'local' profile ... connected to the parallel pool (number of workers: 6).
partition the data set into a training set and test set. when you use cvpartition with tall arrays, the function partitions the data set based on the variable supplied as the first input argument. for classification problems, you typically use the response variable (a grouping variable) and create a random stratified partition to get even distribution between training and test sets for all groups. for regression problems, this stratification is not adequate, and you can use the 'stratify' name-value pair argument to turn off the option.
in this example, specify the predictor variable nychousing2015_t.borough as the first input argument to make the distribution of boroughs roughly the same across the training and tests sets. for reproducibility, set the seed of the random number generator using tallrng. the results can vary depending on the number of workers and the execution environment for the tall arrays. for details, see .
tallrng('default') % for reproducibility c = cvpartition(nychousing2015_t.borough,"holdout",0.3); traindata_t = nychousing2015_t(training(c),:); testdata_t = nychousing2015_t(test(c),:);
because fitlm returns a compact model object for tall arrays, you cannot improve the model using the step function. instead, you can explore the model parameters by using the object functions and then adjust the model as needed. you can also gather a subset of the data into the workspace, use stepwiselm to iteratively develop the model in memory, and then scale up to use tall arrays. for details, see model development of .
in this example, fit a linear regression model using the model formula of newmdl2.
mdl_t = fitlm(traindata_t,newmdl2.formula)
evaluating tall expression using the parallel pool 'local': - pass 1 of 1: completed in 7.4 sec evaluation completed in 9.2 sec
mdl_t =
compact linear regression model:
saleprice ~ 1 borough*buildingclasscategory borough*commercialunits borough*landsquarefeet borough*grosssquarefeet borough*yearbuilt buildingclasscategory*residentialunits buildingclasscategory*landsquarefeet buildingclasscategory*grosssquarefeet buildingclasscategory*yearbuilt residentialunits*landsquarefeet residentialunits*grosssquarefeet residentialunits*yearbuilt commercialunits*grosssquarefeet landsquarefeet*grosssquarefeet landsquarefeet*yearbuilt landsquarefeet*mm yearbuilt*mm
estimated coefficients:
estimate se tstat pvalue
___________ __________ ________ __________
(intercept) -1.3301e 06 5.1815e 05 -2.567 0.010268
borough_brooklyn 4.2583e 06 4.1808e 05 10.185 2.7392e-24
borough_manhattan 2.2758e 07 1.3448e 07 1.6923 0.090614
borough_queens 1.1395e 06 4.1868e 05 2.7216 0.0065035
borough_staten island -1.1196e 05 4.6677e 05 -0.23986 0.81044
buildingclasscategory -8.08e 05 1.6219e 05 -4.9817 6.3705e-07
residentialunits 6.0588e 05 1.2669e 05 4.7822 1.7497e-06
commercialunits 80197 53311 1.5043 0.13252
landsquarefeet -279.94 53.913 -5.1925 2.1009e-07
grosssquarefeet 170.02 13.996 12.147 8.3837e-34
yearbuilt 683.49 268.34 2.5471 0.010872
mm 86488 32725 2.6428 0.0082293
borough_brooklyn:buildingclasscategory -9852.4 12048 -0.81773 0.41352
borough_manhattan:buildingclasscategory 1.3318e 05 1.3592e 05 0.97988 0.32716
borough_queens:buildingclasscategory 15621 11671 1.3385 0.18076
borough_staten island:buildingclasscategory 15132 14893 1.016 0.30964
borough_brooklyn:commercialunits -22060 43012 -0.51289 0.60804
borough_manhattan:commercialunits 4.8349e 05 2.1757e 05 2.2222 0.026282
borough_queens:commercialunits -42023 44736 -0.93936 0.34756
borough_staten island:commercialunits -1.3382e 05 56976 -2.3487 0.018853
borough_brooklyn:landsquarefeet 9.8263 5.2513 1.8712 0.061335
borough_manhattan:landsquarefeet -78.962 78.445 -1.0066 0.31415
borough_queens:landsquarefeet -3.0855 3.9087 -0.78939 0.4299
borough_staten island:landsquarefeet -17.325 3.5831 -4.8351 1.3433e-06
borough_brooklyn:grosssquarefeet 37.689 10.573 3.5646 0.00036548
borough_manhattan:grosssquarefeet 16.107 82.074 0.19625 0.84442
borough_queens:grosssquarefeet 70.381 10.69 6.5837 4.7343e-11
borough_staten island:grosssquarefeet 36.396 12.08 3.0129 0.0025914
borough_brooklyn:yearbuilt -2110.1 216.32 -9.7546 2.0388e-22
borough_manhattan:yearbuilt -11884 7023.9 -1.692 0.090667
borough_queens:yearbuilt -566.44 216.89 -2.6116 0.0090204
borough_staten island:yearbuilt 53.714 239.89 0.22391 0.82283
buildingclasscategory:residentialunits 24088 5574 4.3215 1.5595e-05
buildingclasscategory:landsquarefeet 5.7964 5.8438 0.9919 0.32126
buildingclasscategory:grosssquarefeet -47.079 5.2884 -8.9023 6.0556e-19
buildingclasscategory:yearbuilt 430.97 83.593 5.1555 2.56e-07
residentialunits:landsquarefeet -21.756 5.6485 -3.8517 0.00011778
residentialunits:grosssquarefeet 4.584 1.4586 3.1427 0.0016769
residentialunits:yearbuilt -310.09 65.429 -4.7393 2.1632e-06
commercialunits:grosssquarefeet -27.839 11.463 -2.4286 0.015166
landsquarefeet:grosssquarefeet -0.0068613 0.00094607 -7.2524 4.2832e-13
landsquarefeet:yearbuilt 0.17489 0.028195 6.2028 5.6861e-10
landsquarefeet:mm 0.70295 0.2848 2.4682 0.013589
yearbuilt:mm -43.405 16.871 -2.5728 0.010098
number of observations: 15849, error degrees of freedom: 15805
root mean squared error: 2.26e 05
r-squared: 0.277, adjusted r-squared: 0.275
f-statistic vs. constant model: 141, p-value = 0
mdl_t is a object. mdl_t is not exactly the same as newmdl2 because the partitioned training data set obtained from the tall table is not the same as the one from the in-memory data set.
you cannot use the plotresiduals function to create a histogram of the model residuals because mdl_t is a compact object. instead, compute the residuals directly from the compact object and create the histogram using .
mdl_t_residual = traindata_t.saleprice - predict(mdl_t,traindata_t); histogram(mdl_t_residual)
evaluating tall expression using the parallel pool 'local': - pass 1 of 2: completed in 2.5 sec - pass 2 of 2: completed in 0.63 sec evaluation completed in 3.8 sec
title("histogram of residuals - train data")predict responses to the test data set testdata_t by using .
ypred_t = predict(mdl_t,testdata_t);
plot the residual histogram of the test data set.
errs_t = ypred_t - testdata_t.saleprice; histogram(errs_t)
evaluating tall expression using the parallel pool 'local': - pass 1 of 2: 0% complete evaluation 0% complete
- pass 1 of 2: 6% complete evaluation 3% complete
- pass 1 of 2: completed in 0.79 sec - pass 2 of 2: completed in 0.55 sec evaluation completed in 2 sec
title("histogram of residuals - test data")
you can further assess the fitted model using the compactlinearmodel object functions. for an example, see assess and adjust model of .
see also
| | |
related topics
select a web site
choose a web site to get translated content where available and see local events and offers. based on your location, we recommend that you select: .
you can also select a web site from the following list:
americas
- (español)
- (english)
- (english)
europe
- (english)
- (english)
- (deutsch)
- (español)
- (english)
- (français)
- (english)
- (italiano)
- (english)
- (english)
- (english)
- (deutsch)
- (english)
- (english)
- switzerland
- (english)
asia pacific
- (english)
- (english)
- (english)
- 中国
- (日本語)
- (한국어)