使用sklearn的多元线性回归与成本函数的Normal方程不匹配

I have to fit my data to a multivariate linear Model. But sklearn.linear_model is producing different answers than those predicted by Normal Equation.here is the code for both:

我必须使我的数据适合多元线性模型。但是sklearn.linear_model产生的答案不同于Normal Equation所预测的答案。两者的代码都是:

   x=np.arange(12).reshape(3,4)
   y=np.arange(3,6).reshape(3,1)
   x=np.insert(x,0,1,axis=1)
   def normal(X,y):
       return np.dot(np.dot(linalg.pinv(np.dot(X.T,X)),X.T),y)

   normal(x,y)
   >>> [[ 0.4375 ]
       [-0.59375]
       [-0.15625]
       [ 0.28125]
       [ 0.71875]]
   from sklearn import linear_model
   reg=linear_model.LinearRegression()
   reg.fit(x,y)
   reg.coef_
   >>> [[ 0.    ,  0.0625,  0.0625,  0.0625,  0.0625]]

Is my code correct?

我的代码是否正确?

1 个解决方案

#1

What is happening is that you are including the intercept term in your data matrix. By default, the LinearRegression class of scikit-learn automatically finds the intercept term, so you do not need to insert the column of 1s in your matrix:

发生的事情是您在数据矩阵中包含截距项。默认情况下,scikit-learn的LinearRegression类会自动查找截距项,因此您无需在矩阵中插入1s列:

from sklearn import linear_model
x=np.arange(12).reshape(3,4)
y=np.arange(3,6).reshape(3,1)    
reg=linear_model.LinearRegression()
reg.fit(x,y)

We thus get for the coefficients and intercept terms:

因此,我们得到系数和截距项:

In [32]: reg.coef_
Out[32]: array([[ 0.0625,  0.0625,  0.0625,  0.0625]])

In [33]: reg.intercept_
Out[33]: array([ 2.625])

We can verify that we get the correct output by doing a dot product between each row of the matrix and the coefficients, and add the intercept term at the end

我们可以通过在矩阵的每一行和系数之间做一个点积来验证我们得到正确的输出,并在最后添加截距项

In [34]: x.dot(reg.coef_.T) + reg.intercept_
Out[34]:
array([[ 3.],
       [ 4.],
       [ 5.]])

Now if you want to specifically match what the normal equation gives you, that's fine and you can insert the column of ones. You will however need to disable finding the intercept as you have manually inserted in a feature that would do that for you.

现在,如果你想特别匹配法线方程给你的东西,那很好,你可以插入一列。但是,您需要禁用查找截距,因为您手动插入了可以为您执行此操作的功能。

Therefore:

x=np.arange(12).reshape(3,4)
y=np.arange(3,6).reshape(3,1)
x=np.insert(x,0,1,axis=1)
reg = linear_model.LinearRegression(fit_intercept=False)
reg.fit(x,y)

By doing this, we now get for our coefficients:

通过这样做,我们现在得到我们的系数:

In [37]: reg.coef_
Out[37]: array([[ 0.4375 , -0.59375, -0.15625,  0.28125,  0.71875]])

This matches with the output of the normal equation.

这与正规方程的输出匹配。

#1