一、Logistic回归实现
（一）特征值较少的情况
1. 实验数据
吴恩达《机器学习》第二课时作业提供数据1。判断一个学生能否被一个大学录取，给出的数据集为学生两门课的成绩和是否被录取，通过这些数据来预测一个学生能否被录取。
2. 分类结果评估
横纵轴（特征）为学生两门课成绩，可以在图中清晰地画出决策边界。
3. 代码实现
首先自己实现了梯度下降方法并测试
gradientDesent.m
%Logistic gradientDesent
function [Theta] = gradientDescentLog(X, y, Theta, alpha, counter)
[m,n] = size(X); % m样本数量 n特征数
H = zeros(m,1);
for iter = 1:counter
H = 1./(1 + exp(X * Theta));
Delta = (1/m) * X'*(H-y)
Theta = Theta + alpha * Delta;
Jtheta = (-1/m)*(y'*log(H)+(1-y)'*log(1-H))
end
接下来用课程中讲的高级优化方法，并实现costFunction函数。
%% Machine Learning Online Class - Exercise 2: Logistic Regression %
%% Initialization
clear ; close all; clc
%% Load Data
% The first two columns contains the exam scores and the third column
% contains the label.
data = load('ex2data1.txt');
X = data(:, [1, 2]);
y = data(:, 3);
%% ==================== Part 1: Plotting ====================
%We start the exercise by first plotting the data to understand the
% the problem we are working with.
fprintf(['Plotting data with + indicating (y = 1) examples and o ' ...
'indicating (y = 0) examples.\n']);
plotData(X, y);
xlabel('Exam 1 score')
ylabel('Exam 2 score')
legend('Admitted', 'Not admitted')
hold off;
fprintf('\nProgram paused. Press enter to continue.\n');
pause;
%% ============ Part 2: Compute Cost and Gradient ============
[m, n] = size(X);
% Add intercept term to x and X_test
X = [ones(m, 1) X];
% Initialize fitting parameters
initial_theta = zeros(n + 1, 1);
% Compute and display initial cost and gradient
[cost, grad] = costFunction(initial_theta, X, y);
fprintf('Cost at initial theta (zeros): %f\n', cost);
fprintf('Expected cost (approx): 0.693\n');
fprintf('Gradient at initial theta (zeros): \n');
fprintf(' %f \n', grad);
fprintf('Expected gradients (approx):\n -0.1000\n -12.0092\n -11.2628\n');
% Compute and display cost and gradient with non-zero theta
test_theta = [-24; 0.2; 0.2];
[cost, grad] = costFunction(test_theta, X, y);
fprintf('\nCost at test theta: %f\n', cost);
fprintf('Expected cost (approx): 0.218\n');
fprintf('Gradient at test theta: \n');
fprintf(' %f \n', grad);
fprintf('Expected gradients (approx):\n 0.043\n 2.566\n 2.647\n');
fprintf('\nProgram paused. Press enter to continue.\n');
pause;
%% ============= Part 3: Optimizing using fminunc =============
% Set options for fminunc
options = optimset('GradObj', 'on', 'MaxIter', 40);
% Run fminunc to obtain the optimal theta
% This function will return theta and the cost
[theta, cost] = ...
fminunc(@(t)(costFunction(t, X, y)), initial_theta, options);
% Print theta to screen
fprintf('Cost at theta found by fminunc: %f\n', cost);
fprintf('Expected cost (approx): 0.203\n');
fprintf('theta: \n');
fprintf(' %f \n', theta);
fprintf('Expected theta (approx):\n');
fprintf(' -25.161\n 0.206\n 0.201\n');
% Plot Boundary
plotDecisionBoundary(theta, X, y);
% Put some labels hold on;
% Labels and Legend
xlabel('Exam 1 score')
ylabel('Exam 2 score')
% Specified in plot order
legend('Admitted', 'Not admitted')
hold off;
fprintf('\nProgram paused. Press enter to continue.\n');
pause;
%% ============== Part 4: Predict and Accuracies ==============
prob = sigmoid([1 45 85] * theta);
fprintf(['For a student with scores 45 and 85, we predict an admission ' ...
'probability of %f\n'], prob);
fprintf('Expected value: 0.775 +/- 0.002\n\n');
% Compute accuracy on our training set
p = predict(theta, X);
fprintf('Train Accuracy: %f\n', mean(double(p == y)) * 100);
fprintf('Expected accuracy (approx): 89.0\n');
fprintf('\n');
predict.m
function p = predict(theta, X)
%PREDICT Predict whether the label is or using learned logistic
m = size(X, );
% Number of training examples
p = sigmoid(X * theta)>=0.5;
end
sigmoid.m
function g = sigmoid(z)
%SIGMOID Compute sigmoid function
g = zeros(size(z));
g = 1./(1 + exp(-z));
end
costFunction.m
function [J, grad] = costFunction(theta, X, y)
%COSTFUNCTION Compute cost and gradient for logistic regression
m = length(y);
% number of training examples
J = 0;
grad = zeros(size(theta));
J = 1/m*(-y'*log(sigmoid(X*theta)) - (1-y)'*(log(1-sigmoid(X*theta))));
grad = 1/m * X'*(sigmoid(X*theta) - y);
end
plotData.m
function plotData(X, y)
pos = find(y == 1);
neg = find(y == 0);
plot(X(pos, 1), X(pos, 2), 'k+','LineWidth', 1, ...
'MarkerSize', 7);
hold on；
plot(X(neg, 1), X(neg, 2), 'ko', 'MarkerFaceColor', 'y', ...
'MarkerSize', 7);
plotDecisionBoundary.m（course provide）
function plotDecisionBoundary(theta, X, y)
%函数plotDate
plotData(X(:,:), y);
hold on
if size(X, ) <=
% Only need points to define a line, so choose two endpoints
plot_x = [min(X(:,))-, max(X(:,))+];
% Calculate the decision boundary line
plot_y = (-./theta()).*(theta().*plot_x + theta());
% Plot, and adjust axes for better viewing
plot(plot_x, plot_y)
% Legend, specific for the exercise
legend('Admitted', 'Not admitted', 'Decision Boundary')
axis([, , , ])
else
% Here is the grid range
u = linspace(-, 1.5, );
v = linspace(-, 1.5, );
z = zeros(length(u), length(v));
% Evaluate z = theta*x over the grid
for i = :length(u)
for j = :length(v)
z(i,j) = mapFeature(u(i), v(j))*theta;
end
end
z = z'; % important to transpose z before calling contour
% Plot z =
% Notice you need to specify the range [, ]
contour(u, v, z, [, ], 'LineWidth', ) %画等值线
end
hold off
end
输出结果：
For a student with scores and , we predict an admission probability of 0.771019
Expected value: 0.775 +/- 0.002 Train Accuracy: 89.000000
Expected accuracy (approx): 89.0
（二）特征值较多的情况
1. 实验数据
http://archive.ics.uci.edu/ml/index.php wine数据集，其特征取值是连续的。
2. 分类结果评估
考虑一个二分问题，即将实例分成正类（positive）或负类（negative）。对一个二分问题来说，会出现四种情况：
实例是正类并且也被预测成正类，即为真正类（TP：True positive）
实例是负类被预测成正类，称之为假正类（FP：False positive）
实例是负类被预测成负类，称之为真负类（TN：True negative）
实例是正类被预测成负类则为假负类（FN：false negative）。
评价标准：
精确率：precision = TP / (TP + FP)模型判为正的所有样本中有多少是真正的正样本。
召回率：recall = TP / (TP + FN)
准确率：accuracy = (TP + TN) / (TP + FP + TN + FN)反映了分类器统对整个样本的判定能力——能将正的判定为正，负的判定为负
如何在precision和Recall中权衡？F1 Score = P*R/2(P+R)，其中P和R分别为 precision 和 recall，在precision与recall都要求高的情况下，可以用F1 Score来衡量。
为什么会有这么多指标呢？这是因为模式分类和机器学习的需要。判断一个分类器对所用样本的分类能力或者在不同的应用场合时，需要有不同的指标。
3. 代码实现
%Logistic回归梯度下降法
%logistic梯度下降法
[Data] = xlsread('wine.xlsx',,'B1:N130');
[y] = xlsread('wine.xlsx',,'A1:A130');
[m,n] = size(Data); % m样本数量 n特征数
Data = featureScaling(Data);
for i = :m
if y(i)==
y(i) = ;
end
end
x0 = ones(m,); X = ([x0,Data])';
Theta = zeros(n+,);
alpha = 0.1;
counter = ;
H = gradientDescentLog(X, y, Theta, alpha, counter);
TP = ; TN = ; FP = ; FN = ;
for i = :m
if H(i)<0.5 %判断为negative
if y(i)==
TN = TN+;
else
FN = FN+;
end
else %判断为positive
if y(i)==
TP = TP+;
else
FP = FP+;
end
end
end
precision = TP / (TP + FP)
recall = TP / (TP + FN)
accuracy = (TP + TN) / (TP + FP + TN + FN)
%Logistic gradientDesent
function [H] = gradientDescentLog(X, y, Theta, alpha, counter)
[n,m] = size(X); % m样本数量 n特征数
H = zeros(m,1);
for iter = 1:counter
for i = 1:m
H(i) = 1/(1+exp(-Theta'*X(:,i))); %Logistic回归模型
end
Delta = 1/m * X * (H-y);
Theta = Theta - alpha * Delta;
Jtheta = -1/m*(y'*log(H)+(1-y)'*log(1-H))
end
输出结果：
Jtheta = 0.3497
precision = 0.8983
recall = 0.8983
accuracy = 0.9077
此时alpha = 0.1; counter = 4000;可见此时很可能已经出现过拟合现象，在下一篇笔记中我们针对这种过拟合现象进行讨论。