下面，将UFLDL教程中的sparseae_exercise练习中的各函数及注释列举如下
首先，给出各函数的调用关系
主函数：train.m
（1）调用sampleIMAGES函数从已知图像中扣取多个图像块儿
（2）调用display_network函数，以网格的形式，随机显示多个扣取的图像块儿
（3）梯度校验，该部分的目的是测试函数是否正确，可以由单独的函数checkSparseAutoencoderCost实现
①利用sparseAutoencoderCost函数计算网路的代价函数和梯度值
②利用computeNumericalGradient函数计算梯度值（这里，要利用checkNumericalGradient函数验证该梯度计算函数是否正确）
③比较①和②的梯度计算结果，判断编写的sparseAutoencoderCost函数是否正确
如果sparseAutoencoderCost函数是正确的，那么，在实际训练中，不需要运行checkSparseAutoencoderCost
（4）利用L-BFGS方法对网络进行训练，从而得到最优化的网络的权值和偏执项
（5）对训练结果进行可视化
然后，对个函数给出注释
train.m
%% CS294A/CS294W Programming Assignment Starter Code addpath ..common\ %%======================================================================
%% STEP : Here we provide the relevant parameters values that will
% allow your sparse autoencoder to get good filters; you do not need to change the parameters below.
visibleSize = *; % number of input units
hiddenSize = ; % number of hidden units
sparsityParam = 0.01; % desired average activation of the hidden units.
% (This was denoted by the Greek alphabet rho, which looks like a lower-case "p", in the lecture notes).
lambda = 0.0001; % weight decay parameter
beta = ; % weight of sparsity penalty term %%======================================================================
%% STEP : Implement sampleIMAGES
% After implementing sampleIMAGES, the display_network command should display a random sample of patches from the dataset
%从图像中提取图像块儿，每一个提取到的图像块儿存放在patches的每一列中
patches = sampleIMAGES;
%随机提取patches中的200列，然后显示这200列所对应的图像
IMG=patches(:,randi(size(patches,),,));
display_network(IMG,); %%======================================================================
%% STEP and STEP ：Implement sparseAutoencoderCost and Gradient Checking
checkSparseAutoencoderCost() %%======================================================================
%% STEP : After verifying that your implementation of % Randomly initialize the parameters
theta = initializeParameters(hiddenSize, visibleSize); % Use minFunc to minimize the function
addpath minFunc/
options.Method = 'lbfgs'; % Here, we use L-BFGS to optimize our cost function
% Generally, for minFunc to work, you need a function pointer with two outputs: the function value and the gradient.
% In our problem, sparseAutoencoderCost.m satisfies this.
options.maxIter = ; % Maximum number of iterations of L-BFGS to run
options.display = 'on'; % opttheta是整个神经网络的权值和偏执项构成的向量
[opttheta, cost] = minFunc( @(p) sparseAutoencoderCost(p, ...
visibleSize, hiddenSize, ...
lambda, sparsityParam, ...
beta, patches), ...
theta, options); %%======================================================================
%% STEP : Visualization
W1 = reshape(opttheta(:hiddenSize*visibleSize), hiddenSize, visibleSize);%第一层的权值矩阵
display_network(W1', 12); print -djpeg weights.jpg % save the visualization to a file
checkSparseAutoencoderCost.m
%% 该函数主要目的是检验SparseAutoencoderCost函数是否正确
function checkSparseAutoencoderCost() %% 产生一个稀疏自编码网络（可以与主程序相同，也可以重新产生）
visibleSize = *; % number of input units
hiddenSize = ; % number of hidden units
sparsityParam = 0.01; % desired average activation of the hidden units.
% (This was denoted by the Greek alphabet rho, which looks like a lower-case "p", in the lecture notes).
lambda = 0.0001; % weight decay parameter
beta = ; % weight of sparsity penalty term patches = sampleIMAGES; % Obtain random parameters theta
theta = initializeParameters(hiddenSize, visibleSize); %% 计算代价函数和梯度
[cost, grad] = sparseAutoencoderCost(theta, visibleSize, hiddenSize, lambda, ...
sparsityParam, beta, patches(:,:)); %% 利用近似方法计算梯度（要调用自编码器的代价函数计算程序）
numgrad = computeNumericalGradient( @(x) sparseAutoencoderCost(x, visibleSize, ...
hiddenSize, lambda, ...
sparsityParam, beta, ...
patches(:,:)), theta); %% 比较cost函数计算得到的梯度和由近似得到的梯度之
% Use this to visually compare the gradients side by side
disp([numgrad grad]); % Compare numerically computed gradients with the ones obtained from backpropagation
diff = norm(numgrad-grad)/norm(numgrad+grad);
disp(diff); % Should be small. In our implementation, these values are usually less than 1e-. end
sparseAutoencoderCost.m
%% 计算网络的代价函数和梯度
function [cost,grad] = sparseAutoencoderCost(theta, visibleSize, hiddenSize, ...
lambda, sparsityParam, beta, data) % visibleSize: the number of input units (probably )
% hiddenSize: the number of hidden units (probably )
% lambda: weight decay parameter
% sparsityParam: The desired average activation for the hidden units (denoted in the lecture
% notes by the greek alphabet rho, which looks like a lower-case "p").
% beta: weight of sparsity penalty term
% data: Our 64x10000 matrix containing the training data. So, data(:,i) is the i-th training example. % The input theta is a vector (because minFunc expects the parameters to be a vector).
% We first convert theta to the (W1, W2, b1, b2) matrix/vector format, so that this
% follows the notation convention of the lecture notes. W1 = reshape(theta(:hiddenSize*visibleSize), hiddenSize, visibleSize);
W2 = reshape(theta(hiddenSize*visibleSize+:*hiddenSize*visibleSize), visibleSize, hiddenSize);
b1 = theta(*hiddenSize*visibleSize+:*hiddenSize*visibleSize+hiddenSize);
b2 = theta(*hiddenSize*visibleSize+hiddenSize+:end); % Cost and gradient variables (your code needs to compute these values).
% Here, we initialize them to zeros.
cost = ; m=size(data,); %% ---------- YOUR CODE HERE --------------------------------------
% Instructions: Compute the cost/optimization objective J_sparse(W,b) for the Sparse Autoencoder,
% and the corresponding gradients W1grad, W2grad, b1grad, b2grad.
%
% W1grad, W2grad, b1grad and b2grad should be computed using backpropagation.
% Note that W1grad has the same dimensions as W1, b1grad has the same dimensions
% as b1, etc. Your code should set W1grad to be the partial derivative of J_sparse(W,b) with
% respect to W1. I.e., W1grad(i,j) should be the partial derivative of J_sparse(W,b)
% with respect to the input parameter W1(i,j). Thus, W1grad should be equal to the term
% [(/m) \Delta W^{()} + \lambda W^{()}] in the last block of pseudo-code in Section 2.2
% of the lecture notes (and similarly for W2grad, b1grad, b2grad).
%
% Stated differently, if we were using batch gradient descent to optimize the parameters,
% the gradient descent update to W1 would be W1 := W1 - alpha * W1grad, and similarly for W2, b1, b2.
% %% 前向传播算法
a1=data;
z2=bsxfun(@plus,W1*a1,b1);
a2=sigmoid(z2);
z3=bsxfun(@plus,W2*a2,b2);
a3=sigmoid(z3); %% 计算网络误差
% 误差项J1=所有样本代价函数均值
y=data; % 网络的理想输出值
Ei=sum((a3-y).^)/; %每一个样本的代价函数
J1=sum(Ei)/m;
% 正则化项J2=所有权值项平方和
J2=sum(W1(:).^)+sum(W2(:).^);
% 稀疏项J3=所有隐藏层的神经元相对熵之和
rho_hat=sum(a2,)/m;
KL=sum(sparsityParam*log(sparsityParam./rho_hat)+...
(-sparsityParam)*log((-sparsityParam)./(-rho_hat)));
J3=KL;
% 网络的代价函数
cost=J1+lambda*J2/+beta*J3; %% 反向传播算法计算各层敏感度delta
delta3=-(data-a3).*dsigmoid(z3);
spare_delta=beta*(-sparsityParam./rho_hat+(-sparsityParam)./(-rho_hat));
delta2=bsxfun(@plus,W2'*delta3,spare_delta).*dsigmoid(z2); % 这里加入了稀疏项 %% 计算代价函数对各层权值和偏执项的梯度
W1grad=delta2*a1'/m+lambda*W1;
W2grad=delta3*a2'/m+lambda*W2;
b1grad=sum(delta2,)/m;
b2grad=sum(delta3,)/m; %-------------------------------------------------------------------
% After computing the cost and gradient, we will convert the gradients back
% to a vector format (suitable for minFunc). Specifically, we will unroll
% your gradient matrices into a vector. grad = [W1grad(:) ; W2grad(:) ; b1grad(:) ; b2grad(:)];
%
end %-------------------------------------------------------------------
% Here's an implementation of the sigmoid function, which you may find useful
% in your computation of the costs and the gradients. This inputs a (row or
% column) vector (say (z1, z2, z3)) and returns (f(z1), f(z2), f(z3)). function sigm = sigmoid(x)
sigm = ./ ( + exp(-x));
end %% 求解sigmoid函数的导数(这里的计算公式一定要注意啊，出过错)
function dsigm = dsigmoid(x)
sigx = sigmoid(x);
dsigm=sigx.*(-sigx);
end
梯度检验函数见另一篇博文