Theoretical Analysis of Direct LDA in Small Sample Size Problem
Zhao Wu-feng①②; Shen Hai-bin①; Yan Xiao-lang①
①Institute of VLSI Design, Zhejiang University, Hangzhou 310027,China; ②Department of Information Science & Electronic Engineering, Zhejiang University, Hangzhou 310027, China
Abstract：Direct LDA (DLDA) is an extension of Linear Discriminant Analysis (LDA) to deal with the small sample size problem, which is previously claimed to take advantage of all the information, both within and outside of the within-class scatter's null space. However, a lot of counter-examples show that this is not the case. In order to better understand the characteristics of DLDA, this paper presents its theoretical analysis and concludes that: DLDA based on the traditional Fisher criterion nearly does not make use of the information inside the null space, thus some discriminative information may be lost; while one based on other variants of Fisher criterion is equivalent to null-space LDA and orthogonal LDA under the orthogonal constraints among discriminant vectors and a mild condition which holds in many applications involving high-dimensional data. The comparative results on the face database, ORL and YALE, also consistent with the theory analysis.
2009, Vol. 31 
