To solve the problem of blind source separation for chaotic signals, an improved blind separation algorithm is proposed. A function is constructed by signal separation evaluation index, which adaptively updates the step size and momentum factor, then substitutes the obtained variable step-size function into blind source separation algorithm and introduces the adaptive momentum item. Different from most algorithms which can not estimate the mixing matrix, the proposed algorithm estimates iteratively the mixing matrix by the variable step function, then the global matrix and the estimated evaluation can be obtained on which step and momentum factor are iteratively updated. Finally, the separation matrix is obtained. Simulations show that the algorithm is effective to adjust the step and momentum factor based on the estimated evaluation index constructor. In stationary and non-stationary environments, the algorithm has faster convergence speed and lower steady error for separating the mixed chaotic signals. When mixing color noise, the proposed algorithm is better than that of the traditional algorithm, which shows that the proposed algorithm has certain application value to the chaotic signal blind source separation processing.