One significant advantage of rate selection algorithms based on bandwidth estimation is the high bandwidth utilization rate. They are, however, vulnerable to network bandwidth fluctuations, leading to appearance of rate instantaneous peak value and hence wasting unnecessary bandwidth consumption. To tackle the problem above, this paper proposes a novel rate selection algorithm based on the contrast sensitivity of human eyes, where in the client eyes cutoff spatial frequency under the current viewing conditions is calculated by using the human contrast sensitivity model. The algorithm selects the rate of video fragment which has the minimum absolute difference value to the spatial frequency computed, stored in server as the target rate. Compared with those methods for calculating the target rate based on bandwidth estimation and testing target rate in different angles, the proposed method gets the ladder diagrams of rate calculation of both methods. Experimental results demonstrate that the proposed algorithm is able to save a considerable amount of bandwidth without the loss of video quality, with viewing angle from 5° to 15°.