Abstract In this paper, a novel idea for optimization of image segmentation algorithms based on algorithm evaluation is proposed. According to this idea, a segmentation expert system is designed and realized. Experimental results show that such an idea is feasible and such a system is workable. This also provides an effective way for the optimization of various image processing and analysis algorithms.