In mass fruit production and packaging, there is importance of sorting and segregating the fruits based on various specifications. Such fast sorting system helps farmer or industry personnel with minimum human involvement. In this paper, the auto sorting system for fruits is proposed. It identifies the ripen fruits moving on conveyor belt and accumulates in separate package box. Similarly, the system identifies and sorts fruits of required size in boxes. It also maintains the record of bulk of fruits in each pack. The system is being implemented using image processing techniques on Matlab software to check size and ripeness of the fruit and utilized for sorting mechanism.