Most fruit classification studies rely on color-based features, but shape-based analysis provides a promising alternative for distinguishing subtle variations within the same variety. Tomato shape classification is challenging due to irregular contours, variable imaging conditions, and difficulty in extracting consistent geometric features. In this study, we propose an efficient and structured workflow to address these challenges through contour-based analysis. The process begins with the application of a Mask Region-based Convolutional Neural Network (Mask R-CNN) model to accurately isolate tomatoes from the background. Subsequently, the segmented tomatoes are extracted and encoded using Elliptic Fourier Descriptors (EFDs) to capture detailed shape characteristics. These features are used to train a range of machine learning models, including Support Vector Machine (SVM), Random Forest, One-Dimensional Convolutional Neural Network (1D-CNN), and Bidirectional Encoder Representations from Transformers (BERT). Experimental results observe that the Random Forest model achieved the highest accuracy of 79.4%. This approach offers a robust, interpretable, and quantitative framework for tomato shape classification, reducing manual labor and supporting practical agricultural applications.