Analysis Of The Effect Of Gridsearchcv Hyperparameter Tuning Strategy With Variations In Scoring, Test Size, And Random State On Water Quality Classification Performance Using SVM RBF Kernel

Abstract

The choice of scoring metric in hyperparameter tuning using GridSearchCV is often overlooked, even though this decision directly affects the resulting model's behavior. This study analyzes the impact of scoring metric variations (accuracy, precision, recall), test size (70/30 and 80/20), and random state (0, 21, 42) on the performance of a Support Vector Machine (SVM) with RBF kernel for water quality classification. The dataset used is a combination of two public datasets from Kaggle totaling 4,259 samples, using three parameters: pH, conductivity, and turbidity, labeled based on WHO standards and Indonesian Ministry of Health Regulation No. 32 of 2017. The experiment was designed across 6 scenarios combined with three scoring metrics, resulting in 18 test configurations. Results show that accuracy and precision scoring produce excellent and consistent performance with accuracy ranging from 0.985–0.996 and F1-score of 0.98–1.00 across all scenarios. In contrast, recall scoring causes systematic model degeneration: the model consistently predicts all samples as the positive class (TN=0), resulting in accuracy of only approximately 0.50. This phenomenon occurs because GridSearchCV exploits the mathematical definition of recall by selecting parameters that produce a trivial classifier. These findings demonstrate that on a balanced dataset, using recall as the sole scoring metric is counterproductive, and F1-score or accuracy is more recommended as a safe metric for model optimization.