Abstract:
The naive Bayes classifier is valued for its simplicity, efficiency, and interpretability, but its conditional independence assumption causes redundant features to be counted repeatedly, leading to overconfident predictions and degraded accuracy. We propose Adaptive Bayesian Dependence Naive Bayes (ABD-NB), a method that preserves the standard naive Bayes factorisation while making the independence assumption adaptive through feature-specific weights wi ∈ (0, 1]. These weights decrease according to each feature’s estimated within-class dependence on the others. The proposed weighting scheme combines three components: (i) class-conditional dependence estimation using rank correlation (or alternative dependence measures) with asymptotic soft-thresholding for variance control; (ii) a harmonic graph-based weighting rule that provably assigns equal fractional weights to duplicated features, recovering the exact Bayes posterior under duplication; and (iii) spectral rescaling based on the eigenvalues of the dependence matrix to match the effective number of independent features. The correction parameter γ is selected by internal validation and includes γ = 0, ensuring graceful degradation to standard naive Bayes. We establish theoretical guarantees, including duplicate recovery, weight consistency, and provable reduction of naive Bayes overconfidence under correlated features. Experiments on 15 real and synthetic datasets against 10 baseline classifiers show that ABD-NB improves accuracy by up to 17.3 percentage points in highly redundant settings, significantly reduces logloss (Wilcoxon p = 0.008), achieves a higher average accuracy rank than random forests and logistic regression, and never performs more than 0.1 accuracy points below naive Bayes. A dynamic extension further adapts to evolving dependence structures in data streams while maintaining superior predictive performance.