AUC vs F1 in Imbalanced data

เพราะ imbalanced data คือมี #TP ต่างจาก #TN มาก

F1 ไม่ดู TN แต่ AUC คิด TN ด้วย ดังนั้นควรใช้ F1 เพื่อเลี่ยง TN ที่น้อยไปจนถ้าเอามาพิจารณาร่วมด้วยผลลัพธ์จะเพี้ยน

Geometric Mean of TPR and TNR

Used in binary classification

G-mean = Sqrt of TPR*TNR

AUC is an error or performance metric very useful for replacing accuracy in binary classification with strong class imbalance. https://thedigitalskye.com/2021/04/19/6-useful-metrics-to-evaluate-binary-classification-models/

Geometric mean in general isn’t an error or performance metric, but is just an alternative to an arithmetic mean that’s robust to different normalization schemes.

Metrics in binary classification https://neptune.ai/blog/evaluation-metrics-binary-classification

Metric in medical testing

ควรวัดค่า negative likelihood ratio (LR-) = fnr/tnr ด้วย เพื่อให้ความสำคัญกับค่า False negative rate i.e. Type II error besides Type I error (FPR)

cf.https://medium.com/@saurabhdhandeblog/confusion-matrix-explained-calculating-accuracy-tpr-fpr-tnr-precision-and-prevalence-87557fe8714d

Sensitivity and specificity

 https://en.m.wikipedia.org/wiki/Sensitivity_and_specificity

Expectation values and Variance and Covariance

The expected value should be regarded as the average value. When X is a discrete random variable, then the expected value of X is precisely the mean of the corresponding data. 

The variance should be regarded as (something like) the average of the difference of the actual values from the average. Variance is the expectation of the squared deviation of a random variable from its population mean or sample mean. Variance is a measure of dispersion, meaning it is a measure of how far a set of numbers is spread out from their average value. Variance is squared STD.

https://math.berkeley.edu/~scanlon/m16bs04/ln/16b2lec30.pdf

Covariance is correlation times STD of data point set 1 times STD of data point set 2. Both data sets are to be compared for their relationship. Covariance measures the joint variability of two random variables, indicating the direction of their linear relationship. A positive value means variables tend to move in the same direction, while a negative value indicates they move in opposite directions. Covariance measures the direction of the relationship between two variables. So covariance is hard to interpret directly. Correlation is the standardized version of covariance. It measures both direction and strength of a linear relationship.

Grid search finds optimal parameters values

 https://medium.com/fintechexplained/what-is-grid-search-c01fe886ef0a#:~:text=Grid%20search%20is%20a%20tuning,us%20time%2C%20effort%20and%20resources

Techniques and pitfalls for ML training with small data sets

 https://trustbit.tech/blog/2021/06/30/techniques-and-pitfalls-for-ml-training-with-small-data-sets