about Out-Of-Distribution(OOD) Problem
This page collects papers related to Out-Of-Distribution(OOD) problem to help understand problems of detecting if a test data is misclassified or Out-Of-Distribution Problem.
related project
Not yet
What is Out-Of-Distribution problem?
Out-Of-Distribution problem, also known as OOD. See at unexpected inputs or OOD VS anomaly detection. Cause: The basic principle of neural networks is “distribution”. A problem caused by not expecting the test data to come from a distribution different from that provided by the train data. After training the network with the in-distribution data set, the problem of finding abnormal samples in the test stage is called Out-of-distribution Detection.
How to solve OOD problems? (Not perfectly solved)
- using BNN(Bayesian Neural network).
- read answered Mar21, 2020 at 23:11 and BNN paper below.
- Create a “other” class and put new data into it.
- read ai.stackexchange.com
- using one-class classification
- learning only one class, and set threshold value high to classificate learned class and other. One-class classification is hard to learning because of set threshold value. If threshold value are too high, then it might be overfitting. If threshold value are too low, then hard to classification. Also, This method only do binary classification. In class, or not in class. See at one-class classification
Which paper recommand to read? (wrote as ISO 690 format)
about BNN paper
- BLUNDELL, Charles, et al. Weight uncertainty in neural network. In: International conference on machine learning. PMLR, 2015. p. 1613-1622.
Frist paper that writing OOD problem.
Well established baseline for Out-of-distribution Detection problem. To this end, an experimental protocol and three evaluation indicators are proposed.
- HENDRYCKS, Dan; GIMPEL, Kevin. A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136, 2016.
- Out Of Distribution (OOD), the first method in the paper to filter out out-of-distribution datasets. Set the threshold value and find the largest softmax value among the classes. After that, if (largest softmax value) < (threshold value), the simplest way to think of it as OOD. However, this method only had a slight effect, and was not suitable for two classes.
Solving OOD problems using auxiliary dataset with GAN, adding noise, etc.
- HENDRYCKS, Dan; MAZEIKA, Mantas; DIETTERICH, Thomas. Deep anomaly detection with outlier exposure. arXiv preprint arXiv:1812.04606, 2018.
Other recommend paper.
- LIANG, Shiyu; LI, Yixuan; SRIKANT, Rayadurgam. Enhancing the reliability of out-of-distribution image detection in neural networks. arXiv preprint arXiv:1706.02690, 2017.
review paper
Not Yet
- I don’t read OOD problems review paper yet.