Miss Haiqi Dong1, Professor Amanda S Barnard1, Dr Amanda J Parker1
1School of Computing, The Australian National University, Acton, Australia
Machine learning tasks require a large amount of labeled data to help build a predictive model. However, a large amount of labeled data for many fields means a high annotation cost. Active learning (AL) is a data selection algorithm that can help label data instances selectively so that the model can achieve better generalization ability under the circumstance of limiting the sample size. Traditional model-driven AL methods apply the handcraft acquisition functions based on different criteria, which is challenging to be adaptive to all the data in practice. The data-driven AL methods have better domain adaption capability to select the instances adaptively based on the data characteristics or the model states. However, few data-driven methods involve small sample data because it may suffer from the cold-start problem when there is not enough experience to guide the subsequent sampling.
Our approach focuses on regression problems on small samples, using meta-learning as a sampling mechanism for AL. We use model state (the gradient norm of parameters at each hidden layer) and data diversity measures as meta-learning features to predict which candidate samples can bring about the lower generalization loss. Experiments show that our method outperforms random sampling and classical model-driven methods among different data sets. Our method is not affected by the cold start problem on some small sample data. In some other data sets, compared with classical AL methods, our method has a slight cold-start problem at the beginning but has a higher model performance in the later iterations.
Biography:
Haiqi Dong is currently a PhD candidate at the Australian National University. Her research areas are Active Learning and applied machine learning. She aims to apply machine learning methods to guide scientific experiments in practice. She was awarded her Bachelor’s degrees at Southwest University in China in Software Engineering and Deakin University in Information Technology in 2019. She was awarded her Master’s degree at the Australian National University in 2021.