Meta-DPSTL: meta learning-based differentially private self-taught learning

Self-taught learning models are successfully applied to improve the target model's performance in different low-resource environments. In this setting, features are learned using unlabeled instances in the source domain; thereafter, the learned feature representations are transferred to the target domain for the supervised classification task. Two important challenges in this setup include learning efficient feature representations in the source domain and securing instance privacy against attacks carried out during knowledge transfer from the source to the target domain. We propose Meta-DPSTL\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Meta-DPSTL$$\end{document}, a novel Meta Differentially Private Self-Taught Learning model to overcome these challenges. The proposed approach implements self-taught learning in the meta-learning-based framework; training of meta-learner and base-learner proceeds episodically and is equivalent to estimating source and target domain parameters, respectively. Further, to protect the sensitive source data from a potential attacker, differential privacy is added to the meta-parameters learned in an episode before they are passed to the target domain to train the base-learner. To measure the immunity of the proposed model to an inversion attack, we propose a novel Relative Reconstruction Distance (RRD) metric. Lastly, an inversion attack is carried out on the meta-parameters; empirical results obtained on the handwritten digits recognition dataset, COVID-19\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$COVID-19$$\end{document}X-Ray\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$X-Ray$$\end{document} Radiography dataset, and COVID-19\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$COVID-19$$\end{document} Lung CT Scans dataset confirm the utility of meta-learning-based self-taught features in obtaining richer feature representations and hence, providing base-learners that are more generalizable. Relative reconstruction distance values computed on these datasets show that the differentially-private meta-parameters are robust to inversion attacks. Consequently, the proposed approach may be used in applications where the privacy requirements of sensitive source domain datasets are paramount.