GENERATING META-SUBNETS FOR EFFICIENT MODEL GENERALIZATION IN A MULTI-DISTRIBUTION SCENARIO

Technical limitation of conventional Gradient-Based Meta Learners is their inability to adapt to scenarios where input tasks are sampled from multiple distributions. Training multiple models, with one model per distribution adds to the training time owing to increased compute. A method and system for generating meta-subnets for efficient model generalization in a multi-distribution scenario using Binary Mask Perceptron (BMP) technique or a Multi-modal Meta Supermasks (MMSUP) technique is provided. The BMP utilizes an adaptor which determines a binary mask, thus training only those layers which are relevant for given input distribution, leading to improved training accuracy in a cross-domain scenario. The MMSUP, further determines relevant subnets for each input distribution, thus, generalizing well as compared to standard MAML. The BMP and MMSUP, beat Multi-MAML in terms of training time as they train a single model on multiple distributions as opposed to Multi-MAML which trains multiple models.