publications

2020

1. Unsupervised Adaptation of Question Answering Systems via Generative Self-training

   Rennie, Steven, Marcheret, Etienne, Mallinar, Neil, Nahamoo, David, and Goel, Vaibhava

   In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) Nov 2020

   Abs

   BERT-era question answering systems have recently achieved impressive performance on several question-answering (QA) tasks. These systems are based on representations that have been pre-trained on self-supervised tasks such as word masking and sentence entailment, using massive amounts of data. Nevertheless, additional pre-training closer to the end-task, such as training on synthetic QA pairs, has been shown to improve performance. While recent work has considered augmenting labelled data and leveraging large unlabelled datasets to generate synthetic QA data, directly adapting to target data has received little attention. In this paper we investigate the iterative generation of synthetic QA pairs as a way to realize unsupervised self adaptation. Motivated by the success of the roundtrip consistency method for filtering generated QA pairs, we present iterative generalizations of the approach, which maximize an approximation of a lower bound on the probability of the adaptation data. By adapting on synthetic QA pairs generated on the target data, our method is able to improve QA systems significantly, using an order of magnitude less synthetic data and training computation than existing augmentation approaches.

2. Iterative Data Programming for Expanding Text Classification Corpora

   Mallinar, Neil, Shah, Abhishek, Ho, Tin Kam, Ugrani, Rajendra, and Gupta, Ayush

   Proceedings of the AAAI Conference on Artificial Intelligence Feb 2020

   Abs

   Real-world text classification tasks often require many labeled training examples that are expensive to obtain. Recent advancements in machine teaching, specifically the data programming paradigm, facilitate the creation of training data sets quickly via a general framework for building weak models, also known as labeling functions, and denoising them through ensemble learning techniques. We present a fast, simple data programming method for augmenting text data sets by generating neighborhood-based weak models with minimal supervision. Furthermore, our method employs an iterative procedure to identify sparsely distributed examples from large volumes of unlabeled data. The iterative data programming techniques improve newer weak models as more labeled data is confirmed with human-in-loop. We show empirical results on sentence classification tasks, including those from a task of improving intent recognition in conversational agents.

2019

1. Big-Little Net: An Efficient Multi-Scale Feature Representation for Visual and Speech Recognition

   Chen, Chun-Fu, Fan, Quanfu, Mallinar, Neil, Sercu, Tom, and Feris, Rogerio

   International Conference on Learning Representations 2019

   Abs

   In this paper, we propose a novel Convolutional Neural Network (CNN) architecture for learning multi-scale feature representations with good tradeoffs between speed and accuracy. This is achieved by using a multi-branch network, which has different computational complexity at different branches. Through frequent merging of features from branches at distinct scales, our model obtains multi-scale features while using less computation. The proposed approach demonstrates improvement of model efficiency and performance on both object recognition and speech recognition tasks,using popular architectures including ResNet and ResNeXt. For object recognition, our approach reduces computation by 33% on object recognition while improving accuracy with 0.9%. Furthermore, our model surpasses state-of-the-art CNN acceleration approaches by a large margin in accuracy and FLOPs reduction. On the task of speech recognition, our proposed multi-scale CNNs save 30% FLOPs with slightly better word error rates, showing good generalization across domains. The codes are available at https://github.com/IBM/BigLittleNet

2. Multi-Frame Cross-Entropy Training for Convolutional Neural Networks in Speech Recognition

   Sercu, Tom, and Mallinar, Neil

   arXiv preprint arXiv:1907.13121 2019

   Abs

   We introduce Multi-Frame Cross-Entropy training (MFCE) for convolutional neural network acoustic models. Recognizing that similar to RNNs, CNNs are in nature sequence models that take variable length inputs, we propose to take as input to the CNN a part of an utterance long enough that multiple labels are predicted at once, therefore getting cross-entropy loss signal from multiple adjacent frames. This increases the amount of label information drastically for small marginal computational cost. We show large WER improvements on hub5 and rt02 after training on the 2000-hour Switchboard benchmark.

2018

1. Bootstrapping Conversational Agents With Weak Supervision

   Mallinar, Neil, Shah, Abhishek, Ugrani, Rajendra, Gupta, Ayush, Gurusankar, Manikandan, Ho, Tin Kam, Liao, Q. Vera, Zhang, Yunfeng, Bellamy, Rachel K. E., Yates, Robert, Desmarais, Chris, and McGregor, Blake

   Proceedings of the AAAI Conference on Artificial Intelligence Dec 2018

   Abs

   Many conversational agents in the market today follow a standard bot development framework which requires training intent classifiers to recognize user input. The need to create a proper set of training examples is often the bottleneck in the development process. In many occasions agent developers have access to historical chat logs that can provide a good quantity as well as coverage of training examples. However, the cost of labeling them with tens to hundreds of intents often prohibits taking full advantage of these chat logs. In this paper, we present a framework called }textit{search, label, and propagate} (SLP) for bootstrapping intents from existing chat logs using weak supervision. The framework reduces hours to days of labeling effort down to minutes of work by using a search engine to find examples, then relies on a data programming approach to automatically expand the labels. We report on a user study that shows positive user feedback for this new approach to build conversational agents, and demonstrates the effectiveness of using data programming for auto-labeling. While the system is developed for training conversational agents, the framework has broader application in significantly reducing labeling effort for training text classifiers.

2. Deep Canonically Correlated LSTMs

   Mallinar, Neil, and Rosset, Corbin

   The Johns Hopkins University Bachelors Thesis Jan 2018

   Abs

   We examine Deep Canonically Correlated LSTMs as a way to learn nonlinear transformations of variable length sequences and embed them into a correlated, fixed dimensional space. We use LSTMs to transform multi-view time-series data non-linearly while learning temporal relationships within the data. We then perform correlation analysis on the outputs of these neural networks to find a correlated subspace through which we get our final representation via projection. This work follows from previous work done on Deep Canonical Correlation (DCCA), in which deep feed-forward neural networks were used to learn nonlinear transformations of data while maximizing correlation.

2017

1. Probabilistic Cross-Identification of Galaxies with Realistic Clustering

   Mallinar, Neil, Budavari, Tamas, and Lemson, Gerard

   Astronomy and Computing Jul 2017

   Abs

   Probabilistic cross-identification has been successfully applied to a number of problems in astronomy from matching simple point sources to associating stars with unknown proper motions and even radio observations with realistic morphology. Here we study the Bayes factor for clustered objects and focus in particular on galaxies to assess the effect of typical angular correlations. Numerical calculations provide the modified relationship, which (as expected) suppresses the evidence for the associations at the shortest separations where the 2-point auto-correlation function is large. Ultimately this means that the matching probability drops at somewhat shorter scales than in previous models.