Word Embeddings are the bread and butter of Natural Language Processing. But are they free from inherent bias? Imagine googling “cool programmer t-shirts” and Google responds with only male-form tees. With BERT, which has shown signs of gender bias, being incorporated in Google Search, one doesn’t need to imagine. Keeping the “intelligence” that defines AI, inherently bias-free is imperative as we continuously see increased usage of these systems in our daily lives.
These days BERT, ELMo and Ernie reminds one of pre-trained generative models rather than Sesame Street characters, such has been their hegemony over the Natural Language Processing landscape. These models can serve as general purpose encoders, and can even perform some tasks like text classification without requiring further modification. However, limited research has been conducted on the reverse-case, exploiting these models for use as general purpose decoders. This article is a summary of this paper by researchers at New York University which tries to ascertain exactly this, whether these models can recover an arbitrary sentence from its encoded representation.
Machine Learning and Artificial Intelligence are currently driving innovation in the field of Computer Science and they are being applied on a multitude of fields across disciplines. However, traditional ML models can be still be broadly categorized into solutions of two types of problems.
Full-time Software Engineer, Machine Learning enthusiast and foodie.