Abstract: 

With the advent of deep learning and neural methods, NLP research over the last decade has shifted from feature engineering to model engineering, primarily focusing on inventing new architectures for NLP problems. Two other related factors that are getting more attention only recently are: (i) which objectives (or tasks) to optimize, and (ii) how to better use the available data; referred to as task engineering and data engineering, respectively. In this talk, I will present our recent work along these three dimensions. In particular, I will first present novel neural architectures for parsing texts into hierarchical structures and efficient parallel encoding of such structures for better language understanding and generation. I will then present a new objective for natural language generation (NLG) tasks that aims to mitigate the degeneration issues prevalent in neural generation models. Finally, I will present effective data augmentation methods for supervised and unsupervised machine translation and other cross-lingual tasks. With empirical results, I will argue that while model engineering is crucial to the advancement of the field, the other two factors are more important to build robust NLP systems.

Background Knowledge
ML/NLP

Bio: 

Shafiq Joty is an Asst. Prof. in the School of Computer Science and Engineering (SCSE) at NTU, where he leads the NTU-NLP group. He is also a senior manager of NLP research and a founding member at Salesforce AI Research Asia. His work has primarily focused on developing language analysis tools (e.g., syntactic parsers, NER, discourse parser, coherence models) and downstream NLP applications including machine translation, question answering, text summarization, image/video captioning and visual question answering. A significant part of his current research focuses on multilingual processing and robustness of NLP models. His work has mostly relied on deep learning for better representation of the input text and on probabilistic graphical models and reinforcement learning for capturing dependencies in the output. He served (or will serve) as a (senior) area chair for ACL'19-21, EMNLP'19,21 and NAACL’21, EACL’21, and a senior program committee member for AAAI’21 and IJCAI'19. He gave tutorials at ACL-2019 and ICDM-2018. He was an associate editor for ACM Transactions on Asian and Low Resource Language Processing. He has published more than 95 papers in top-tier NLP/AI conferences and journals including ACL, EMNLP, NAACL, NeurIPS, ICML, ICLR, CVPR, ECCV, ICCV, CL and JAIR.