eprintid: 10192104 rev_number: 19 eprint_status: archive userid: 699 dir: disk0/10/19/21/04 datestamp: 2024-07-08 19:20:01 lastmod: 2024-10-31 09:25:57 status_changed: 2024-07-08 19:20:01 type: thesis metadata_visibility: show sword_depositor: 699 creators_name: Liu, Linqing title: Towards Generalized Open Domain Question Answering Systems ispublished: unpub divisions: UCL divisions: B04 divisions: C05 divisions: F48 note: Copyright © The Author 2024. Original content in this thesis is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) Licence (https://creativecommons.org/licenses/by-nc/4.0/). Any third-party copyright material present remains the property of its respective owner(s) and is licensed under its existing terms. Access may initially be restricted at the author’s request. abstract: Generalization remains a paramount yet unresolved challenge for open-domain question answering (ODQA) systems, impeding their capacity to adeptly handle novel queries and responses beyond the confines of their training data. This thesis conducts a comprehensive exploration of ODQA generalization. We commence with a meticulous investigation into the underlying challenges. Drawing upon studies on systematic generalization, we introduce and annotate questions according to three categories that measure different levels and kinds of generalization: training set overlap, compositional generalization and novel-entity generalization. When evaluating six popular parametric and non-parametric models, we find non-parametric models demonstrate proficiency with novel entities but encounter difficulties with compositional generalization. Noteworthy correlations emerge, such as a positive association between question pattern frequency and test accuracy, juxtaposed with a strong negative correlation between entity frequency and test accuracy, attributable to closely related distractors. Factors influencing generalization include cascading errors originating from the retrieval component, question pattern frequency, and entity prevalence. Building on these insights, the focus pivots towards the enhancement of passage retrieval. We propose a novel contextual clue sampling strategy using language models to address the vocabulary mismatch challenge in lexical retrieval for ODQA. This two-step method, comprising filtering and fusion, generates a diverse set of query expansion terms, yielding retrieval accuracy similar to dense methods while notably reducing the index size. The subsequent phase concentrates on refining reader models in ODQA through flat minima optimization techniques, incorporating Stochastic Weight Averaging (SWA) and Sharpness Aware Minimization (SAM). Rigorous benchmarking under- scores the impact of dataset characteristics and model architecture on optimizer effectiveness, with SAM particularly excelling in Natural Language Processing tasks. The combination of SWA and SAM yields additional gains, underscoring the pivotal role of flatter minimizers in fostering enhanced generalization for reader models in ODQA. date: 2024-05-28 date_type: published oa_status: green full_text_type: other thesis_class: doctoral_open thesis_award: Ph.D language: eng primo: open primo_central: open_green verified: verified_manual elements_id: 2274027 lyricists_name: Liu, Linqing lyricists_id: LLIUC79 actors_name: Liu, Linqing actors_id: LLIUC79 actors_role: owner full_text_status: public pages: 151 institution: UCL (University College London) department: Computer Science thesis_type: Doctoral citation: Liu, Linqing; (2024) Towards Generalized Open Domain Question Answering Systems. Doctoral thesis (Ph.D), UCL (University College London). Green open access document_url: https://discovery.ucl.ac.uk/id/eprint/10192104/1/Liu_10192104_thesis_revised.pdf