Fan Bai (JHU) “From General Intelligence to Task Expertise: Infusing Task-Specific Insights in the Era of Large Language Models”
3400 N CHARLES ST
Baltimore
MD 21218
Abstract
In this talk, we will discuss two of our recent works aimed at enhancing the task-specific performance of Large Language Models (LLMs). In the first part, we explore the generation of high-quality synthetic data for Clinical Question Answering in a zero-shot setting. We reveal that naïve prompting tends to produce overly simplistic examples, and promoting syntactic diversity in the generated data fails to address this issue. We propose a more effective strategy: concretizing challenging examples and guiding LLMs towards targeted generation, which boosts the performance of the resulting models significantly. In the second part, we focus on Named Entity Recognition (NER), a fundamental NLP task where the in-context learning performance of state-of-the-art LLMs still lags behind that of fine-tuned models. We show that current sentence embedding-based methods select suboptimal examples for token-level tasks like NER, resulting in ungrounded predictions. To overcome this, we propose a novel method that use token statistics to help LLMs better leverage the training examples, leading to improved in-context learning performance.