BEGIN:VCALENDAR VERSION:2.0 PRODID:-//128.220.36.25//NONSGML kigkonsult.se iCalcreator 2.26.9// CALSCALE:GREGORIAN METHOD:PUBLISH X-FROM-URL:https://www.clsp.jhu.edu X-WR-TIMEZONE:America/New_York BEGIN:VTIMEZONE TZID:America/New_York X-LIC-LOCATION:America/New_York BEGIN:STANDARD DTSTART:20231105T020000 TZOFFSETFROM:-0400 TZOFFSETTO:-0500 RDATE:20241103T020000 TZNAME:EST END:STANDARD BEGIN:DAYLIGHT DTSTART:20240310T020000 TZOFFSETFROM:-0500 TZOFFSETTO:-0400 RDATE:20250309T020000 TZNAME:EDT END:DAYLIGHT END:VTIMEZONE BEGIN:VEVENT UID:ai1ec-21270@www.clsp.jhu.edu DTSTAMP:20240329T131608Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION:Abstract\nSocial media allows researchers to track societal and cultural changes over time based on language analysis tools. Many of thes e tools rely on statistical algorithms which need to be tuned to specific types of language. Recent studies have questioned the robustness of longit udinal analyses based on statistical methods due to issues of temporal bia s and semantic shift. To what extent are changes in semantics over time af fecting the reliability of longitudinal analyses? We examine this question through a case study: understanding shifts in mental health during the co urse of the COVID-19 pandemic. We demonstrate that a recently-introduced m ethod for measuring semantic shift may be used to proactively identify fai lure points of language-based models and improve predictive generalization over time. Ultimately\, we find that these analyses are critical to produ cing accurate longitudinal studies of social media. DTSTART;TZID=America/New_York:20220207T120000 DTEND;TZID=America/New_York:20220207T131500 LOCATION:In Person or Virtual Option @ https://wse.zoom.us/j/96735183473 @ 234 Ames Hall\, 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Student Seminar – Keith Harrigian “The Problem of Semantic Shift in Longitudinal Monitoring of Social Media: A Case Study on Mental Health d uring the COVID-19 Pandemic” URL:https://www.clsp.jhu.edu/events/student-seminar-keith-harrigian-the-pro blem-of-semantic-shift-in-longitudinal-monitoring-of-social-media-a-case-s tudy-on-mental-health-during-the-covid-19-pandemic/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n
\\nAbstr act
\nSocial media allows researchers to track societal and cultural changes over time based on language analysis tools. Many of thes e tools rely on statistical algorithms which need to be tuned to specific types of language. Recent studies have questioned the robustness of longit udinal analyses based on statistical methods due to issues of temporal bia s and semantic shift. To what extent are changes in semantics over time af fecting the reliability of longitudinal analyses? We examine this question through a case study: understanding shifts in mental health during the co urse of the COVID-19 pandemic. We demonstrate that a recently-introduced m ethod for measuring semantic shift may be used to proactively identify fai lure points of language-based models and improve predictive generalization over time. Ultimately\, we find that these analyses are critical to produ cing accurate longitudinal studies of social media.
\n X-TAGS;LANGUAGE=en-US:2022\,February\,Harrigian END:VEVENT BEGIN:VEVENT UID:ai1ec-21275@www.clsp.jhu.edu DTSTAMP:20240329T131608Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION:Abstract\n\n\n\nAutomatic discovery of phone or word-like units is one of the core objectives in zero-resource speech processing. Recent attempts employ contrastive predictive coding (CPC)\, where the model lear ns representations by predicting the next frame given past context. Howeve r\, CPC only looks at the audio signal’s structure at the frame level. The speech structure exists beyond frame-level\, i.e.\, at phone level or eve n higher. We propose a segmental contrastive predictive coding (SCPC) fram ework to learn from the signal structure at both the frame and phone level s.\n\nSCPC is a hierarchical model with three stages trained in an end-to- end manner. In the first stage\, the model predicts future feature frames and extracts frame-level representation from the raw waveform. In the seco nd stage\, a differentiable boundary detector finds variable-length segmen ts. In the last stage\, the model predicts future segments to learn segmen t representations. Experiments show that our model outperforms existing ph one and word segmentation methods on TIMIT and Buckeye datasets. DTSTART;TZID=America/New_York:20220211T120000 DTEND;TZID=America/New_York:20220211T131500 LOCATION:Ames Hall 234 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Student Seminar – Saurabhchand Bhati “Segmental Contrastive Predict ive Coding for Unsupervised Acoustic Segmentation” URL:https://www.clsp.jhu.edu/events/student-seminar-saurabhchand-bhati/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\n\n\n\n\nAutomatic discovery of phone or word-like units is one of the core objectives in zero-resource speech processing. Recent attempts employ contrastive predictive coding (CPC)\, where the model learns repre sentations by predicting the next frame given past context. However\, CPC only looks at the audio signal’s structure at the frame level. The speech structure exists beyond frame-level\, i.e.\, at phone level or even higher . We propose a segmental contrastive predictive coding (SCPC) framework to learn from the signal structure at both the frame and phone levels.\n\n\nSCPC is a hierarchical mode l with three stages trained in an end-to-end manner. In the first stage\, the model predicts future feature frames and extracts frame-level represen tation from the raw waveform. In the second stage\, a differentiable bound ary detector finds variable-length segments. In the last stage\, the model predicts future segments to learn segment representations. Experiments sh ow that our model outperforms existing phone and word segmentation methods on TIMIT and Buckeye datasets.
Abstr act
\nAdversarial attacks deceive neural network systems by adding carefully crafted perturbations to benign signals. Being almost imperceptible to humans\, these attacks pose a severe security thr eat to the state-of-the-art speech and speaker recognition systems\, makin g it vital to propose countermeasures against them. In this talk\, we focu s on 1) classification of a given adversarial attack into attack algorithm type\, threat model type\, and signal-to-adversarial-noise ratios\, 2) de veloping a novel speech denoising solution to further improve the classifi cation performance.
\nOur proposed approach uses a n x-vector network as a signature extractor to get embeddings\, which we c all signatures. These signatures contain information about the attack and can help classify different attack algorithms\, threat models\, and signal -to-adversarial-noise ratios. We demonstrate the transferability of such s ignatures to other tasks. In particular\, a signature extractor trained to classify attacks against speaker identification can also be used to class ify attacks against speaker verification and speech recognition. We also s how that signatures can be used to detect unknown attacks i.e. attacks not included during training. Lastly\, we propose to improve the signature e xtractor by making the job of the signature extractor easier by removing t he clean signal from the adversarial example (which consists of clean sign al+perturbation). We train our signature extractor using adversarial pertu rbation. At inference time\, we use a time-domain denoiser to obtain adver sarial perturbation from adversarial examples. Using our improved approach \, we show that common attacks in the literature (Fast Gradient Sign Metho d (FGSM)\, Projected Gradient Descent (PGD)\, Carlini-Wagner (CW) ) can be classified with accuracy as high as 96%. We also detect unknown attacks w ith an equal error rate (EER) of about 9%\, which is very promising.
\n X-TAGS;LANGUAGE=en-US:2022\,Joshi\,March END:VEVENT BEGIN:VEVENT UID:ai1ec-21615@www.clsp.jhu.edu DTSTAMP:20240329T131608Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION:Abstract\n\n\nWe consider a problem of data collection for sema ntically rich NLU tasks\, where detailed semantics of documents (or uttera nces) are captured using a complex meaning representation. Previously\, d ata collection for such tasks was either handled at the cost of extensive annotator training (e.g. in FrameNet or PropBank) or simplified meaning re presentation (e.g. in QA-SRL or Overnight). In this talk\, we present two systems [1\, 2] that aim to support fast\, accurate\, and expressive sema ntic annotations by pairing human workers with a trained model in the loop .\n\nThe first system\, called Guided K-best [1]\, is an annotation toolki t for conversational semantic parsing. Instead of typing annotations from scratch\, data specialists choose a correct parse from the K-best output of a few-shot prototyped model. As the K-best list can be large (e.g. K=1 00)\, we guide the annotators’ exploration of the K-best list via explaina ble hierarchical clustering. In addition\, we experiment with RoBERTa-bas ed reranking of the K-best list to recalibrate the few-shot model towards Accuracy@K. The final system allows to annotate data up to 35% faster tha n the standard\, non-guided K-best and improves the few-shot model’s top-1 accuracy by up to 18%. The second system\, called SchemaBlocks [2]\, is an annotation toolkit for schemas\, or structured descriptions of frequent real-world scenarios (e.g.\, cooking a meal). It represents schemas in t he annotation UI as nested blocks. Using a novel Causal ARM model\, we fu rther speed up the annotation process and guide data specialists towards e xpressive and diverse schemas. As part of this work\, we collect 232 sche mas\, evaluating their internal coherence and their coverage on large-scal e newswire corpora.\n\n\n DTSTART;TZID=America/New_York:20220311T120000 DTEND;TZID=America/New_York:20220311T131500 LOCATION:Virtual Seminar SEQUENCE:0 SUMMARY:Student Seminar – Anton Belyy “Systems for Human-AI Cooperation on Collecting Semantic Annotations” URL:https://www.clsp.jhu.edu/events/student-seminar-anton-belyy-systems-for -human-ai-cooperation-on-collecting-semantic-annotations/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\n\n X-TAGS;LANGUAGE=en-US:2022\,Belyy\,March END:VEVENT BEGIN:VEVENT UID:ai1ec-21616@www.clsp.jhu.edu DTSTAMP:20240329T131608Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION:Abstract\nSocial media allows researchers to track societal and cultural changes over time based on language analysis tools. Many of thes e tools rely on statistical algorithms which need to be tuned to specific types of language. Recent studies have shown the absence of appropriate tu ning\, specifically in the presence of semantic shift\, can hinder robustn ess of the underlying methods. However\, little is known about the practic al effect this sensitivity may have on downstream longitudinal analyses. W e explore this gap in the literature through a timely case study: understa nding shifts in depression during the course of the COVID-19 pandemic. We find that inclusion of only a small number of semantically-unstable featur es can promote significant changes in longitudinal estimates of our target outcome. At the same time\, we demonstrate that a recently-introduced met hod for measuring semantic shift may be used to proactively identify failu re points of language-based models and\, in turn\, improve predictive gene ralization. DTSTART;TZID=America/New_York:20220318T120000 DTEND;TZID=America/New_York:20220318T131500 LOCATION:Ames Hall 234 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Student Seminar – Keith Harrigian “The Problem of Semantic Shift in Longitudinal Monitoring of Social Media” URL:https://www.clsp.jhu.edu/events/student-seminar-keith-harrigian-the-pro blem-of-semantic-shift-in-longitudinal-monitoring-of-social-media/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\n
Abstr act
\nSocial media allows researchers to track societal and cultural changes over time based on language analysis tools. Many of thes e tools rely on statistical algorithms which need to be tuned to specific types of language. Recent studies have shown the absence of appropriate tu ning\, specifically in the presence of semantic shift\, can hinder robustn ess of the underlying methods. However\, little is known about the practic al effect this sensitivity may have on downstream longitudinal analyses. W e explore this gap in the literature through a timely case study: understa nding shifts in depression during the course of the COVID-19 pandemic. We find that inclusion of only a small number of semantically-unstable featur es can promote significant changes in longitudinal estimates of our target outcome. At the same time\, we demonstrate that a recently-introduced met hod for measuring semantic shift may be used to proactively identify failu re points of language-based models and\, in turn\, improve predictive gene ralization.
\n X-TAGS;LANGUAGE=en-US:2022\,Harrigian\,March END:VEVENT BEGIN:VEVENT UID:ai1ec-23555@www.clsp.jhu.edu DTSTAMP:20240329T131608Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION: DTSTART;TZID=America/New_York:20230327T120000 DTEND;TZID=America/New_York:20230327T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Student Seminar – Desh Raj URL:https://www.clsp.jhu.edu/events/student-seminar-desh-raj-2/ X-COST-TYPE:free X-TAGS;LANGUAGE=en-US:2023\,March\,Raj END:VEVENT BEGIN:VEVENT UID:ai1ec-23515@www.clsp.jhu.edu DTSTAMP:20240329T131608Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION:Abstract\n\n\n\nHow important are different temporal speech mod ulations for speech recognition? We answer this question from two compleme ntary perspectives. Firstly\, we quantify the amount of phonetic informati on in the modulation spectrum of speech by computing the mutual informatio n between temporal modulations with frame-wise phoneme labels. Looking fro m another perspective\, we ask – which speech modulations an Automatic Spe ech Recognition (ASR) system prefers for its operation. Data-driven weight s are learned over the modulation spectrum and optimized for an end-to-end ASR task. Both methods unanimously agree that speech information is mostl y contained in slow modulation. Maximum mutual information occurs around 3 -6 Hz which also happens to be the range of modulations most preferred by the ASR. In addition\, we show that the incorporation of this knowledge in to ASRs significantly reduces their dependency on the amount of training d ata.\n DTSTART;TZID=America/New_York:20230403T120000 DTEND;TZID=America/New_York:20230403T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Student Seminar – Samik Sadhu (JHU) “Importance of Different Tempor al Modulations of Speech: A Tale of Two Perspectives” URL:https://www.clsp.jhu.edu/events/student-seminar-samik-sadhu/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nHow important are different temporal speech modulations for speec h recognition? We answer this question from two complementary perspectives . Firstly\, we quantify the amount of phonetic information in the modulati on spectrum of speech by computing the mutual information between temporal modulations with frame-wise phoneme labels. Looking from another perspect ive\, we ask – which speech modulations an Automatic Speech Recognition (A SR) system prefers for its operation. Data-driven weights are learned over the modulation spectrum and optimized for an end-to-end ASR task. Both me thods unanimously agree that speech information is mostly contained in slo w modulation. Maximum mutual information occurs around 3-6 Hz which also h appens to be the range of modulations most preferred by the ASR. In additi on\, we show that the incorporation of this knowledge into ASRs significan tly reduces their dependency on the amount of training data.
\n< p> \nAbstr act
\nEmbedding text sequ ences is a widespread requirement in modern language understanding. Existi ng approaches focus largely on constant-size representations. This is prob lematic\, as the amount of information contained in text often varies with the length of the input. We propose a solution called Nugget\, which enco des language into a representation based on a dynamically selected subset of input tokens. These nuggets are learned through tasks like autoencoding and machine translation\, and intuitively segment language into meaningfu l units. We demonstrate Nugget outperforms related approaches in tasks inv olving semantic comparison. Finally\, we illustrate these compact units al low for expanding the contextual window of a language model (LM)\, suggest ing new future LMs that can condition on significantly larger amounts of c ontent.
\n X-TAGS;LANGUAGE=en-US:2023\,Qin\,September END:VEVENT BEGIN:VEVENT UID:ai1ec-23898@www.clsp.jhu.edu DTSTAMP:20240329T131608Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION:Abstract\nAny valuable NLP dataset has traditionally been shipp ed with crowdsourced categorical labels. Instructions for collecting these labels are easy to communicate and the labels themselves are easy to anno tate. However\, as self-supervision based methods are getting better at ba sically everything\, human annotations may need to provide more nuanced su pervision or enable more detailed evaluation in order to be worth further collecting. One natural extension to existing categorical annotation schem es is to obtain uncertainty information beyond a single hard label. In thi s talk\, I will discuss my recent efforts on introducing scalar labels in place of categorical labels as a form of uncertainty annotation. We demons trate that\, compared to other more obvious annotation schemes for eliciti ng uncertainty information\, scalar labels are significantly more cost-eff ective to annotate\, provide reliable evaluation\, and have a theoretical connection to existing predictive uncertainty metrics. In particular\, the y motivate using other losses as surrogates for calibration evaluation. DTSTART;TZID=America/New_York:20230929T120000 DTEND;TZID=America/New_York:20230929T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:CLSP Student Seminar – Zhengping Jiang “Scalar Labels for Capturing Human Uncertainty” URL:https://www.clsp.jhu.edu/events/clsp-student-seminar-zhengping-jiang/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nAny valuable NLP d ataset has traditionally been shipped with crowdsourced categorical labels . Instructions for collecting these labels are easy to communicate and the labels themselves are easy to annotate. However\, as self-supervision bas ed methods are getting better at basically everything\, human annotations may need to provide more nuanced supervision or enable more detailed evalu ation in order to be worth further collecting. One natural extension to ex isting categorical annotation schemes is to obtain uncertainty information beyond a single hard label. In this talk\, I will discuss my recent effor ts on introducing scalar labels in place of categorical labels as a form o f uncertainty annotation. We demonstrate that\, compared to other more obv ious annotation schemes for eliciting uncertainty information\, scalar lab els are significantly more cost-effective to annotate\, provide reliable e valuation\, and have a theoretical connection to existing predictive uncer tainty metrics. In particular\, they motivate using other losses as surrog ates for calibration evaluation.
\n X-TAGS;LANGUAGE=en-US:2023\,Jiang\,September END:VEVENT BEGIN:VEVENT UID:ai1ec-23900@www.clsp.jhu.edu DTSTAMP:20240329T131608Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION: DTSTART;TZID=America/New_York:20231002T120000 DTEND;TZID=America/New_York:20231002T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:CLSP Student Seminar – Anna Favaro URL:https://www.clsp.jhu.edu/events/clsp-student-seminar-anna-favaro/ X-COST-TYPE:free X-TAGS;LANGUAGE=en-US:2023\,Favaro\,October END:VEVENT BEGIN:VEVENT UID:ai1ec-24115@www.clsp.jhu.edu DTSTAMP:20240329T131608Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION:Abstract\nOur goal is to use AI to automatically find tax minim ization strategies\, an approach which we call “Shelter Check.” It would c ome in two variants. Existing-Authority Shelter Check would aim to find wh ether existing tax law authorities can be combined to create tax minimizat ion strategies\, so the IRS or Congress can shut them down. New-Authority Shelter Check would automate checking whether a new tax law authority – li ke proposed legislation or a draft court decision – would combine with exi sting authorities to create a new tax minimization strategy. We had initia lly had high hopes for GPT-* large language models for implementing Shelte r Check\, but our tests have showed that they do very poorly at basic lega l reasoning and handling legal text. So we are now creating a benchmark an d training data for LLM’s handling legal text\, hoping to spur improvement s. DTSTART;TZID=America/New_York:20231006T120000 DTEND;TZID=America/New_York:20231006T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:CLSP Student Seminar – Andrew Blair-Stanek “Shelter Check and GPT-4 ’s Bad Legal Performance” URL:https://www.clsp.jhu.edu/events/clsp-student-seminar-andrew-blair-stane k-shelter-check-and-gpt-4s-bad-legal-performance/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nOur goal is to use AI to automatically find tax minim ization strategies\, an approach which we call “Shelter Check.” It would c ome in two variants. Existing-Authority Shelter Check would aim to find wh ether existing tax law authorities can be combined to create tax minimizat ion strategies\, so the IRS or Congress can shut them down. New-Authority Shelter Check would automate checking whether a new tax law authority – li ke proposed legislation or a draft court decision – would combine with exi sting authorities to create a new tax minimization strategy. We had initia lly had high hopes for GPT-* large language models for implementing Shelte r Check\, but our tests have showed that they do very poorly at basic lega l reasoning and handling legal text. So we are now creating a benchmark an d training data for LLM’s handling legal text\, hoping to spur improvement s.
\n X-TAGS;LANGUAGE=en-US:2023\,Blair-Stanek\,October END:VEVENT BEGIN:VEVENT UID:ai1ec-23904@www.clsp.jhu.edu DTSTAMP:20240329T131608Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION: DTSTART;TZID=America/New_York:20231016T120000 DTEND;TZID=America/New_York:20231016T131500 SEQUENCE:0 SUMMARY:CLSP Student Seminar – Maliha Jahan URL:https://www.clsp.jhu.edu/events/clsp-student-seminar-maliha-jahan/ X-COST-TYPE:free X-TAGS;LANGUAGE=en-US:2023\,Jahan\,October END:VEVENT BEGIN:VEVENT UID:ai1ec-23906@www.clsp.jhu.edu DTSTAMP:20240329T131608Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION: DTSTART;TZID=America/New_York:20231023T120000 DTEND;TZID=America/New_York:20231023T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:CLSP Student Seminar – David Mueller URL:https://www.clsp.jhu.edu/events/clsp-student-seminar-david-mueller/ X-COST-TYPE:free X-TAGS;LANGUAGE=en-US:2023\,Mueller\,October END:VEVENT BEGIN:VEVENT UID:ai1ec-24155@www.clsp.jhu.edu DTSTAMP:20240329T131608Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION:Abstract\nMultilingual machine translation has proven immensely useful for both parameter efficiency and overall performance for many lan guage pairs via complete parameter sharing. However\, some language pairs in multilingual models can see worse performance than in bilingual models\ , especially in the one-to-many translation setting. Motivated by their em pirical differences\, we examine the geometric differences in representati ons from bilingual models versus those from one-to-many multilingual model s. Specifically\, we measure the isotropy of these representations using i ntrinsic dimensionality and IsoScore\, in order to measure how these repre sentations utilize the dimensions in their underlying vector space. We fin d that for a given language pair\, its multilingual model decoder represen tations are consistently less isotropic than comparable bilingual model de coder representations. Additionally\, we show that much of this anisotropy in multilingual decoder representations can be attributed to modeling lan guage-specific information\, therefore limiting remaining representational capacity. DTSTART;TZID=America/New_York:20231106T120000 DTEND;TZID=America/New_York:20231106T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Student Seminar – Neha Verma “Exploring Geometric Representational Disparities Between Multilingual and Bilingual Translation Models” URL:https://www.clsp.jhu.edu/events/student-seminar-neha-verma-exploring-ge ometric-representational-disparities-between-multilingual-and-bilingual-tr anslation-models/ X-COST-TYPE:free X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\nAbstr act
\nMultilingual machine translation has proven immensely useful for both parameter efficiency and overall perf ormance for many language pairs via complete parameter sharing. However\, some language pairs in multilingual models can see worse performance than in bilingual models\, especially in the one-to-many translation setting. M otivated by their empirical differences\, we examine the geometric differe nces in representations from bilingual models versus those from one-to-man y multilingual models. Specifically\, we measure the isotropy of these rep resentations using intrinsic dimensionality and IsoScore\, in order to mea sure how these representations utilize the dimensions in their underlying vector space. We find that for a given language pair\, its multilingual mo del decoder representations are consistently less isotropic than comparabl e bilingual model decoder representations. Additionally\, we show that muc h of this anisotropy in multilingual decoder representations can be attrib uted to modeling language-specific information\, therefore limiting remain ing representational capacity.
\n X-TAGS;LANGUAGE=en-US:2023\,November\,Verma END:VEVENT BEGIN:VEVENT UID:ai1ec-24159@www.clsp.jhu.edu DTSTAMP:20240329T131608Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION: DTSTART;TZID=America/New_York:20231113T120000 DTEND;TZID=America/New_York:20231113T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Student Seminar – Kate Sanders URL:https://www.clsp.jhu.edu/events/student-seminar-kate-sanders/ X-COST-TYPE:free X-TAGS;LANGUAGE=en-US:2023\,November\,Sanders END:VEVENT BEGIN:VEVENT UID:ai1ec-24165@www.clsp.jhu.edu DTSTAMP:20240329T131608Z CATEGORIES;LANGUAGE=en-US:Student Seminars CONTACT: DESCRIPTION: DTSTART;TZID=America/New_York:20231127T120000 DTEND;TZID=America/New_York:20231127T131500 LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218 SEQUENCE:0 SUMMARY:Student Seminar – Aleem Khan URL:https://www.clsp.jhu.edu/events/student-seminar-aleem-khan/ X-COST-TYPE:free X-TAGS;LANGUAGE=en-US:2023\,Khan\,November END:VEVENT END:VCALENDAR