BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//128.220.36.25//NONSGML kigkonsult.se iCalcreator 2.26.9//
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-FROM-URL:https://www.clsp.jhu.edu
X-WR-TIMEZONE:America/New_York
BEGIN:VTIMEZONE
TZID:America/New_York
X-LIC-LOCATION:America/New_York
BEGIN:STANDARD
DTSTART:20231105T020000
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
RDATE:20241103T020000
TZNAME:EST
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:20240310T020000
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
RDATE:20250309T020000
TZNAME:EDT
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
UID:ai1ec-21267@www.clsp.jhu.edu
DTSTAMP:20240329T063624Z
CATEGORIES;LANGUAGE=en-US:Seminars
CONTACT:
DESCRIPTION:Abstract\nIn this talk\, I present a multipronged strategy for
zero-shot cross-lingual Information Extraction\, that is the construction
of an IE model for some target language\, given existing annotations exclu
sively in some other language. This work is part of the JHU team’s effort
under the IARPA BETTER program. I explore data augmentation techniques inc
luding data projection and self-training\, and how different pretrained en
coders impact them. We find through extensive experiments and extension of
techniques that a combination of approaches\, both new and old\, leads to
better performance than any one cross-lingual strategy in particular.\nBi
ography\nMahsa Yarmohammadi is an assistant research scientist in CLSP\, J
HU\, who leads state-of-the-art research in cross-lingual language and spe
ech applications and algorithms. A primary focus of Yarmohammadi’s researc
h is using deep learning techniques to transfer existing resources into ot
her languages and to learn representations of language from multilingual d
ata. She also works in automatic speech recognition and speech translation
. Yarmohammadi received her PhD in computer science and engineering from O
regon Health & Science University (2016). She joined CLSP as a post-doctor
al fellow in 2017.
DTSTART;TZID=America/New_York:20220204T120000
DTEND;TZID=America/New_York:20220204T131500
LOCATION:Ames 234 Presented Virtually via Zoom https://wse.zoom.us/j/967351
83473
SEQUENCE:0
SUMMARY:Mahsa Yarmohammadi (Johns Hopkins University) “Data Augmentation fo
r Zero-shot Cross-Lingual Information Extraction”
URL:https://www.clsp.jhu.edu/events/mahsa-yarmohammadi-johns-hopkins-univer
sity-data-augmentation-for-zero-shot-cross-lingual-information-extraction/
X-COST-TYPE:free
X-ALT-DESC;FMTTYPE=text/html:\\n\\n
\\n\\n\\nAbstr
act
\nIn this talk\, I present a multipronged strategy for
zero-shot cross-lingual Information Extraction\, that is the construction
of an IE model for some target language\, given existing annotations exclu
sively in some other language. This work is part of the JHU team’s effort
under the IARPA BETTER program. I explore data augmentation techniques inc
luding data projection and self-training\, and how different pretrained en
coders impact them. We find through extensive experiments and extension of
techniques that a combination of approaches\, both new and old\, leads to
better performance than any one cross-lingual strategy in particular.
\nBiography
\nMahsa Yarmohammadi is an assista
nt research scientist in CLSP\, JHU\, who leads state-of-the-art research
in cross-lingual language and speech applications and algorithms. A primar
y focus of Yarmohammadi’s research is using deep learning techniques to tr
ansfer existing resources into other languages and to learn representation
s of language from multilingual data. She also works in automatic speech r
ecognition and speech translation. Yarmohammadi received her PhD in comput
er science and engineering from Oregon Health & Science University (2016).
She joined CLSP as a post-doctoral fellow in 2017.
\n\n
BODY>
X-TAGS;LANGUAGE=en-US:2022\,February\,Yarmohammadi
END:VEVENT
BEGIN:VEVENT
UID:ai1ec-22423@www.clsp.jhu.edu
DTSTAMP:20240329T063624Z
CATEGORIES;LANGUAGE=en-US:Seminars
CONTACT:
DESCRIPTION:
DTSTART;TZID=America/New_York:20221007T120000
DTEND;TZID=America/New_York:20221007T131500
LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218
SEQUENCE:0
SUMMARY:Ariya Rastrow (Amazon)
URL:https://www.clsp.jhu.edu/events/ariya-rastrow-amazon-2/
X-COST-TYPE:free
X-TAGS;LANGUAGE=en-US:2022\,October\,Rastrow
END:VEVENT
BEGIN:VEVENT
UID:ai1ec-23304@www.clsp.jhu.edu
DTSTAMP:20240329T063624Z
CATEGORIES;LANGUAGE=en-US:Seminars
CONTACT:
DESCRIPTION:Abstract\nTransformers are essential to pretraining. As we appr
oach 5 years of BERT\, the connection between attention as architecture an
d transfer learning remains key to this central thread in NLP. Other archi
tectures such as CNNs and RNNs have been used to replicate pretraining res
ults\, but these either fail to reach the same accuracy or require supplem
ental attention layers. This work revisits the semanal BERT result and con
siders pretraining without attention. We consider replacing self-attention
layers with recently developed approach for long-range sequence modeling
and transformer architecture variants. Specifically\, inspired by recent p
apers like the structured space space sequence model (S4)\, we use simple
routing layers based on state-space models (SSM) and a bidirectional model
architecture based on multiplicative gating. We discuss the results of th
e proposed Bidirectional Gated SSM (BiGS) and present a range of analysis
into its properties. Results show that architecture does seem to have a no
table impact on downstream performance and a different inductive bias that
is worth exploring further.\nBiography\nAlexander “Sasha” Rush is an Asso
ciate Professor at Cornell Tech. His work is at the intersection of natura
l language processing and generative modeling with applications in text ge
neration\, efficient inference\, and controllability. He has written sever
al popular open-source software projects supporting NLP research and data
science\, and works part-time as a researcher at Hugging Face. He is the s
ecretary of ICLR and developed software used to run virtual conferences du
ring COVID. His work has received paper and demo awards at major NLP\, vis
ualization\, and hardware conferences\, an NSF Career Award\, and a Sloan
Fellowship. He tweets and blogs\, mostly about coding and ML\, at @srush_n
lp.
DTSTART;TZID=America/New_York:20230203T120000
DTEND;TZID=America/New_York:20230203T131500
LOCATION:Hackerman Hall B17 @ 3400 N. Charles Street\, Baltimore\, MD 21218
SEQUENCE:0
SUMMARY:Sasha Rush (Cornell University) “Pretraining Without Attention”
URL:https://www.clsp.jhu.edu/events/sasha-rush-cornell-university/
X-COST-TYPE:free
X-ALT-DESC;FMTTYPE=text/html:\\n\\n\\n\\n\\nAbstr
act
\nTransformers are essential to pretraining. As we appr
oach 5 years of BERT\, the connection between attention as architecture an
d transfer learning remains key to this central thread in NLP. Other archi
tectures such as CNNs and RNNs have been used to replicate pretraining res
ults\, but these either fail to reach the same accuracy or require supplem
ental attention layers. This work revisits the semanal BERT result and con
siders pretraining without attention. We consider replacing self-attention
layers with recently developed approach for long-range sequence modeling
and transformer architecture variants. Specifically\, inspired by recent p
apers like the structured space space sequence model (S4)\, we use simple
routing layers based on state-space models (SSM) and a bidirectional model
architecture based on multiplicative gating. We discuss the results of th
e proposed Bidirectional Gated SSM (BiGS) and present a range of analysis
into its properties. Results show that architecture does seem to have a no
table impact on downstream performance and a different inductive bias that
is worth exploring further.
\nBiography
\n
Alexander “Sasha”
Rush is an Associate Professor at Cornell Tech. His work is at the
intersection of natural language processing and generative modeling with
applications in text generation\, efficient inference\, and controllabilit
y. He has written several popular open-source software projects supporting
NLP research and data science\, and works part-time as a researcher at Hu
gging Face. He is the secretary of ICLR and developed software used to run
virtual conferences during COVID. His work has received paper and demo aw
ards at major NLP\, visualization\, and hardware conferences\, an NSF Care
er Award\, and a Sloan Fellowship. He tweets and blogs\, mostly about codi
ng and ML\, at
@srush_nlp.
\n\n
X-TAGS;LANGUAGE=en-US:2023\,February\,Rush
END:VEVENT
END:VCALENDAR