About Me
I am a Postdoctoral Scholar at the University of Washington and AI2, working with Yejin Choi.
Previously, I earned a PhD at New York University, advised by Kyunghyun Cho and Zheng Zhang.
I host the Thesis Review Podcast.
Research
My research focuses on machine learning methods for generating and reasoning with natural language. This includes:
 Developing learning [1, 2, 3, 4], decoding [1, 2, 3, 4], and evaluation [1, 2] methods for neural sequence generation.
 Bridging symbolic structure and neural methods in commonsense [1, 2, 3, 4] and mathematical [1, 2, 3] reasoning.
My longterm goal is enabling highimpact applications in scientific discovery and education through advances in these areas [1, 2, 3].
News
 [12.2022] Keynote at EMNLP 2022 Generation, Evaluation, & Metrics Workshop.
 [11.2022] Talk at Microsoft Research on NaturalProver and Draft, Sketch, and Prove.
 [10.2022] Talk at NSF Expeditions: Understanding the World Through Code seminar.
 [10.2022] COLING Tutorial on Neurosymbolic NLP [slides].
 [10.2022] Papers on logical, mathematical, and commonsense reasoning accepted at EMNLP 2022.
 [09.2022] NaturalProver, Quark, and COLD Decoding accepted at NeurIPS 2022!
 [09.2022] Talk at Google on Maeutic Prompting [slides].
 [09.2022] Talk at AITP 2022 on NaturalProver [slides].
 [07.2022] Coorganizing MathAI: Towards HumanLevel Mathematical Reasoning at Neurips 2022. [webpage]
 [06.2022] Best Paper Award at NAACL 2022 for our paper on constrained A*esque decoding!
 [06.2022] Talk at Google on NaturalProver.
 [05.2022] Talks at IFDS and Cohere.ai on constrained text generation [slides].
 [04.2022] Papers on A* decoding, commonsense knowledge, and continuous prompts accepted at NAACL 2022.
 [03.2022] Guest lecture on neural sequence generation [slides].
 [02.2022] Paper on generated knowledge prompting accepted at ACL 2022.
 [12.2021] Held the first MATHAI for Education workshop at NeurIPS 2021.
 [12.2021] Paper on generalization in symbolic mathematics accepted to AAAI 2022.
 [12.2021] Mauve received an Outstanding Paper Award at NeurIPS 2021.
 [12.2021] Coorganizing a Workshop on Mathematical Language Processing at EMNLP 2022. [webpage]
 [10.2021] NaturalProofs accepted for an oral presentation at NeurIPS 2021 [slides].
Preprints

Draft, Sketch, and Prove: Guiding Formal Theorem Provers with Informal Proofs
A. Jiang*, S. Welleck*, J. Zhou*, T. Lacroix, J. Liu, W. Li, M. Jamnik, G. Lample+, Y. Wu+
In submission.
[data]

Generating Sequences by Learning to [Self]Correct
S. Welleck*, X. Lu*, P. West+, F. Brahman+, T. Shen, D. Khashabi, Y. Choi
In submission.
[poster]
Publications
 NaturalProver: Grounded Mathematical Proof Generation with Language Models
S. Welleck, J. Liu, X. Lu, H. Hajishirzi, Y. Choi.
NeurIPS 2022.
[code][slides][poster]
 Quark: Controllable Text Generation with Reinforced [Un]learning
X. Lu, S. Welleck, L. Jiang, J. Hessel, L. Qin, P. West, P. Ammanabrolu, Y. Choi.
NeurIPS 2022 (Oral).
[poster]
 COLD Decoding: Energybased Constrained Text Generation with Langevin Dynamics
L. Qin, S. Welleck, D. Khasabi, Y. Choi.
NeurIPS 2022 (Oral).
[code][poster]
 Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations
J. Jung, L. Qin, S. Welleck, F. Brahman, C. Bhagavatula, R. Le Bras, Y. Choi.
EMNLP 2022.
[code][slides]
 Lila: A Unified Benchmark for Mathematical Reasoning
S. Mishra, M. Finlayson, P. Lu, L. Tang, S. Welleck, C. Baral, T. Rajpurohit, O. Tafjord, A. Sabharwal, P. Clark, A. Kalyan
EMNLP 2022.
[project page][data/code]
 Rainier: Reinforced Knowledge Introspector for Commonsense Question Answering
J. Liu, S. Hallinan, X. Lu, P. He, S. Welleck, H. Hajishirzi, Y. Choi.
EMNLP 2022.
 NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristics
X. Lu, S. Welleck, P. West, L. Jiang, J. Kasai, D. Khasabi, R. Le Bras, L. Qin, Y. Yu, R. Zellers, N. Smith, Y. Choi.
NAACL 2022.
[code][slides]
 Symbolic Knowledge Distillation: from General Language Models to Commonsense Models
P. West, C. Bhagavatula, J. Hessel, J. Hwang, L. Jiang, R. Le Bras, X. Lu, S. Welleck, Y. Choi.
NAACL 2022.
[code]
 Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Prompts
D. Khasabi, S. Lyu, S. Min, L. Qin, K. Richardson, S. Singh, S. Welleck, H. Hajishirzi, T. Khot, A. Sabharway, Y. Choi.
NAACL 2022.
 Generated Knowledge Prompting for Commonsense Reasoning
J. Liu, A. Liu, X. Lu, S. Welleck, P. West, R. Le Bras, Y. Choi, H. Hajishirzi.
ACL 2022.
[code]
 Symbolic Brittleness in Sequence Models: on Systematic Generalization in Symbolic Mathematics
S. Welleck, P. West, J. Cao, Y. Choi.
AAAI 2022.
[code][slides][talk]
 Towards Grounded Natural Language Proof Generation
S. Welleck, J. Liu, J. Han, Y. Choi.
MathAI4Ed Workshop at NeurIPS 2021 (Contributed Talk).
[poster][slides]
 NaturalProofs: Mathematical Theorem Proving in Natural Language
S. Welleck, J. Liu, R. Le Bras, H. Hajishirzi, Y. Choi, K. Cho.
NeurIPS 2021 Datasets and Benchmarks (Oral (Top 1%)).
[data/code][talk]
 MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers
K. Pillutla, S. Swayamdipta, R. Zellers, J. Thickstun, S. Welleck, Y. Choi, Z. Harchaoui.
NeurIPS 2021 (Oral, Outstanding Paper Award (Top 0.1%)).
[code][press]
 Divergence Frontiers for Generative Models: Sample Complexity, Quantization Level, and Frontier Integral
L. Liu, K. Pillutla, S. Welleck, S. Oh, Y. Choi, Z. Harchaoui.
NeurIPS 2021.
 Mode recovery in neural autoregressive sequence modeling
I. Kulikov, S. Welleck, K. Cho.
SPNLP 2021.
[code]
 Order and Learning in Sequential Neural Structured Prediction
S. Welleck.
PhD Thesis, New York University.
[slides]
 MLEguided parameter search for task loss minimization in neural sequence modeling
S. Welleck, K. Cho.
AAAI 2021.
[code][poster][talk]
 Consistency of a Recurrent Language Model With Respect to Incomplete Decoding
S. Welleck, I. Kulikov, J. Kim, R. Pang, K. Cho.
EMNLP 2020.
[code][talk]
 A Generalized Framework of Sequence Generation with Application to Undirected Sequence Models
E. Mansimov, A. Wang, S. Welleck, K. Cho.
arXiv preprint 2020.
[code]
 Making Inconsistent Dialogue Unlikely with Unlikelihood Training
M. Li, S. Roller, I. Kulikov, S. Welleck, Y.L. Boureau, K. Cho, J. Weston.
ACL 2020.
 Neural Text Generation with Unlikelihood Training
S. Welleck, I. Kulikov, S. Roller, E. Dinan, K. Cho, J. Weston.
ICLR 2020.
[code]
 NonMonotonic Sequential Text Generation
S. Welleck, K. Brantley, H. Daume III, K. Cho.
ICML 2019.
[code] [slides] [poster]
 Sequential Graph Dependency Parser
S. Welleck, K. Cho.
RANLP 2019.
[slides]
 Dialogue Natural Language Inference
S. Welleck, J. Weston, A. Szlam, K. Cho.
ACL 2019.
[dataset][poster][press]
 Loss Functions for Multiset Prediction
S. Welleck, Z. Yao, Y. Gai, J. Mao, Z. Zhang, K. Cho.
NeurIPS 2018.
NVIDIA AI Labs Pioneering Research Award 2018.
[poster]
 Saliencybased Sequential Image Attention with Multiset Prediction
S. Welleck, J. Mao, K. Cho, Z. Zhang.
NeurIPS 2017.
NVIDIA AI Labs Pioneering Research Award 2017.
[poster][press]
 Efficient AUC Optimization for Information Ranking Applications
S. Welleck.
ECIR 2016.
Teaching
 Guest Lecture: Reliable text generation through graph search (CSE 373) [slides]
University of Washington
November 2022.
 Guest Lecture: Neural sequence generation (DATA 598) [slides]
University of Washington
March 2022.
 Deep Learning (DSGA 1008)
New York University
Fall 2020
 Deep Learning for NLP
African Masterâ€™s Program in Machine Intelligence
March 2020
 Introduction to Machine Learning (CSCIUA 0473)
New York University
Spring 2020
 NLP with Representation Learning (DSGA 1011)
New York University
Fall 2019
Tutorials
 Neurosymbolic NLP: Modularity & Constraints for Neural Language Models [slides][tutorial site]
COLING 2022.
 Denoising Diffusion Models [slides]
July 2022.
 Generative Modeling with (W)GAN [slides]
NYU Shanghai
April 2018.
Past
 Facebook, AI Research Team (FAIR), May. 2019  Sep. 2019
 Facebook, AI Research Team (FAIR), May. 2018  Jan. 2019
 Primer AI, Feb. 2016  Aug. 2016
 IBM, Sep. 2014  Feb. 2016
 University of Pennsylvania (Computer Science, MSE), May. 2013  May. 2014
 University of Pennsylvania (Computer Science, BSE), Sep. 2009  Feb. 2013