Andrea Santilli

Andrea Santilli

PhD Student in Computer Science

GLADIA, Sapienza University of Rome

Biography

Hi! I’m Andrea Santilli a third-year Ph.D. student in Computer Science at GLADIA, Sapienza University of Rome, supervised by Prof. Emanuele Rodolà.

My main research interests lie at the intersection of natural language processing, representation learning and machine intelligence. In particular, I have explored several topics like syntax in transformers, instruction-tuned LMs, audio LM, multimodal neural databases and efficient decoding techniques (checkout my publications!).

I’m an enthusiast of Open Source and Open Science initiatives. I have contributed to Hugginface’s BigScience and whenever possible I contribute to open-source projects (GitHub profile). If you are interested in what I’m doing and would like to contact me, you can send me a DM on Twitter or use the contact form below!

Interests
  • Natural Language Processing
  • Representation Learning
  • Machine Intelligence
Education

  • PhD in Artificial Intelligence

    Sapienza University of Rome

  • MSc in Computer Science, 2020

    University of Roma Tor Vergata

  • BSc in Computer Science, 2018

    University of Roma Tor Vergata

Publications

Accelerating Transformer Inference for Translation via Parallel Decoding
ACL 2023
Autoregressive decoding limits the efficiency of transformers for Machine Translation (MT). The community proposed specific network …
Andrea Santilli, Silvio Severino, Emilian Postolache, Valentino Maiorca, Michele Mancusi, Riccardo Marin, Emanuele Rodolà
PDF Cite arXiv GitHub
Accelerating Transformer Inference for Translation via Parallel Decoding
Multimodal Neural Databases
SIGIR 2023
The rise in loosely-structured data available through text, images, and other modalities has called for new ways of querying them. …
Giovanni Trappolini, Andrea Santilli, Emanuele Rodolà, Alon Halevy, Fabrizio Silvestri
PDF Cite arXiv
Multimodal Neural Databases
Latent Autoregressive Source Separation
AAAI 2023
Autoregressive models have achieved impressive results over a wide range of domains in terms of generation quality and downstream task …
Emilian Polostache, Giorgio Mariani, Michele Mancusi, Andrea Santilli, Luca Cosmo, Emanuele Rodolà
Cite AAAI 2023
Latent Autoregressive Source Separation
Bloom: A 176b-parameter open-access multilingual language model
arXiv
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language …
BIG-Science contributors including, Andrea Santilli
PDF Cite arXiv
Bloom: A 176b-parameter open-access multilingual language model
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
TMLR
Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their …
BIG-bench contributors including, Andrea Santilli, Antonio Norelli, Emanuele Rodolà, Giambattista Parascandolo, Giorgio Mariani, Luca Moschella, Simone Melzi
PDF Cite arXiv
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts
ACL 2022
PromptSource is a system for creating, sharing, and using natural language prompts. Prompts are functions that map an example from a …
Stephen Bach, Victor Sanh, Zheng Xin Yong, Albert Webson, Colin Raffel, BIG-Science contributors including, Andrea Santilli
PDF Cite DOI ACL 2022 Demo
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts
Multitask Prompted Training Enables Zero-Shot Task Generalization
ICLR 2022
Large language models have recently been shown to attain reasonable zero-shot generalization on a diverse set of tasks (Brown et al., …
Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, BIG-Science contributors including, Andrea Santilli
PDF Cite ICLR 2022 (Oral)
Multitask Prompted Training Enables Zero-Shot Task Generalization
KERMIT: Complementing Transformer Architectures with Encoders of Explicit Syntactic Interpretations
EMNLP 2020
Syntactic parsers have dominated natural language understanding for decades. Yet, their syntactic interpretations are losing centrality …
Fabio Massimo Zanzotto, Andrea Santilli, Leonardo Ranaldi, Dario Onorati, Pierfrancesco Tommasino, Francesca Fallucchi
PDF Cite DOI EMNLP 2020
KERMIT: Complementing Transformer Architectures with Encoders of Explicit Syntactic Interpretations
Unsupervised Source Separation via Bayesian Inference in the Latent Domain
arXiv preprint
State of the art audio source separation models rely on supervised data-driven approaches, which can be expensive in terms of labeling …
Michele Mancusi, Emilian Postolache, Giorgio Mariani, Marco Fumero, Andrea Santilli, Luca Cosmo, Emanuele Rodolà
PDF Cite arXiv
Unsupervised Source Separation via Bayesian Inference in the Latent Domain
Explanatory Learning: Beyond Empiricism in Neural Networks
arXiv preprint
We introduce Explanatory Learning (EL), a framework to let machines use existing knowledge buried in symbolic sequences – e.g. …
Antonio Norelli, Giorgio Mariani, Luca Moschella, Andrea Santilli, Giambattista Parascandolo, Simone Melzi, Emanuele Rodolà
PDF Cite arXiv
Explanatory Learning: Beyond Empiricism in Neural Networks
A Kernel-based Approach for Irony and Sarcasm Detection in Italian
EVALITA 2018
This paper describes the UNITOR system that participated to the Irony Detection in Italian Tweets task (IronITA) within the context of …
Andrea Santilli, Danilo Croce, Roberto Basili
PDF Cite DOI EVALITA 2018
A Kernel-based Approach for Irony and Sarcasm Detection in Italian
SyntNN at SemEval-2018 Task 2: is Syntax Useful for Emoji Prediction? Embedding Syntactic Trees in Multi Layer Perceptrons
SemEval2018
In this paper, we present SyntNN as a way to include traditional syntactic models in multilayer neural networks used in the task of …
Andrea Santilli, Fabio Massimo Zanzotto
PDF Cite DOI SemEval 2018
SyntNN at SemEval-2018 Task 2: is Syntax Useful for Emoji Prediction? Embedding Syntactic Trees in Multi Layer Perceptrons
Browse all

Research Experience

 
 
 
 
 
Hugginface BigScience
Open Science Researcher
Hugginface BigScience
Jun 2021 – Jun 2022 Remote
Researcher at Hugginface’s workshop on large language models. Worked in the prompt-engineering working group on zero-shot generalization. Two publications.
 
 
 
 
 
ART Lab, University of Roma Tor Vergata
University Research Assistant
ART Lab, University of Roma Tor Vergata
Jan 2018 – Jun 2020 Rome
Worked on syntax in deep neural network models and BERT-based NLP models.
 
 
 
 
 
Pi School, School of Artificial Intelligence
Research Engineer
Pi School, School of Artificial Intelligence
Oct 2019 – Dec 2019 Rome
Worked on a European Commission project to promote entrepreneurship and tech transfer in the R&D area (“Started Project”) via NLP-based tools.
 
 
 
 
 
Mashfrog Group
NLP Research Engineer
Mashfrog Group
Jul 2019 – Oct 2019 Rome
Research and development of NLP models (grammar error correction, language generation) for a web-based text editor for press releases.

Grants Awarded

translated
Our project on efficient Machine Translation (MT) was selected as the winner of the category ‘Machine Learning Algorithms For Translation’ among different proposals submitted by world experts and professors (7% acceptance rate). We develop a novel decoding algorithm to speedup autoregressive transformers up to 2x and published the results at ACL 2023. PI: Andrea Santilli. Budget: 20.000€
ufi
Multimodal Artificial Intelligence for 3D shape analysis, modeling and applications
ufi Dec 2021
Joint project on multimodal 3D and NLP applications between our research group GLADIA at Sapienza and Maks Ovsjanikov’s group at Ecole Polytechnique. PI: Simone Melzi, Maks Ovsjanikov. Budget: 10.000€

Contact