Computer engineer with 11 years of experience, specialized in mission critical, high traffic, high available Linux architectures and infrastructures (before the cloud was out), with a relevant experience in development and management of web services. He has served as Infrastructure Lead in 4 companies (Translated, N26, Wanderio, Klar) and participated in 2 EU multimillion funded NLP research projects (MateCAT, ModernMT). Alberto has a variegated bundle of experience, that ranges from devops to machine learning, from the corporate banking to the mutable startup world.Back to speakers list
Awake: Attention mechanisms in Neural Networks
Attention is one of the most influential ideas in the Deep Learning community: the net can learn how to focus on relevant details of its training samples to filter out noise and improve outcome. Born as a supporting system for the “sequence to sequence” models in Machine Translation, it has rose to prominence in Image Captioning, Object Identification, Question Answering, Language Modeling and even Natural Language Generation. Let’s explore the underlying mechanics of Attention in Neural Networks, along with Transformer, BERT and GPT-2.