Deep learning for graphs and sets.

Deep learning has achieved remarkable results in fields such as computer vision, speech recognition and natural language processing. More recently, there has been a surge in research on graph and set representation learning, leading to new state-of-the-art in different domains such as 3D vision, biological networks and social networks analysis.

In this talk, I will present our work on Graph Attention Networks (GATs). I will start by reviewing early approaches to leverage neural networks for processing graph structured data, with special emphasis on graph convolutions, highlighting potential issues and motivating our work. Then, I will introduce GATs, a neural network architecture that leverages masked self-attentional layers to address the shortcomings of prior methods based on graph convolutions or their approximations. The second part of the talk will be devoted to set prediction in the context of generating recipes from images of food. A crucial part of this task is to predict ingredients sets, which are characterized by variable cardinality and dependencies among different elements in the set. I will give an overview of existing architectures to tackle this task showcasing their potential limitations and introduce our model, suitable for image to ingredients set prediction. I will wrap up the talk showing how sets of ingredients together with images can be used to generate food recipes.

Speaker Bio:

Adriana Romero is a research scientist at Facebook AI Research and an adjunct professor at McGill University. She is currently involved in a number of projects at the intersection of conditional generation, multi-modality and graph-structured data. Previously, she was a post-doctoral researcher at Montreal Institute for Learning algorithms, advised by Prof. Yoshua Bengio. Her postdoctoral research revolved around deep learning techniques to tackle biomedical challenges, such as the ones posed by imaging multi-modality, high dimensional data and graph structures. Adriana received her Ph.D. from the University of Barcelona in 2015 with a thesis on assisting the training of deep neural networks with applications to computer vision, advised by Dr. Carlo Gatta. Her PhD included contributions in the fields of representation learning and model compression, with applications to image classification, image segmentation and remote sensing.