Dr. Petar Veličković is a Staff Research Scientist at DeepMind, he has firmly established himself as one of the most significant up and coming researchers in the deep learning space. He invented Graph Attention Networks in 2017 and has been a leading light in the field ever since pioneering research in Graph Neural Networks, Geometric Deep Learning and also Neural Algorithmic reasoning. If you haven’t already, you should check out our video on the Geometric Deep learning blueprint, featuring Petar. I caught up with him last week at NeurIPS. In this show, from NeurIPS 2022 we discussed his recent work on category theory and graph neural networks. https://petar-v.com/ https://twitter.com/PetarV_93/ TOC: Categories (Cats for AI) [00:00:00] Reasoning [00:14:44] Extrapolation [00:19:09] Ishan Misra Skit [00:27:50] Graphs (Expander Graph Propagation) [00:29:18] Pod: https://anchor.fm/machinelearningstreettalk/episodes/85-Dr--Petar-Velikovi-Deepmind---Categories--Graphs-NEURIPS22-UNPLUGGED-e1rvtha MLST Discord: https://discord.gg/V25vQeFwhS Support us! https://www.patreon.com/mlst Host: Dr. Tim Scarfe References MLST#60 Geometric Deep Learning Blueprint (Special Edition) https://www.youtube.com/watch?v=bIZB1hIJ4u8 Categories for AI https://cats.for.ai/ Organised by: Andrew Dudzik - DeepMind Bruno Gavranović - University of Strathclyde João Guilherme Araújo - Cohere / Universidade de São Paulo Petar Veličković - DeepMind / University of Cambridge Pim de Haan - University of Amsterdam / Qualcomm AI Research [Petar Veličković] Graph Attention Networks https://arxiv.org/abs/1710.10903 Learning to Configure Computer Networks with Neural Algorithmic Reasoning [NeurIPS 2022] [Luca Beurer-Kellner, Martin Vechev, Laurent Vanbever, Petar Veličković] https://openreview.net/forum?id=AiY6XvomZV4 Graph Neural Networks are Dynamic Programmers [Andrew Joseph Dudzik, Petar Veličković] https://arxiv.org/abs/2203.15544 Expander Graph Propagation [Andreea Deac, Marc Lackenby, Petar Veličković] https://openreview.net/forum?id=IKevTLt3rT [Pim de Haan, Taco Cohen, Max Welling] Natural Graph Networks https://papers.nips.cc/paper/2020/file/2517756c5a9be6ac007fe9bb7fb92611-Paper.pdf [Uri Alon, Eran Yahav] On the Bottleneck of Graph Neural Networks and its Practical Implications (they discovered oversquashing) https://arxiv.org/abs/2006.05205 [Topping,...,Bronstein] Understanding over-squashing and bottlenecks on graphs via curvature https://arxiv.org/abs/2111.14522 [Andreea Deac, Petar Velickovic, Ognjen Milinkovic et al] XLVIN: eXecuted Latent Value Iteration Nets https://arxiv.org/abs/2010.13146 [Petar Veličković et al] Reasoning-Modulated Representations (RMR) https://openreview.net/forum?id=cggphp7nPuI Dual Algorithmic Reasoning [PetarV, under review ICLR] https://openreview.net/pdf?id=hhvkdRdWt1F [Petar Veličković, Charles Blundell] Neural Algorithmic Reasoning https://arxiv.org/abs/2105.02761 [Andreea Deac, Petar Veličković, ...] Neural Algorithmic Reasoners are Implicit Planners (which got a NeurIPS spotlight in 2021!) https://arxiv.org/abs/2110.05442 A Generalist Neural Algorithmic Learner https://arxiv.org/abs/2209.11142 ETA Prediction with Graph Neural Networks in Google Maps https://arxiv.org/abs/2108.11482 [Randall Balestriero] A Spline Theory of Deep Networks https://proceedings.mlr.press/v80/balestriero18b/balestriero18b.pdf [Ahmed Imtiaz Humayun ] Exact Visualization of Deep Neural Network Geometry and Decision Boundary https://arxiv.org/pdf/2009.11848.pdf [Ahmed Imtiaz Humayun ] MaGNET: Uniform Sampling from Deep Generative Network Manifolds Without Retraining | ICLR 2022 https://www.youtube.com/watch?v=0Muk7nKzOW8 [Randall Balestriero, Jerome Pesenti, Yann LeCun] Learning in High Dimension Always Amounts to Extrapolation https://arxiv.org/abs/2110.09485 [Hattie Zhou] Teaching Algorithmic Reasoning via In-context Learning https://arxiv.org/abs/2211.09066 [Ahmed Imtiaz] Exact Visualization of Deep Neural Network Geometry and Decision Boundary https://openreview.net/pdf?id=VSLbmsoZxai [Beatrice Bevilacqua] Size-Invariant Graph Representations for Graph Classification Extrapolations https://arxiv.org/pdf/2103.05045.pdf [Brendan Fong David I. Spivak] Seven Sketches in Compositionality: An Invitation to Applied Category Theory https://math.mit.edu/~dspivak/teaching/sp18/7Sketches.pdf Tim’s examples of applied Category theory cited: [Lennox] Robert Rosen and Relational System Theory: An Overview https://academicworks.cuny.edu/cgi/viewcontent.cgi?article=5866&context=gc_etds [Bob Coecke] Introducing categories to the practicing physicist https://www.cs.ox.ac.uk/bob.coecke/Cats.pdf [Bob Coecke] Categorical Quantum Mechanics I: Causal Quantum Processes https://www.researchgate.net/publication/283043644_Categorical_Quantum_Mechanics_I_Causal_Quantum_Processes [Bob Coecke] Quantum Natural Language Processing https://www.cs.ox.ac.uk/people/bob.coecke/QNLP-ACT.pdf