Forrest Iandola

PhD in EECS, University of California, Berkeley, 2016.
B.S. in Computer Science, University of Illinois at Urbana-Champaign, 2012.
Illinois Math and Science Academy, 2008.

Forrest Iandola completed a PhD in EECS at UC Berkeley, where his research focused on squeezing deep neural networks onto small devices. As part of his dissertation research, he developed the energy-efficient SqueezeNet neural network. His advances in deep learning led to the founding of DeepScale, which was acquired by Tesla in 2019. He is currently an AI Research Scientist at Meta.

Research and Industry Experience

Meta
AI Research Scientist, 2022 to present

After spending a few years on self-driving cars, robotics, drones, and startups, I'm excited to get back to my dissertation research topic of "what's the best way to squeeze state-of-the-art AI onto tiny computers?"
I'm in an on-device AI research team in Reality Labs, and we're making AI run fast & efficiently on AR/VR headsets.

Anduril Industries
Head of Perception, 2020 to 2022

Led computer vision and AI engineering for Anduril's drones and our counter-drone system.
Helped to win a billion-dollar deal with the US Special Operations Command.
Developed & deployed an AI-enabled autofocus system for Anduril's counter-drone system that can see drones in the sky miles away.

Tesla, Palo Alto, CA
Sr Staff Machine Learning Scientist, 2019 to 2020

Autopilot.

DeepScale, Mountain View, CA
Co-founder and CEO, 2015 to 2019

Led a team of 30+ people to develop computationally-efficient computer vision and perception software for road vehicles.
We recruited some of the world's best engineers in deep learning and high-performance computing.
Raised 3 rounds of venture funding.
Acquired by Tesla.

University of California, Berkeley, CA
Graduate Student Researcher, 2012 to 2016

Worked with Kurt Keutzer on resource-efficient deep learning, ranging from training on hundreds of GPUs, to deploying on embedded platforms.
Developed the popular SqueezeNet deep neural network architecture.
Dissertation topic: Exploring the Design Space of Deep Convolutional Neural Networks at Large Scale

2006 -- 2014: internships at Microsoft Research, NVIDIA, SLAC, LLNL, and Fermilab

Selected Publications

Yunyang Xiong, Bala Varadarajan*, Lemeng Wu*, Xiaoyu Xiang, Fanyi Xiao, Chenchen Zhu, Xiaoliang Dai, Dilin Wang, Fei Sun, Forrest Iandola, Raghuraman Krishnamoorthi, Vikas Chandra
EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything
arXiv, 2023 (Paper) (Project Page) (LeCun Tweet)

Yang Li, Liangzhen Lai, Yuan Shangguan, Forrest N. Iandola, Ernie Chang, Yangyang Shi, Vikas Chandra
Folding Attention: Memory and Power Optimization for On-Device Transformer-based Streaming Speech Recognition
arXiv, 2023 (Paper)

Yangyang Shi, Gael Le Lan, Varun Nagaraja, Zhaoheng Ni, Xinhao Mei, Ernie Chang, Forrest Iandola, Yang Liu, Vikas Chandra
Enhance audio generation controllability through representation similarity regularization
arXiv, 2023 (Paper)

Gael Le Lan, Varun Nagaraja, Ernie Chang, David Kant, Zhaoheng Ni, Yangyang Shi, Forrest Iandola, Vikas Chandra
Stack-and-Delay: a new codebook pattern for music generation
arXiv, 2023 (Paper)

Forrest N. Iandola, Albert E. Shaw, Ravi Krishna, Kurt W. Keutzer
SqueezeBERT: What can computer vision teach NLP about efficient neural networks?
EMNLP SustaiNLP Workshop, 2020 (Paper) (Code in Hugging Face Transformers) (bibtex) (Slides) (Video) (News article in VentureBeat)

Albert Shaw, Daniel Hunter, Forrest Iandola, Sammy Sidhu
SqueezeNAS: Fast neural architecture search for faster semantic segmentation
ICCV Neural Architects Workshop, 2019 (Paper) (Code on GitHub) (bibtex) (News article in EE Times)

Forrest Iandola, Kurt Keutzer
Small Neural Nets Are Beautiful: Enabling Embedded Systems with Small Deep-Neural-Network Architectures
Keynote at ESWEEK, 2017 (Paper) (bibtex)

Bichen Wu, Forrest Iandola, Peter H. Jin, Kurt Keutzer
SqueezeDet: Unified, Small, Low Power Fully Convolutional Neural Networks for Real-Time Object Detection for Autonomous Driving
CVPR Embedded Vision Workshop, 2017 (Paper) (Code on GitHub) (bibtex)

Forrest Iandola
Exploring the Design Space of Deep Convolutional Neural Networks at Large Scale
UC Berkeley PhD Dissertation, 2016 (Dissertation on arXiv) (Slides) (bibtex)

Forrest N. Iandola, Song Han, Matthew W. Moskewicz, Khalid Ashraf, William J. Dally, and Kurt Keutzer
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 0.5MB model size
arXiv, 2016 (Paper) (Code on GitHub) (bibtex)

Forrest N. Iandola, Khalid Ashraf, Matthew W. Moskewicz, and Kurt Keutzer
FireCaffe: near-linear acceleration of deep neural network training on compute clusters
Computer Vision and Pattern Recognition (CVPR), 2016 (Paper)

Hao Fang, Saurabh Gupta, Forrest Iandola, Rupesh Srivastava, Li Deng, Piotr Dollar, Jianfeng Gao, Xiaodong He, Margaret Mitchell, John C. Platt, C. Lawrence Zitnick, Geoffrey Zweig
From Captions to Visual Concepts and Back
Computer Vision and Pattern Recognition (CVPR), 2015 (Paper)

Ross Girshick, Forrest Iandola, Trevor Darrell, and Jitendra Malik
Deformable Part Models are Convolutional Neural Networks
Computer Vision and Pattern Recognition (CVPR), 2015 (Paper) (Code on GitHub)

Ning Zhang, Ryan Farrell, Forrest Iandola, and Trevor Darrell
Deformable Part Descriptors for Fine-grained Recognition and Attribute Prediction
International Conference on Computer Vision (ICCV), 2013 (Paper) (Project Page)

Forrest N. Iandola, David Sheffield, Michael Anderson, Phitchaya Mangpo Phothilimthana, and Kurt Keutzer
Communication-Minimizing 2D Convolution in GPU Registers
IEEE International Conference on Image Processing (ICIP), 2013 (Paper) (Slides) (Code on GitHub)

Selected Talks

Albert Shaw, Daniel Hunter, Sammy Sidhu, Forrest Iandola
Squeezing down the computing requirements of deep neural networks
IEEE Santa Clara Valley Section, 2019 (Slides)

Albert Shaw, Daniel Hunter, Sammy Sidhu, Forrest Iandola
SqueezeNAS: Fast neural architecture search for faster semantic segmentation
CVPR Workshop on Autonomous Driving, 2019 (Invited Talk) (Slides)

Forrest Iandola, Kurt Keutzer, Joseph E. Gonzalez
Survey of Enabling Technologies for Automated Driving
Short Course at AutoSens, 2019 (Slides)

Forrest Iandola, Kurt Keutzer, Joseph E. Gonzalez
Developing Enabling Technologies for Automated Driving
Short Course at Electronic Imaging, 2019 (Slides) (Video)

Forrest Iandola
Tips and Tricks for Developing Smaller Neural Nets
CVPR Workshop on Efficient Deep Learning for Computer Vision, 2018 (Invited Talk) (Slides) (Video)

Forrest Iandola
Scaling the Training of Deep Neural Networks
Guest Lecture in CS267 at UC Berkeley, 2018 (Video)

Forrest Iandola and Kurt Keutzer
Small Deep-Neural-Networks: Their Advantages and Their Design
ICML TinyML Workshop, 2017 (Invited Talk) (Slides) (Video)

Forrest Iandola
Distributed deep neural network training: A measurement study
UC Berkeley CS268 class project, May 2016 (Paper) (Slides)

Contact

Email: forrest.dnn@gmail.com

My Blog

My Old Blog