ICML 2026
Dirichlet-Prior Shaping
Guiding expert specialization in upcycled mixture-of-experts.
Read paper
Babak Ehteshami Bejnordi
Principal Research Scientist at
NVIDIA
in Zurich, working on the Nemotron team and developing efficient architectures for large language models.
I am interested in building capable, efficient AI models and advancing the field through open and reproducible research.
Previously, I was a Senior Staff Research Scientist and Manager at
AI Research
in Amsterdam, where I led a team focused on efficient LLM architectures. My work covered mixture-of-experts models, efficient and latent reasoning, on-device LLM deployment, computer vision, multi-task learning, and continual learning. I also organized the Qualcomm Innovation Fellowship Program in Europe from 2019 to 2023.
I obtained my PhD at the Diagnostic Image Analysis Group,
Radboud University,
where I developed machine-learning algorithms for breast cancer diagnostics and organized the CAMELYON16 challenge.
From June to November 2016, I was a visiting researcher at
Harvard University,
studying tumor-associated stroma as a prognostic biomarker in breast cancer with collaborators from Harvard, NIH, and Mayo Clinic.
NVIDIA Β· Zurich, Switzerland
Research updates
- : I joined NVIDIA as a Principal Research Scientist on the Nemotron team.
- : Dirichlet-Prior Shaping: Guiding Expert Specialization in Upcycled MoEs was accepted at ICML 2026.
- : We published the Qualcomm technical report for Efficient Reasoning on the Edge.
- : KaVa: Latent Reasoning via Compressed KV-Cache Distillation was accepted at ICLR 2026.
- : We demonstrated efficient LLM reasoning at the edge at NeurIPS 2025.
- : Mixture of Cache-Conditional Experts was accepted by TMLR.
- : Invited talk on efficient deployment of LLMs on edge devices at GHOST Day in PoznaΕ, Poland.
- : Invited talks on LLM efficiency at Apple and Cisco.
- : We demonstrated Cache-MoE running efficiently on a smartphone at NeurIPS 2024.
- : We introduced mixture-of-experts inference for mobile devices with limited available DRAM.
- : We published work on efficient MoE architectures and multi-task learning at NeurIPS and BMVC.
- : Scalarization for Multi-Task and Multi-Domain Learning at Scale was accepted at NeurIPS 2023.
- : We organized the Resource-Efficient Deep Learning for Computer Vision workshop at ICCV 2023.
- : I taught efficient deep learning at DeepLearn 2023 in Bari, Italy.
- : Our efficient video object detection paper was accepted at ECCV 2022.
- : I gave a keynote at the ELLIS PhD and Postdoc Summit.
- : We open-sourced FrameExit and Skip-Convolutions.
- : Two papers were accepted at CVPR 2021: FrameExit and Skip-Convolutions.
- : I discussed conditional computation on the TWIML AI Podcast.
- : Our work on channel-gated networks for continual learning appeared at CVPR 2020.
- : Batch-Shaping for Learning Conditional Channel Gated Nets was accepted at ICLR 2020.
- : Our real-time human pose estimation work was demonstrated at NeurIPS.
- : Cancer Today published an interview about our work.
- : Our collaboration with Harvard, NIH, and Mayo Clinic on tumor-associated stroma was published.
- : I publicly defended my PhD; the recording is available.
- : Our breast-cancer metastasis detection study was published in JAMA.
- : Follow the article's Altmetric attention score.
- : CAMELYON16 won the 2016 MedicalPhit Innovation Award.
- : I gave a talk on deep learning at the Broad Institute of MIT and Harvard.
- : CAMELYON16 was cited in the White House AI strategic plan.
- : I began organizing the CAMELYON16 challenge.
Latest research
View all research
Technical report 2026
Reasoning on the Edge
Reasoning in small LLMs with LoRA adapters, supervised fine-tuning, and reinforcement learning.
Read paper
ICLR 2026
Latent Reasoning
Distilling knowledge from a compressed teacher KV-cache into a latent-reasoning student.
Read paper
TMLR 2025
Cache-MoE
Efficient mixture-of-experts inference on mobile devices with limited DRAM.
Read paper
NeurIPS 2024
Refactor LLM into MoE
Refactorizing LLMs as router-decoupled mixture-of-experts with system co-design.
Read paper
ICML workshop 2024
LLM-to-SLM
Combining large and small language models for fast autoregressive decoding.
Read paper
BMVC 2024
InterroGate for MTL
Learning to share, specialize, and prune representations for multi-task learning.
Read paper
NeurIPS 2023
Scalarization for MTL
Scalarization for multi-task and multi-domain learning at scale.
Read paper