01
Topics of Interest
Predictive Theory for Efficient Learning: Scaling laws for efficient models, compute-optimal training and inference, predicting capability under resource constraints, optimization and learning dynamics, generalization under limited compute or data, phase transitions in efficient learningSparsity, Compression, and Model Structure: Foundations of pruning and quantization, sparse and modular neural networks, lottery tickets and subnetworks, adaptive computation, neural architecture design, representation learning for efficiencyEfficient Foundation Models: Efficient LLMs and multimodal models, efficient reasoning and adaptive inference, mixture-of-experts and modular architectures, test-time adaptation, memory-efficient training and inference, distillation and compressionFoundations and Future Directions: Theoretical limits of efficient AI, new efficiency metrics and benchmarks, predictive models of training dynamics, interpretability of efficient models, mathematical foundations of efficient deep learning, emerging theoretical paradigms for efficient AI