JOURNAL OF THE INDIAN ACADEMY OF MATHEMATICS

(Published by Indian Academy of Mathematics since 1978)

Promoting Research in Pure and Applied Mathematics

ISSN 0970-5120


HomeArchiveContents
← Back to Volume 47, Issue 2 Contents

Multi-Time Evolution Models in Deep Learning Dynamics

Massimiliano Ferrara
Department of Law, Economics and Human Sciences - Decisions LAB, University Mediterranea of Reggio Calabria, Italy
Volume: 47Issue: 2Pages: 282–294Year: 2025ISSN: 0970-5120
⬇ Download Full PDF

Abstract

This paper extends the multi-time optimal control theory to deep learning frameworks, developing a mathematical foundation for understanding neural network training as a process evolving across multiple time scales. We formulate neural network optimization as a multi-time evolution problem where different components of the network evolve at different rates. This perspective yields two novel theorems: the first establishes conditions for path-independent convergence in multi-time gradient descent, while the second provides a framework for analyzing the interplay between feature extraction and classification layers in deep networks. Experimental validation on synthetic data demonstrates the practical implications of our theoretical framework, showing improved convergence properties and enabling adaptive learning rate scheduling based on multi-time principles. Our approach opens new avenues for understanding the dynamics of deep learning systems and suggests practical improvements to optimization algorithms.
Keywords: Multi-time dynamics; Deep learning Optimization; Multi-time Learning Algorithm (MTAG)
MSC 2020: 35F21, 49Nxx, 34A26, 68T07

How to Cite

Massimiliano Ferrara, Multi-Time Evolution Models in Deep Learning Dynamics, Journal of the Indian Academy of Mathematics, 47(2) (2025), 282–294.

References

  • Udriste, C., & Ferrara, M. (2008). Multitime Models of Optimal Growth. WSEAS Transactions on Mathematics, 7(1), 51-55.
  • Udriste, C., & Tevy, I. (2007). Multi-Time Euler-Lagrange-Hamilton Theory. WSEAS Transactions on Mathematics, 6(6), 701-709.
  • Udrişte, C. (2007). Multi-Time Controllability, Observability and Bang-Bang Principle. In Proceedings of the 6th Congress of Romanian Mathematicians.
  • Kingma, D. P., & Ba, J. (2015). Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR).
  • Tieleman, T., & Hinton, G. (2012). Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks for Machine Learning.
  • Singh, S. P., & Singh, S. (2020). Layer-specific adaptive learning rates for deep networks. In IEEE International Conference on Machine Learning and Applications (ICMLA).
  • You, Y., Li, J., Reddi, S., Hseu, J., Kumar, S., Bhojanapalli, S., Song, X., Demmel, J., Keutzer, K., & Hsieh, C. J. (2020). Large batch optimization for deep learning: Training BERT in 76 minutes. In International Conference on Learning Representations (ICLR).
  • Wang, G., Peng, J., Luo, P., Wang, X., & Lin, L. (2019). Batch Kalman normalization: Towards training deep neural networks with micro-batches. arXiv preprint arXiv:1802.03133.
  • Raghu, M., Gilmer, J., Yosinski, J., & Sohl-Dickstein, J. (2017). SVCCA: Singular vector canonical correlation analysis for deep learning dynamics and interpretability. In Advances in Neural Information Processing Systems (NeurIPS).
  • Li, H., Xu, Z., Taylor, G., Studer, C., & Goldstein, T. (2018). Visualizing the loss landscape of neural nets. In Advances in Neural Information Processing Systems (NeurIPS).
  • Smith, L. N., & Topin, N. (2018). Super-convergence: Very fast training of neural networks using large learning rates. arXiv preprint arXiv:1708.07120.
  • Kusupati, A., Ramanujan, V., Somani, R., Wortsman, M., Jain, P., Kakade, S., & Farhadi, A. (2020). Soft threshold weight reparameterization for learnable sparsity. In International Conference on Machine Learning (ICML).

© Journal of the Indian Academy of Mathematics