B.E. (Computer Engineering) Optimization in Machine Learning Syllabus - Mumbai University 2026
The University has moved this degree onto NEP 2020 one year at a time. The first and second years are NEP 2020 syllabi; the third and fourth years are still examined on the REV-2019 'C' Scheme, which is what the University sets for them this year.
Loading syllabus...
Syllabus for Optimization in Machine Learning
Module 1: Introduction and Background to Optimization Theory
- 1.1 Basic Ingredients of Optimization Problems, Optimization Problem Classifications, Optima Types, Optimization Method Classes, Overview of Unconstrained and Constrained Optimization, Basics of convex optimization
Module 2: Derivative based Optimization
- 2.1 The Basics of Optimization (univariate, bivariate and multivariate optimization), Convex Objective Functions
- 2.2 First-Order optimization Methods : Gradient Descent, Conjugate Gradient, Momentum, Nesterov Momentum, Adagrad, RMSProp, learning rate optimization
- 2.3 Second order optimization: Newton method
Module 3: Stochastic Methods
- Noisy Descent, Mesh Adaptive Direct Search, Cross-Entropy Method, Natural Evolution Strategies, Covariance Matrix Adaptation
Module 4: Convex Optimization
- Optimization problems, Convex optimization, Linear optimization problems, Quadratic optimization problems, Geometric programming, Overview of Generalized inequality constraints and Vector optimization
Module 5: Evolutionary Methods
- Introduction to Evolutionary Computation: Generic Evolutionary Algorithm, Representation: The Chromosome, 5.1 Initial Population, Fitness Function, Selection: Selective Pressure, Random Selection, Proportional Selection,Tournament Selection, Rank-Based Selection, Elitism and Evolutionary Computation versus Classical Optimization, Stopping conditions Canonical Genetic Algorithm, Binary Representations of 5.2 Crossover and Mutation: Binary Representations, Control Parameters
Module 6: Advance Evolutionary Methods
- Basic Particle Swarm Optimization, Global Best PSO, Local Best PSO, g-best versus l-best PSO, Velocity Components, 6.1 Geometric Illustration, Algorithm Aspects, Social Network Structures Ant Colony Optimization Meta-Heuristic, Foraging Behavior
- 6.2 of Ants, Stigmergy and Artificial Pheromone, Simple Ant Colony Optimization, Ant System, Ant Colony System
Textbooks
- 1 Mykel J. Kochenderfer, Tim A.Wheeler, Algorithms for Optimization, MIT Press (2019)
- 2 Andries P Engelbrecht, Computational Intelligence-An Introduction, Second-Edition, Wiley publication
- 3 Charu C. Aggarwal, Linear Algebra and Optimization for Machine Learning, , Springer ,2020.
- 1 SuvritSra, Sebastian Nowozin, Stephen J. Wright, Optimization for Machine Learning, The MIT Press
- 2 Xin-She Yang Middlesex ,Optimization techniques and applications with examples, Wiley
- 3 A.E. Eiben, J. E. Smith, Introduction to Evolutionary Computing, Springer
- 1 Convex optimization (NPTEL)
- 2 Constrained and Unconstrained optimization (NPTEL)
- 3 Machine-learning-model-performance (Coursera)
- 4 Deep-neural-network optimization (Coursera)
Reproduced from the University of Mumbai syllabus for B.E. (Computer Engineering) under REV-2019 'C' Scheme, in force from the academic year 2022-23. Wording is as printed in that syllabus. Module numbering is as printed there too.
The complete syllabus
This subject is cut from the University circular for its year. Open a document here if you want the whole thing rather than a single subject.