A list of all the posts and pages found on the site. For you robots out there is an XML version available for digesting as well.
Published:
Multinomial logistic regression is a generalization of binary logistic regression to multiclass problems. This note will explain the nice geometry of the likelihood function in estimating the model parameters by looking at the Hessian of the MLR objective function.
Published:
This note describes the EM algorithm for estimating a Dirichlet distribution. One result that I found particularly interesting is the posterior distributions of gamma-distributed random variables given the observation of their corresponding Dirichlet distribution are also gamma-distributed.
Published:
This note collects a set of inequalities for smooth convex functions. As an introduction, we shall prove the convergence of gradient descent method using some of these inequalities. In particular, we show that gradient descent with fixed step size converges to a global minimum at a sublinear rate when the objective is smooth convex, and at a linear rate when strong convexity is added. It is also surprising to me that while the loose bound, $1-\mu/L$, is commonly used in standard convex analysis, the tight bound, $(1-\mu/L)^2$, is a lesser known fact and can hardly be found in notes/lectures on this topic.
Published:
We will study the proof of convergence of two well-known acceleration techniques - Heavy-Ball Method and Nesterov’s Accelerated Gradient on minimizing a convex, quadratic function.
Published:
A quick reference to the list of items that I often use in my research.
Published:
Random Matrix Theory
Published:
Some interesting thought on basic probability question. It is embarrasing that I did not give the correct answers at my first try.
Published:
List of papers I had chance to go over during ICASSP 2019 in Brighton, UK.
Published:
In this note we will study matrix perturbation theory and find out the answer to some basic questions such as what happens when adding small perturbations to a symmetric matrix, or how much the invariant subspace spanned by its eigenvectors can change. Understanding the effect of small perturbation on matrices is the key to analysis of local convergence in many optimization algorithms.
Published:
As a follow-up from my previous note on convex optimization, this note studies the so-called projected gradient descent method and its sibling, proximal gradient descent. Using the fundamental inequalities from convex analysis, we shall show that both of the methods enjoy similar convergence properties to gradient descent for unconstrained optimization.
Tran Duc-Khanh, Vu Viet-Trung, Doan The-Vinh and Nguyen Minh-Tam
IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), 2013
[Paper]@inproceedings{tran2013fuzzy,
title={Fuzzy linguistic propositional logic based on refined hedge algebra},
author={Tran, Duc-Khanh and Vu, Viet-Trung and Doan, The-Vinh and Nguyen, Minh-Tam},
booktitle={2013 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE)},
pages={1--8},
year={2013},
organization={IEEE}
}Thi-Minh-Tam Nguyen, Vu Viet-Trung, Tran Duc-Khanh and The-Vinh Doan
International Conference on Information Processing and Management of Uncertainty (IPMU) in Knowledge-Based Systems, 2014
[Paper]@inproceedings{vu2014resolution,
title={Resolution in linguistic first order logic based on linear symmetrical hedge algebra},
author={Thi-Minh-Tam Nguyen and Vu, Viet-Trung and Tran, Duc-Khanh and The-Vinh Doan},
booktitle={International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems},
pages={345--354},
year={2014},
organization={Springer}
}Thi-Minh-Tam Nguyen, Vu Viet-Trung, Tran Duc-Khanh and The-Vinh Doan
International Conference on Knowledge and Systems Engineering (KSE), 2014
[Paper]@inproceedings{vu2014resolution,
title={Resolution in linguistic propositional logic based on linear symmetrical hedge algebra},
author={Thi-Minh-Tam Nguyen and Vu, Viet-Trung and Tran, Duc-Khanh and The-Vinh Doan},
booktitle={Knowledge and Systems Engineering},
pages={327--338},
year={2014},
organization={Springer}
}Trung Vu and Raviv Raich
IEEE Statistical Signal Processing (SSP) Workshop, 2018
[Paper] [Poster] [Appendix] [PDF] [Code]@inproceedings{vu2018adaptive,
title={Adaptive Step Size Momentum Method For Deconvolution},
author={Vu, Trung and Raich, Raviv},
booktitle={IEEE Statistical Signal Processing Workshop (SSP)},
pages={438--442},
year={2018},
organization={IEEE}
}Trung Vu and Raviv Raich
IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), 2019
[Paper] [Poster] [Appendix] [PDF] [Code]@inproceedings{vu2019local,
title={Local Convergence of the {H}eavy {B}all Method in Iterative Hard Thresholding for Low-rank Matrix Completion},
author={Vu, Trung and Raich, Raviv},
booktitle={IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages={3417--3421},
year={2019},
organization={IEEE}
}Trung Vu and Raviv Raich
IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), 2019
[Paper] [Slides] [Appendix] [PDF] [Code]@inproceedings{vu2019accelerating,
title={Accelerating Iterative Hard Thresholding for Low-Rank Matrix Completion via Adaptive Restart},
author={Vu, Trung and Raich, Raviv},
booktitle={IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages={2917--2921},
year={2019},
organization={IEEE}
}Student Paper Award (2nd place)
Trung Vu, Raviv Raich and Xiao Fu
IEEE International Workshop on Machine Learning for Signal Processing (MLSP), 2019
[Paper] [Slides] [Appendix] [PDF] [Errata] [Code]@inproceedings{vu2019convergence,
title={On Convergence of Projected Gradient Descent for Minimizing a Large-Scale Quadratic Over the Unit Sphere},
author={Vu, Trung and Raich, Raviv and Xiao, Fu},
booktitle={IEEE International Workshop on Machine Learning for Signal Processing (MLSP)},
pages={1--6},
year={2019},
organization={IEEE}
}Trung Vu, Phung Lai, Raviv Raich, Anh Pham, Xiaoli Z. Fern and UK Arvind Rao
IEEE Transactions on Medical Imaging, 2020
[Paper] [Slides] [PDF]@article{vu2020novel,
title={A Novel Attribute-based Symmetric Multiple Instance Learning for Histopathological Image Analysis},
author={Vu, Trung and Lai, Phung and Raich, Raviv and Pham, Anh and Fern, Xiaoli Z and Rao, UK Arvind},
journal={IEEE Transactions on Medical Imaging},
volume={39},
number={10},
pages={3125--3136},
year={2020},
publisher={IEEE}
}Trung Vu and Raviv Raich
IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), 2021
[Paper] [Poster] [Slides] [PDF] [Code]@inproceedings{vu2021exact,
title={Exact Linear Convergence Rate Analysis for Low-Rank Symmetric Matrix Completion via Gradient Descent},
author={Vu, Trung and Raich, Raviv},
booktitle={IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages={3240--3244},
year={2021},
organization={IEEE}
}Trung Vu, Evgenia Chunikhina, and Raviv Raich
Linear Algebra and Its Applications, 2021
[Paper] [PDF] [Code]@article{vu2021perturbation,
title={Perturbation Expansions and Error Bounds for the Truncated Singular Value Decomposition},
author={Vu, Trung and Chunikhina, Evgenia and Raich, Raviv},
journal={Linear Algebra and its Applications},
year={2021},
publisher={Elsevier}
}Trung Vu and Raviv Raich
Optimization Letters, 2022
[Paper] [PDF] [Code]@article{vu2021closed,
title={A Closed-Form Bound on the Asymptotic Linear Convergence of Iterative Methods via Fixed Point Analysis},
author={Vu, Trung and Raich, Raviv},
journal={Optimization Letters},
year={2022},
publisher={Elsevier}
}Trung Vu, Raviv Raich, and Xiao Fu
Under review, 2022
[Paper]Trung Vu and Raviv Raich
IEEE Transactions on Signal Processing, 2022
[Paper] [Appendix] [PDF]@article{vu2020novel,
title={On Asymptotic Linear Convergence of Projected Gradient Descent for Constrained Least Squares},
author={Vu, Trung and Raich, Raviv},
journal={IEEE Transactions on Signal Processing},
volume={70},
pages={4061--4076},
year={2022},
publisher={IEEE}
}Trung Vu, Evgenia Chunikhina, and Raviv Raich
IEEE Transactions on Signal Processing, 2022
[Paper] [Appendix] [PDF]@article{vu2020novel,
title={On Local Linear Convergence Rate of Iterative Hard Thresholding for Matrix Completion},
author={Vu, Trung and Chunikhina, Evgenia and Raich, Raviv},
journal={IEEE Transactions on Signal Processing},
volume={70},
pages={5940--5953},
year={2022},
publisher={IEEE}
}Online Course, EECS, Oregon State University - Fall 2016, Fall 2019
Online Course, EECS, Oregon State University - Spring 2019, Summer 2019, Fall 2020, Winter 2021, Spring 2021, Fall 2021