TODO: an important technique for applications of neural networks: "Decoupled Neural Interfaces using Synthetic Gradients" from Google Deepmind (Jaderberg et al., 2016)

Beyond back-propagation Edit

Border Pairs Method and Bipropagation.

Classification region Edit

Fawzi et al. (2017)[1] show that neural nets' classification regions are connected.

Activation functions Edit

Sigmoid Edit

Reference about modeling probablility distribution: Baum and Wilczek (1988)[2]

Optimization Edit

Hardware Edit

Phase-change memory is potentially faster than GPU (shown via partial implementation + simulation).[3]

References Edit

  1. Fawzi, Alhussein, Seyed-Mohsen Moosavi-Dezfooli, Pascal Frossard, and Stefano Soatto. "Classification regions of deep neural networks." arXiv preprint arXiv:1705.09552 (2017).
  2. E.B. Baum and F. Wilczek. Supervised Learning of Probability Distributions by Neural Net- works. Neural Information Processing Systems, American Institute of Physics, 1988.
  3. "Training a neural network in phase-change memory beats GPUs". Arstechnica.