Binary arithmetic
Godefroy-Guillaume Leibnitz. Explication de l’arithmétique binaire, qui se sert des
seuls caractères O et I avec des remarques sur son utilité
et sur ce qu’elle donne le sens des anciennes figures
chinoises de Fohy. (1703).
Gradient Descent (GD)
C. Lemarechal. Cauchy and the Gradient Method. Doc Math Extra, pp. 251-254. (2012)
Von Neumann Architecture
J Von Neumann. First Draft of a Report on EDVAC. (1945)
Automata
J Von Neumann, AW Burks. Theory of self-reproducing automata. (1966)
Shallow Learning(Least Squares)
Stephen M. Stigler. Gauss and the Invention of Least Squares. (1981)
Turing Machine
A. M. Turing. Intelligent Machinery. Unpublished Technical Report. (1948)
Backpropagation
H. J. Kelley. Gradient Theory of Optimal Flight Paths. ARS Journal, Vol. 30, No. 10, pp.
947-954. (1960)
Neuroscience
David Hubel, Torsten Wiesel. Receptive fields of single neurons in the cat’s striate cortex (1959)
Lawrence Roberts. Machine perception of three-dimensional solids (1963)
David Mar. Vision: A computational investigation into the human representation and processing of visual information(1982)
Hidden Markov Models
Rabiner, L. A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition.(Proceedings of the IEEE 1989)
Object Recognition
David Lowe. Object Recognition from Local Scale-Invariant Features. (1992)
Maximum Entropy
Adwait R. A Maximum Entropy Model for POS tagging. (1994)
SVM
Corinna Cortes, Vladimir Vapnik. Support-vector networks. (1995)
Statistical Machine Learning
Vladimir Vapnik. The Nature of Statistical Learning Theory. (1995)
MEMM
McCallum et al., 2000, Maximum Entropy Markov Models for Information Extraction and Segmentation (ICML'00)
CRFs
J. Lafferty et al. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling SequenceData. (ICML'01)
Perceptron
M Collins. Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms (EMNLP'02)
MNIST
LeCun et al. Gradient-based learning applied to document recognition (IEEE 1998)
Adaboost
Yoav Freund, Robert E. Schapire. Experiments with a New Boosting Algorithm. (1996)
LeNet
Yann LeCun Leon Bottou Yoshua Bengio, Patrick Haner. Gradient-Based Learning Applied to Document Recognition (1998)
DNN
Yann LeCun Leon Bottou Yoshua Bengio, Patrick Haner. Gradient-Based Learning Applied to Document Recognition (1998)
RNN
Rumelhart, David E; Hinton, Geoffrey E, and Williams, Ronald J. Learning internal representations by error propagation. (Sept. 1985)
Jordan, Michael I. Serial order: a parallel distributed processing approach. (1986)
DENDRAL
Edward A. Feigenbaum, Bruce G. Buchanan. DENDRAL and Meta-DENDRAL roots of knowledge systems and expert system applications. (1993)
LSTM
S. Hochreiter and J. Schmidhuber. Long Short-Term Memory. (1995)
EQP(Equation Prover)
William Mccune. Deep Blue. (1997)
Deap Blue
M Campbell. Solution of the Robbins Problem. (2002)
Feature
David G. Lowe. Distinctive Image Features from Scale-Invariant Keypoints (IJCV 2004)
Navneet Dalal, Bill Triggs. Histograms of Oriented Gradients for Human Detection (CVPR 2005)
Reconstruction
Noah Snavely, Steven M. Seitz, Richard Szeliski. Photo Tourism: Exploring Photo Collections in 3D. (ACM 2006)
SLAM
Davison et al. MonoSLAM: Real-Time Single Camera SLAM (TPAMI 2007)
The Four- Color Theorem
Georges Gonthier. Formal Proof—The Four- Color Theorem (2008)
Deformable Part Model
Felzenszwalb, David McAllester, Deva Ramanan. A discriminatively trained, multiscale, deformable part model. (2008)
ImageNet
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. (CVPR 2009)
PASCAL VOC
Mark Everingham et al. The PASCAL Visual Object Classes (VOC) Challenge, IJCV 2010
Graphical Models
Sebastian Nowozin and Christoph H. Lampert. Structured Learning and Prediction in Computer Vision (2011)
Random Search
J. Bergstra and Y. Bengio. Random search for hyper-parameter optimization. (2012)
CNN(Alexnet)
A. Krizhevsky, I. Sutskever, G. E. Hinton. ImageNet Classification with Deep
Convolutional Neural Networks. (2012)
Matthew D. Zeiler and Rob Fergus. Visualizing and Understanding Convolutional Networks (ECCV 2014)
VAE
Diederik P Kingma, Max Welling. Auto-Encoding Variational Bayes. (2013)
R-CNN
Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. (2013)
Word2Vec
T. Mikolov et al. Efficient estimation of word representations in vector space. (2013)
Dropout
N. Srivastava et al. Dropout: A simple way to prevent neural networks from overfitting. (2014)
Adam
D. Kingma and J. Ba. Adam: A method for stochastic optimization. (2014)
DeCAF
Boris van Breugel, Trent Kyono, Jeroen Berrevoets, Mihaela van der Schaar. DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition (ICML 2014)
GAN
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A.
Courville, Y. Bengio. Generative adversarial nets. (2014)
COCO
Tsung-Yi Lin et al. Microsoft COCO: Common Objects in Context. (2014)
FCN
Jonathan Long, Evan Shelhamer, Trevor Darrell. Fully Convolutional Networks for Semantic Segmentation. (2014)
DeepFace
Y. Taigman et al.DeepFace: Closing the gap to human-level performance in face verification. (2014)
Seq2Seq
I. Sutskever et al. Sequence to sequence learning with neural networks. (2014)
ResNet
Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun. Deep Residual Learning for Image Recognition. (2015)
Batch Normalization
S. Loffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. (2015)
YOLO
Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi. You Only Look Once: Unified, Real-Time Object Detection. (2015)
TensorFlow
M. Abadi et al. TensorFlow: Large-scale machine learning on heterogeneous distributed systems. (2016)
MCTS
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., ... & Chen, Y. Mastering the game of Go without human knowledge. (2017)
Transformer
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, I.
Polosukhin. Attention is all you need. (2017)
BERT
J. Devlin, M. W. Chang, K. Lee, K. Toutanova. Bert: Pre-training of deep
bidirectional transformers for language understanding. (2018)
Model Compression
O. Vinyals, J. A. Dean, G. E. Hinton. Distilling the Knowledge in a Neural Network. (2015)
GPT
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever. Improving Language Understanding by Generative Pre-Training. (2018)
NeRF
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, Ren Ng. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. (2020)
AlphaFold
Richard Evans et al. Protein complex prediction with AlphaFold-Multimer. (2021)
LLAMA
Touvron et al. LLaMA: Open and Efficient Foundation Language Models. (2023)
Reference
Annotated History of Modern AI and Deep Learning (Juergen Schmidhuber)
https://arxiv.org/abs/2212.11279
Classical Paper List on Machine Learning andNatural Language Processing (Zhiyuan Liu)
http://nlp.csai.tsinghua.edu.cn/~lzy/resources/paper_list_ml_nlp.html
Award-winning classic papers in ML and NLP (Desh Raj)
https://desh2608.github.io/2018-08-30-classic-papers/
Computer Vision: 10 Papers to Start (Chenxi Liu)
https://www.cs.jhu.edu/~cxliu/2015/computer-vision-10-papers-to-start.html
Awesome - Most Cited Deep Learning Papers (Terryum)
https://github.com/terryum/awesome-deep-learning-papers