Accuracy of Deep Learning Models Over the Years

Over the years, there were many achievements in deep learning, many of which were directly related to the ImageNet Large Scale Visual Recognition Challenge (ILSVRC, or ImageNet challenge for short). We talked about some of those milestones in deep learning in the past and how their unique innovations have helped shape the deep learning landscape today.

Today let us look at how the accuracy of these significant models has increased over the years.

Deep Learning Models Over the Years

When reporting the accuracy of classification models two accuracy measures are typically used: Top-1 Accuracy, and Top-5 Accuracy.

• Top-1 Accuracy - Where the highest probability/confidence prediction from the model matches the expected class
• Top-5 Accuracy - Where the expected class is within the top 5 predictions of the model

Based on these two matrices, we can see how the model accuracies on the ImageNet dataset has improved over the years.

Year	Model	Top-1 Accuracy	Top-5 Accuracy
2012	AlexNet	0.633	0.846
2013	ZF Net	0.64	0.853
2014	VGG16	0.713	0.901
2014	VGG19	0.714	0.91
2015	ResNet50	0.749	0.921
2015	ResNet101	0.764	0.928
2015	ResNet152	0.766	0.931
2016	ResNet50V2	0.76	0.93
2016	ResNet101V2	0.772	0.938
2016	InceptionV3	0.779	0.937
2016	ResNet152V2	0.78	0.942
2016	Xception	0.79	0.945
2017	DenseNet121	0.75	0.923
2017	DenseNet169	0.762	0.932
2017	DenseNet201	0.773	0.936
2017	InceptionResNetV2	0.803	0.953
2018	NasNet	0.825	0.96

If we visualize these numbers, we will get a chart such as the following:

Accuracy of Deep Learning Models Over the Years

We can see that the models have had a gradual but steady increase in the accuracy - in both Top-5 and Top-1 accuracies - over the years. We can see this more clearly if we graph just the trendlines:

Trendlines of Top-1 and Top-5 Accuracies of Models Over the Years

Based on these charts, we can make some interesting observations:

• Within just 6 years - from AlexNet in 2012 to NasNet in 2018 - the top-1 accuracy has increased by around 20% and the top-5 accuracy by around 12%
• The latest models are performing beyond human level accuracy. (It is measured that the accuracy level of a human in similar classification tasks is around 90-95% - Source:'Deep Residual Learning for Image Recognition')
• As the accuracy increased, the gap between top-1 accuracy and the top-5 accuracy is also narrowing. This is another indication of the consistent improvements of the models
• The top-1 accuracies of the latest models are now approaching the top-5 accuracies of the earlier models, showing how much deep learning models have improved in just a few years

There is more analysis we can perform and insights to gain from these models, which I will cover on future articles.

Stats and Referances:

Milestones of Deep Learning - https://www.codesofinterest.com/2017/07/milestones-of-deep-learning.html

Keras Applications - https://keras.io/api/applications/

ImageNet Model Stats - https://paperswithcode.com/sota/image-classification-on-imagenet

NasNet - https://medium.com/@sh.tsang/review-nasnet-neural-architecture-search-network-image-classification-23139ea0425d