Over the years, there were many achievements in deep learning, many of which were directly related to the ImageNet Large Scale Visual Recognition Challenge (ILSVRC, or ImageNet challenge for short). We talked about some of those milestones in deep learning in the past and how their unique innovations have helped shape the deep learning landscape today.
Today let us look at how the accuracy of these significant models has increased over the years.
 |
| Deep Learning Models Over the Years |
- Top-1 Accuracy - Where the highest probability/confidence prediction from the model matches the expected class
- Top-5 Accuracy - Where the expected class is within the top 5 predictions of the model
Based on these two matrices, we can see how the model accuracies on the ImageNet dataset has improved over the years.
|
Year |
Model |
Top-1 Accuracy |
Top-5 Accuracy |
|
2012 |
AlexNet |
0.633 |
0.846 |
|
2013 |
ZF Net |
0.64 |
0.853 |
|
2014 |
VGG16 |
0.713 |
0.901 |
|
2014 |
VGG19 |
0.714 |
0.91 |
|
2015 |
ResNet50 |
0.749 |
0.921 |
|
2015 |
ResNet101 |
0.764 |
0.928 |
|
2015 |
ResNet152 |
0.766 |
0.931 |
|
2016 |
ResNet50V2 |
0.76 |
0.93 |
|
2016 |
ResNet101V2 |
0.772 |
0.938 |
|
2016 |
InceptionV3 |
0.779 |
0.937 |
|
2016 |
ResNet152V2 |
0.78 |
0.942 |
|
2016 |
Xception |
0.79 |
0.945 |
|
2017 |
DenseNet121 |
0.75 |
0.923 |
|
2017 |
DenseNet169 |
0.762 |
0.932 |
|
2017 |
DenseNet201 |
0.773 |
0.936 |
|
2017 |
InceptionResNetV2 |
0.803 |
0.953 |
|
2018 |
NasNet |
0.825 |
0.96 |
If we visualize these numbers, we will get a chart such as the following:
 |
| Accuracy of Deep Learning Models Over the Years |
We can see that the models have had a gradual but steady increase in the accuracy - in both Top-5 and Top-1 accuracies - over the years. We can see this more clearly if we graph just the trendlines:
 |
| Trendlines of Top-1 and Top-5 Accuracies of Models Over the Years |
Based on these charts, we can make some interesting observations:
- Within just 6 years - from AlexNet in 2012 to NasNet in 2018 - the top-1 accuracy has increased by around 20% and the top-5 accuracy by around 12%
- The latest models are performing beyond human level accuracy. (It is measured that the accuracy level of a human in similar classification tasks is around 90-95% - Source:'Deep Residual Learning for Image Recognition')
- As the accuracy increased, the gap between top-1 accuracy and the top-5 accuracy is also narrowing. This is another indication of the consistent improvements of the models
- The top-1 accuracies of the latest models are now approaching the top-5 accuracies of the earlier models, showing how much deep learning models have improved in just a few years
There is more analysis we can perform and insights to gain from these models, which I will cover on future articles.
Stats and Referances:
Milestones of Deep Learning - https://www.codesofinterest.com/2017/07/milestones-of-deep-learning.html
Keras Applications - https://keras.io/api/applications/
ImageNet Model Stats - https://paperswithcode.com/sota/image-classification-on-imagenet
No comments:
Post a Comment