Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update simdlib to 4.9.107 + image IO backend #947

Closed
wants to merge 19 commits into from

Conversation

s-trinh
Copy link
Contributor

@s-trinh s-trinh commented Nov 21, 2021

TODO:

  • add image encode/decode
  • test image encode with RealSense?
  • use SimdFree()
  • fix Java binding
  • fix build
  • check Simd CMakeLists.txt for VS
  • clean code
  • docs

Some benchmark numbers for Simd image loading/writing: ermig1979/Simd#183
The idea is to have a backend option to select which library is used for image loading/writing, with a default backend.
Always build ViSP with stb and Simd support.

ViSP images: lagadic/visp-images#17

@s-trinh
Copy link
Contributor Author

s-trinh commented Nov 22, 2021

Some benchmark results on an Intel(R) Core(TM) i5-8500 CPU @ 3.00GHz:

nThreads: 0 / available threads: 6

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
perfImageLoadSave is a Catch v2.9.2 host application.
Run with -? for options

-------------------------------------------------------------------------------
Benchmark JPEG image loading
  Grayscale
  Solvay (640x440)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:71
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libjpeg backend                                         100            1    153.83 ms 
                                                 1.53087 ms   1.52093 ms   1.54685 ms 
                                                  63.649 us    43.366 us    90.075 us 
                                                                                      
OpenCV backend                                          100            1   153.522 ms 
                                                 1.53928 ms   1.52803 ms   1.56028 ms 
                                                  76.182 us    47.529 us    121.58 us 
                                                                                      
simd backend                                            100            1   217.282 ms 
                                                 2.16545 ms   2.15691 ms   2.17847 ms 
                                                  53.005 us    39.215 us    77.594 us 
                                                                                      
stb backend                                             100            1   202.907 ms 
                                                 2.03213 ms   2.02325 ms   2.04684 ms 
                                                  57.145 us    39.354 us    83.546 us 
                                                                                      

-------------------------------------------------------------------------------
Benchmark JPEG image loading
  Grayscale
  Solvay (1024x705)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:71
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libjpeg backend                                         100            1   380.099 ms 
                                                 3.81511 ms   3.79299 ms   3.84903 ms 
                                                 137.529 us    99.835 us   185.662 us 
                                                                                      
OpenCV backend                                          100            1   374.388 ms 
                                                 3.75463 ms   3.73408 ms   3.78707 ms 
                                                 128.541 us     91.92 us   175.815 us 
                                                                                      
simd backend                                            100            1   553.773 ms 
                                                 5.58394 ms   5.55323 ms    5.6286 ms 
                                                 186.919 us   140.659 us   255.842 us 
                                                                                      
stb backend                                             100            1   501.748 ms 
                                                 5.20286 ms   5.17852 ms   5.23262 ms 
                                                 136.717 us   113.303 us   185.422 us 
                                                                                      

-------------------------------------------------------------------------------
Benchmark JPEG image loading
  Grayscale
  Solvay (1280x881)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:71
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libjpeg backend                                         100            1    631.73 ms 
                                                 6.03899 ms   6.00799 ms   6.07896 ms 
                                                  178.47 us   144.099 us   218.571 us 
                                                                                      
OpenCV backend                                          100            1   604.911 ms 
                                                 6.09227 ms   6.05483 ms   6.14877 ms 
                                                 230.392 us   168.025 us   335.251 us 
                                                                                      
simd backend                                            100            1   860.565 ms 
                                                  8.5973 ms   8.55661 ms   8.64897 ms 
                                                 232.242 us   191.017 us     284.1 us 
                                                                                      
stb backend                                             100            1   785.959 ms 
                                                 8.02501 ms   7.98438 ms   8.07126 ms 
                                                 220.451 us    190.85 us   259.298 us 
                                                                                      

-------------------------------------------------------------------------------
Benchmark JPEG image loading
  Grayscale
  Solvay (2126x1463)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:71
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libjpeg backend                                         100            1    1.90163 s 
                                                 18.9223 ms   18.8678 ms   18.9799 ms 
                                                 286.153 us   261.497 us   316.576 us 
                                                                                      
OpenCV backend                                          100            1    1.28804 s 
                                                 12.9206 ms   12.8667 ms   12.9834 ms 
                                                 296.164 us   256.522 us   340.545 us 
                                                                                      
simd backend                                            100            1    2.38737 s 
                                                 23.7065 ms   23.6442 ms   23.7698 ms 
                                                 319.122 us   287.303 us   360.779 us 
                                                                                      
stb backend                                             100            1    1.67203 s 
                                                 16.7081 ms    16.657 ms   16.7653 ms 
                                                 274.569 us   244.336 us    313.43 us 
                                                                                      

-------------------------------------------------------------------------------
Benchmark JPEG image loading
  vpRGBa
  Solvay (640x440)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:86
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libjpeg backend                                         100            1   113.583 ms 
                                                 1.13863 ms   1.12984 ms    1.1552 ms 
                                                  59.373 us    35.501 us    91.705 us 
                                                                                      
OpenCV backend                                          100            1   115.409 ms 
                                                  1.1504 ms   1.14293 ms   1.16336 ms 
                                                  48.936 us    32.111 us    72.643 us 
                                                                                      
simd backend                                            100            1   203.727 ms 
                                                 2.03611 ms   2.02539 ms   2.05454 ms 
                                                  69.887 us    44.549 us   100.997 us 
                                                                                      
stb backend                                             100            1   148.182 ms 
                                                  1.4803 ms   1.47392 ms   1.49238 ms 
                                                   43.38 us    26.092 us    67.248 us 
                                                                                      

-------------------------------------------------------------------------------
Benchmark JPEG image loading
  vpRGBa
  Solvay (1024x705)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:86
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libjpeg backend                                         100            1   270.178 ms 
                                                  2.6968 ms    2.6801 ms   2.72585 ms 
                                                 109.911 us    69.734 us   162.372 us 
                                                                                      
OpenCV backend                                          100            1   271.536 ms 
                                                 2.71411 ms   2.70235 ms   2.73386 ms 
                                                  76.413 us    51.083 us   113.358 us 
                                                                                      
simd backend                                            100            1   500.012 ms 
                                                 4.98948 ms   4.96848 ms   5.02033 ms 
                                                 129.293 us    95.491 us     173.5 us 
                                                                                      
stb backend                                             100            1   354.587 ms 
                                                 3.54776 ms   3.53081 ms    3.5756 ms 
                                                 108.405 us    75.374 us   155.765 us 
                                                                                      

-------------------------------------------------------------------------------
Benchmark JPEG image loading
  vpRGBa
  Solvay (1280x881)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:86
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libjpeg backend                                         100            1   413.999 ms 
                                                 4.16171 ms    4.1422 ms   4.19061 ms 
                                                 119.397 us    88.788 us   157.363 us 
                                                                                      
OpenCV backend                                          100            1   418.968 ms 
                                                 4.18545 ms   4.16487 ms   4.21648 ms 
                                                 126.847 us     91.33 us   169.586 us 
                                                                                      
simd backend                                            100            1   774.542 ms 
                                                 7.72429 ms   7.69719 ms   7.75931 ms 
                                                 156.402 us   125.159 us   201.363 us 
                                                                                      
stb backend                                             100            1   537.702 ms 
                                                 5.39076 ms   5.37241 ms   5.41685 ms 
                                                 110.593 us    82.173 us   148.903 us 
                                                                                      

-------------------------------------------------------------------------------
Benchmark JPEG image loading
  vpRGBa
  Solvay (2126x1463)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:86
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libjpeg backend                                         100            1    2.50561 s 
                                                 25.0041 ms    24.856 ms   25.4804 ms 
                                                 1.22316 ms   429.232 us   2.70602 ms 
                                                                                      
OpenCV backend                                          100            1   774.493 ms 
                                                 7.80326 ms   7.76956 ms   7.84689 ms 
                                                 194.743 us   158.617 us   235.724 us 
                                                                                      
simd backend                                            100            1    2.12031 s 
                                                 21.3024 ms    21.252 ms   21.3553 ms 
                                                  263.63 us   240.918 us   292.345 us 
                                                                                      
stb backend                                             100            1    1.09973 s 
                                                 10.9136 ms   10.8689 ms   10.9699 ms 
                                                 255.141 us   211.894 us   309.196 us 
                                                                                      

-------------------------------------------------------------------------------
Benchmark PNG image loading
  Grayscale
  Solvay (640x440)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:103
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libpng backend                                          100            1   350.146 ms 
                                                  3.5237 ms   3.51201 ms   3.54282 ms 
                                                  74.756 us    51.988 us    120.22 us 
                                                                                      
OpenCV backend                                          100            1    340.09 ms 
                                                 3.38435 ms   3.37199 ms   3.40619 ms 
                                                  81.402 us    51.626 us   132.913 us 
                                                                                      
simd backend                                            100            1   405.934 ms 
                                                 4.03774 ms   4.01798 ms   4.06961 ms 
                                                 125.402 us    86.191 us   178.094 us 
                                                                                      
stb backend                                             100            1   423.318 ms 
                                                  4.1425 ms   4.12411 ms    4.1698 ms 
                                                 112.698 us    83.461 us   148.369 us 
                                                                                      

-------------------------------------------------------------------------------
Benchmark PNG image loading
  Grayscale
  Solvay (1024x705)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:103
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libpng backend                                          100            1    837.92 ms 
                                                 8.42321 ms   8.38728 ms   8.47049 ms 
                                                 209.138 us   171.983 us   264.083 us 
                                                                                      
OpenCV backend                                          100            1   815.233 ms 
                                                 8.03553 ms   7.99495 ms   8.08754 ms 
                                                 232.691 us   193.181 us   286.987 us 
                                                                                      
simd backend                                            100            1    1.00572 s 
                                                 10.0892 ms   10.0456 ms   10.1422 ms 
                                                 243.851 us    207.22 us   292.914 us 
                                                                                      
stb backend                                             100            1    1.01305 s 
                                                 10.0935 ms   10.0564 ms   10.1396 ms 
                                                 209.508 us   176.807 us   250.999 us 
                                                                                      

-------------------------------------------------------------------------------
Benchmark PNG image loading
  Grayscale
  Solvay (1280x881)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:103
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libpng backend                                          100            1    1.28067 s 
                                                 12.8678 ms   12.8236 ms   12.9187 ms 
                                                 242.689 us   212.995 us   283.509 us 
                                                                                      
OpenCV backend                                          100            1    1.22785 s 
                                                 12.2091 ms   12.1642 ms   12.2624 ms 
                                                 248.986 us   215.108 us   288.119 us 
                                                                                      
simd backend                                            100            1    1.53763 s 
                                                  15.426 ms   15.3812 ms   15.4753 ms 
                                                 238.738 us   212.912 us   272.651 us 
                                                                                      
stb backend                                             100            1    1.55686 s 
                                                 15.5726 ms   15.5186 ms   15.6387 ms 
                                                  303.52 us   254.405 us   397.868 us 
                                                                                      

-------------------------------------------------------------------------------
Benchmark PNG image loading
  Grayscale
  Solvay (2126x1463)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:103
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libpng backend                                          100            1    6.25322 s 
                                                 62.3872 ms   62.3396 ms   62.4422 ms 
                                                 259.899 us   222.769 us   309.612 us 
                                                                                      
OpenCV backend                                          100            1    6.33496 s 
                                                 63.4203 ms   63.3537 ms   63.4974 ms 
                                                 365.275 us   315.874 us   421.129 us 
                                                                                      
simd backend                                            100            1     6.7109 s 
                                                 65.5325 ms   65.4641 ms   65.6072 ms 
                                                 364.607 us    323.79 us   413.523 us 
                                                                                      
stb backend                                             100            1    6.61853 s 
                                                 65.3358 ms   65.2721 ms   65.4057 ms 
                                                 341.472 us   300.571 us   393.751 us 
                                                                                      

-------------------------------------------------------------------------------
Benchmark PNG image loading
  vpRGBa
  Solvay (640x440)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:118
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libpng backend                                          100            1   401.103 ms 
                                                 3.95231 ms   3.93266 ms   3.98465 ms 
                                                 126.694 us    87.564 us   186.367 us 
                                                                                      
OpenCV backend                                          100            1   298.352 ms 
                                                 2.99827 ms    2.9854 ms   3.02011 ms 
                                                  83.791 us    55.474 us   121.649 us 
                                                                                      
simd backend                                            100            1   368.295 ms 
                                                 3.68675 ms   3.67347 ms    3.7088 ms 
                                                  85.877 us    58.818 us   127.788 us 
                                                                                      
stb backend                                             100            1   343.884 ms 
                                                 3.44575 ms   3.43125 ms   3.47255 ms 
                                                  97.442 us    60.998 us   160.303 us 
                                                                                      

-------------------------------------------------------------------------------
Benchmark PNG image loading
  vpRGBa
  Solvay (1024x705)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:118
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libpng backend                                          100            1   935.712 ms 
                                                 9.39996 ms   9.36731 ms   9.44222 ms 
                                                 189.999 us   154.711 us   237.585 us 
                                                                                      
OpenCV backend                                          100            1   691.434 ms 
                                                 6.91756 ms   6.89536 ms   6.94895 ms 
                                                 132.856 us   103.251 us   181.031 us 
                                                                                      
simd backend                                            100            1   885.977 ms 
                                                 8.88894 ms   8.86235 ms   8.92385 ms 
                                                 155.252 us    126.33 us   188.566 us 
                                                                                      
stb backend                                             100            1   837.456 ms 
                                                  8.1827 ms    8.1539 ms   8.22155 ms 
                                                 169.773 us   134.042 us   212.397 us 
                                                                                      

-------------------------------------------------------------------------------
Benchmark PNG image loading
  vpRGBa
  Solvay (1280x881)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:118
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libpng backend                                          100            1     1.4262 s 
                                                 14.1657 ms   14.1272 ms    14.213 ms 
                                                 217.707 us   187.111 us    271.48 us 
                                                                                      
OpenCV backend                                          100            1    1.02975 s 
                                                 10.3347 ms   10.3007 ms    10.378 ms 
                                                 195.195 us    161.37 us   243.971 us 
                                                                                      
simd backend                                            100            1    1.34274 s 
                                                 13.5075 ms   13.4693 ms   13.5511 ms 
                                                 208.969 us   184.415 us   238.918 us 
                                                                                      
stb backend                                             100            1    1.24153 s 
                                                 12.3461 ms   12.3101 ms   12.3889 ms 
                                                 199.307 us    172.01 us   233.778 us 
                                                                                      

-------------------------------------------------------------------------------
Benchmark PNG image loading
  vpRGBa
  Solvay (2126x1463)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:118
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libpng backend                                          100            1    6.86843 s 
                                                 68.4066 ms    68.136 ms    69.015 ms 
                                                 2.01001 ms   1.12083 ms   3.26361 ms 
                                                                                      
OpenCV backend                                          100            1     5.8158 s 
                                                 58.3884 ms   58.3145 ms   58.4693 ms 
                                                 394.014 us    353.06 us   442.032 us 
                                                                                      
simd backend                                            100            1    6.16909 s 
                                                 60.8358 ms   60.7622 ms   60.9391 ms 
                                                 439.618 us   339.444 us   691.096 us 
                                                                                      
stb backend                                             100            1    6.80366 s 
                                                 68.9638 ms   68.8775 ms   69.0549 ms 
                                                 453.582 us   396.288 us   526.457 us 
                                                                                      

-------------------------------------------------------------------------------
Benchmark JPEG image saving
  Grayscale
  Solvay (640x440)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:150
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libjpeg backend                                         100            1   96.2915 ms 
                                                 877.685 us   870.321 us   901.423 us 
                                                  60.475 us     17.59 us   133.234 us 
                                                                                      
OpenCV backend                                          100            1   89.9738 ms 
                                                 878.081 us   870.541 us   900.126 us 
                                                  60.131 us    20.778 us   129.075 us 
                                                                                      
simd backend                                            100            1   88.9506 ms 
                                                 889.542 us     881.4 us   914.133 us 
                                                  64.913 us    18.655 us   140.398 us 
                                                                                      
stb backend                                             100            1   99.9306 ms 
                                                 1.00615 ms   996.054 us   1.03009 ms 
                                                  74.878 us    37.849 us   150.132 us 
                                                                                      

-------------------------------------------------------------------------------
Benchmark JPEG image saving
  Grayscale
  Solvay (1024x705)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:150
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libjpeg backend                                         100            1   253.384 ms 
                                                 2.55933 ms   2.40204 ms   2.80262 ms 
                                                 983.231 us   707.255 us   1.35304 ms 
                                                                                      
OpenCV backend                                          100            1   212.564 ms 
                                                 2.26952 ms   2.19679 ms   2.42505 ms 
                                                 518.585 us   293.031 us   928.577 us 
                                                                                      
simd backend                                            100            1   215.716 ms 
                                                 2.16846 ms   2.15501 ms   2.18811 ms 
                                                  82.308 us    55.151 us   125.574 us 
                                                                                      
stb backend                                             100            1   411.322 ms 
                                                  3.0237 ms   2.80531 ms   3.30581 ms 
                                                 1.26399 ms   1.00505 ms   1.55145 ms 
                                                                                      

-------------------------------------------------------------------------------
Benchmark JPEG image saving
  Grayscale
  Solvay (1280x881)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:150
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libjpeg backend                                         100            1   396.384 ms 
                                                 3.55807 ms   3.39358 ms   3.82744 ms 
                                                 1.05997 ms   718.363 us   1.51247 ms 
                                                                                      
OpenCV backend                                          100            1   437.574 ms 
                                                 3.82279 ms   3.60761 ms   4.13469 ms 
                                                 1.31092 ms    980.53 us   1.71486 ms 
                                                                                      
simd backend                                            100            1   400.718 ms 
                                                 3.87371 ms   3.66018 ms   4.19463 ms 
                                                 1.31688 ms   973.879 us   1.71779 ms 
                                                                                      
stb backend                                             100            1   461.855 ms 
                                                 4.62447 ms   4.32624 ms   4.99314 ms 
                                                 1.68552 ms   1.40157 ms   1.98697 ms 
                                                                                      

-------------------------------------------------------------------------------
Benchmark JPEG image saving
  Grayscale
  Solvay (2126x1463)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:150
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libjpeg backend                                         100            1   951.051 ms 
                                                 9.20141 ms   8.83652 ms   9.78649 ms 
                                                 2.32144 ms   1.53078 ms   3.12791 ms 
                                                                                      
OpenCV backend                                          100            1   930.692 ms 
                                                  9.3856 ms   8.98869 ms   9.99423 ms 
                                                 2.46993 ms    1.7395 ms    3.2542 ms 
                                                                                      
simd backend                                            100            1   862.409 ms 
                                                 9.09231 ms   8.80068 ms   9.62528 ms 
                                                 1.95418 ms   1.11658 ms   2.88665 ms 
                                                                                      
stb backend                                             100            1   964.244 ms 
                                                 10.0385 ms   9.58585 ms   10.7053 ms 
                                                 2.76485 ms   2.03108 ms    3.4943 ms 
                                                                                      

-------------------------------------------------------------------------------
Benchmark JPEG image saving
  vpRGBa
  Solvay (640x440)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:166
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libjpeg backend                                         100            1   163.851 ms 
                                                 1.63803 ms   1.62652 ms   1.65766 ms 
                                                  75.109 us     49.67 us   108.982 us 
                                                                                      
OpenCV backend                                          100            1     162.9 ms 
                                                 1.63322 ms   1.62326 ms   1.65207 ms 
                                                  67.891 us    41.518 us   107.308 us 
                                                                                      
simd backend                                            100            1   165.616 ms 
                                                  1.6461 ms   1.63621 ms   1.66745 ms 
                                                    70.7 us     39.06 us   125.914 us 
                                                                                      
stb backend                                             100            1   164.522 ms 
                                                 1.65002 ms   1.64422 ms   1.67328 ms 
                                                  51.089 us     11.19 us   118.323 us 
                                                                                      

-------------------------------------------------------------------------------
Benchmark JPEG image saving
  vpRGBa
  Solvay (1024x705)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:166
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libjpeg backend                                         100            1   410.526 ms 
                                                  4.1218 ms   4.10457 ms   4.14554 ms 
                                                 102.595 us    79.965 us   142.059 us 
                                                                                      
OpenCV backend                                          100            1   427.438 ms 
                                                 4.38102 ms   4.34545 ms    4.4202 ms 
                                                 190.702 us   169.968 us   216.303 us 
                                                                                      
simd backend                                            100            1   419.591 ms 
                                                   4.373 ms    4.3367 ms   4.41392 ms 
                                                 196.003 us   175.939 us   212.801 us 
                                                                                      
stb backend                                             100            1   443.973 ms 
                                                 4.53653 ms   4.42161 ms    4.8874 ms 
                                                 935.774 us   358.377 us   2.04972 ms 
                                                                                      

-------------------------------------------------------------------------------
Benchmark JPEG image saving
  vpRGBa
  Solvay (1280x881)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:166
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libjpeg backend                                         100            1   699.333 ms 
                                                 6.77872 ms   6.74224 ms   6.82083 ms 
                                                 200.365 us   172.656 us   229.773 us 
                                                                                      
OpenCV backend                                          100            1   670.561 ms 
                                                 6.95875 ms   6.91667 ms    7.0014 ms 
                                                 216.329 us   201.081 us   236.669 us 
                                                                                      
simd backend                                            100            1   685.293 ms 
                                                 6.88789 ms   6.87039 ms   6.90503 ms 
                                                  88.394 us    79.001 us   103.344 us 
                                                                                      
stb backend                                             100            1   695.342 ms 
                                                 6.80199 ms   6.76979 ms   6.84084 ms 
                                                 179.274 us   151.679 us    219.31 us 
                                                                                      

-------------------------------------------------------------------------------
Benchmark JPEG image saving
  vpRGBa
  Solvay (2126x1463)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:166
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libjpeg backend                                         100            1    1.75747 s 
                                                 17.7993 ms   17.7342 ms   17.8698 ms 
                                                 345.248 us   318.302 us   373.806 us 
                                                                                      
OpenCV backend                                          100            1    1.77176 s 
                                                 17.7068 ms   17.6509 ms   17.7688 ms 
                                                 299.649 us    267.24 us   339.449 us 
                                                                                      
simd backend                                            100            1    1.78103 s 
                                                 17.8398 ms   17.7803 ms   17.9032 ms 
                                                 313.347 us   288.105 us   341.716 us 
                                                                                      
stb backend                                             100            1      1.812 s 
                                                 18.0205 ms   17.9557 ms    18.095 ms 
                                                 355.762 us   315.743 us   457.858 us 
                                                                                      

-------------------------------------------------------------------------------
Benchmark PNG image saving
  Grayscale
  Solvay (640x440)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:193
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libpng backend                                          100            1    2.59689 s 
                                                 24.7446 ms   24.0845 ms   25.7482 ms 
                                                 4.09717 ms   3.09984 ms   5.88382 ms 
                                                                                      
OpenCV backend                                          100            1    1.29945 s 
                                                 12.3875 ms    11.768 ms   13.1662 ms 
                                                 3.51075 ms    2.8447 ms   4.21644 ms 
                                                                                      
simd backend                                            100            1    1.04394 s 
                                                 10.4544 ms   10.4256 ms   10.4886 ms 
                                                 159.542 us   137.732 us   184.998 us 
                                                                                      
stb backend                                             100            1    2.24138 s 
                                                 22.4299 ms   22.3744 ms   22.5177 ms 
                                                 350.374 us   253.172 us    613.69 us 
                                                                                      

-------------------------------------------------------------------------------
Benchmark PNG image saving
  Grayscale
  Solvay (1024x705)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:193
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libpng backend                                          100            1    5.36344 s 
                                                 62.6263 ms   61.6662 ms   63.9361 ms 
                                                 5.68864 ms   4.27695 ms   7.25471 ms 
                                                                                      
OpenCV backend                                          100            1    2.86143 s 
                                                 33.6916 ms   32.2668 ms   35.3424 ms 
                                                 7.81206 ms   6.53512 ms   9.32598 ms 
                                                                                      
simd backend                                            100            1    2.84958 s 
                                                 28.4049 ms   28.1749 ms   29.3065 ms 
                                                 2.14304 ms    251.73 us   5.08036 ms 
                                                                                      
stb backend                                             100            1    5.59034 s 
                                                 55.9088 ms   55.8023 ms   56.0961 ms 
                                                 702.408 us   470.612 us   1.23841 ms 
                                                                                      

-------------------------------------------------------------------------------
Benchmark PNG image saving
  Grayscale
  Solvay (1280x881)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:193
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libpng backend                                          100            1     10.128 s 
                                                 118.446 ms   115.907 ms   121.124 ms 
                                                 13.2971 ms   11.7864 ms    16.099 ms 
                                                                                      
OpenCV backend                                          100            1    7.87345 s 
                                                 85.4054 ms   82.4818 ms   87.4564 ms 
                                                 12.3543 ms   9.60928 ms   19.1697 ms 
                                                                                      
simd backend                                            100            1    7.01451 s 
                                                 86.4711 ms   84.3409 ms   88.7274 ms 
                                                 11.2118 ms   9.75409 ms   14.0609 ms 
                                                                                      
stb backend                                             100            1    10.2213 s 
                                                 85.5909 ms    85.447 ms   85.7794 ms 
                                                 836.714 us   674.589 us   1.14326 ms 
                                                                                      

-------------------------------------------------------------------------------
Benchmark PNG image saving
  Grayscale
  Solvay (2126x1463)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:193
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libpng backend                                          100            1    33.6868 s 
                                                 391.652 ms   388.718 ms   394.586 ms 
                                                 14.9632 ms   13.4629 ms   18.4593 ms 
                                                                                      
OpenCV backend                                          100            1    10.8623 s 
                                                 121.879 ms   118.334 ms   124.661 ms 
                                                 15.9369 ms   13.0899 ms   24.2123 ms 
                                                                                      
simd backend                                            100            1    11.4928 s 
                                                 183.752 ms   177.104 ms    191.39 ms 
                                                 36.3486 ms   32.0565 ms   40.9486 ms 
                                                                                      
stb backend                                             100            1    22.8268 s 
                                                 230.279 ms   229.946 ms   230.685 ms 
                                                 1.86963 ms   1.59981 ms    2.4441 ms 
                                                                                      

-------------------------------------------------------------------------------
Benchmark PNG image saving
  vpRGBa
  Solvay (640x440)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:209
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libpng backend                                          100            1    7.68027 s 
                                                  78.204 ms   77.5251 ms   79.0794 ms 
                                                 3.91557 ms   3.12743 ms   4.71952 ms 
                                                                                      
OpenCV backend                                          100            1    3.73312 s 
                                                 31.3652 ms   29.7701 ms   33.3377 ms 
                                                 9.02796 ms   7.54849 ms   10.6042 ms 
                                                                                      
simd backend                                            100            1    2.44331 s 
                                                 24.5451 ms   24.4855 ms   24.6084 ms 
                                                 311.143 us   272.903 us   378.684 us 
                                                                                      
stb backend                                             100            1    4.20317 s 
                                                 42.0508 ms   41.9711 ms   42.1342 ms 
                                                 417.262 us   368.005 us   480.832 us 
                                                                                      

-------------------------------------------------------------------------------
Benchmark PNG image saving
  vpRGBa
  Solvay (1024x705)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:209
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libpng backend                                          100            1    17.1448 s 
                                                 203.971 ms   201.909 ms   206.373 ms 
                                                  11.348 ms   9.78414 ms   13.6768 ms 
                                                                                      
OpenCV backend                                          100            1    10.9123 s 
                                                 109.005 ms   105.759 ms    111.37 ms 
                                                 13.9317 ms   10.8837 ms   22.4991 ms 
                                                                                      
simd backend                                            100            1     6.3323 s 
                                                 63.1418 ms   62.9814 ms   63.5268 ms 
                                                 1.19132 ms   437.596 us   2.30733 ms 
                                                                                      
stb backend                                             100            1    11.4923 s 
                                                 105.728 ms   105.634 ms   105.863 ms 
                                                 568.767 us   433.951 us    857.47 us 
                                                                                      

-------------------------------------------------------------------------------
Benchmark PNG image saving
  vpRGBa
  Solvay (1280x881)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:209
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libpng backend                                          100            1    26.6938 s 
                                                 295.871 ms   294.004 ms   297.978 ms 
                                                  10.034 ms   8.63481 ms   11.9513 ms 
                                                                                      
OpenCV backend                                          100            1    10.0955 s 
                                                 115.576 ms   112.696 ms   117.893 ms 
                                                 13.1813 ms   10.4708 ms   20.4545 ms 
                                                                                      
simd backend                                            100            1    9.32062 s 
                                                 93.1716 ms    93.059 ms    93.469 ms 
                                                 873.982 us   404.694 us   1.85461 ms 
                                                                                      
stb backend                                             100            1    15.7613 s 
                                                  157.52 ms   157.365 ms    157.73 ms 
                                                 907.036 us   688.526 us   1.36923 ms 
                                                                                      

-------------------------------------------------------------------------------
Benchmark PNG image saving
  vpRGBa
  Solvay (2126x1463)
-------------------------------------------------------------------------------
ViSP-src/modules/io/test/perfImageLoadSave.cpp:209
...............................................................................

benchmark name                                  samples       iterations    estimated
                                                mean          low mean      high mean
                                                std dev       low std dev   high std dev
-------------------------------------------------------------------------------
libpng backend                                          100            1    1.06365 m 
                                                 739.741 ms   735.306 ms   743.824 ms 
                                                 21.7899 ms   18.4744 ms   29.8473 ms 
                                                                                      
OpenCV backend                                          100            1    10.5479 s 
                                                 295.327 ms   279.626 ms   311.491 ms 
                                                 81.8107 ms   74.7139 ms   91.1077 ms 
                                                                                      
simd backend                                            100            1    25.1553 s 
                                                 238.917 ms   238.321 ms    239.53 ms 
                                                 3.08736 ms    2.8611 ms   3.36123 ms 
                                                                                      
stb backend                                             100            1    43.3215 s 
                                                 428.967 ms   428.186 ms   429.836 ms 
                                                 4.21187 ms    3.7049 ms   4.92016 ms 
                                                                                      

===============================================================================
All tests passed (24 assertions in 4 test cases)

@fspindle
Copy link
Contributor

@s-trinh Your PR is impressive :-) Tell me if I can help

On my laptop running Ubuntu 18.04 I found this warnings:

In file included from /home/fspindle/visp_ws/test-pr/visp-s-trinh/modules/io/src/image/private/vpImageIoStb.cpp:56:0:
/home/fspindle/visp_ws/test-pr/visp-s-trinh/3rdparty/stb_image/stb_image_write.h: In function ‘int stbi_write_bmp_to_func(void (*)(void*, void*, int), void*, int, int, int, const void*)’:
/home/fspindle/visp_ws/test-pr/visp-s-trinh/3rdparty/stb_image/stb_image_write.h:514:32: warning: missing initializer for member ‘stbi__write_context::context’ [-Wmissing-field-initializers]
    stbi__write_context s = { 0 };

I don't know if we should initialize all the elements of the struct or just turn off this warning using -Wno-missing-field-initializers flag

@s-trinh
Copy link
Contributor Author

s-trinh commented Nov 24, 2021

Thanks! @fspindle

I will ping you back when it is finished. Don't know yet when 😃

I think it would be interesting to run the benchmarks also on Ubuntu with OpenCV installed from package, macOS and Windows? Warning, I think currently an exception is thrown if a backend is selected but not available (e.g. libjpeg, libpng on Windows).
This way, we could decide for a priority order when choosing the different backends for JPEG and PNG IO and for the default flag IO_DEFAULT_BACKEND.


On a different topic and if you have the ressource for a GSoC or an internship.
Currently builds are failing also because of Java bindings. Could be interesting, depending if there are enough works to have a student writing a full tutorial about how the Java binding works and how to use it. Plus some evolutions?


Looks like Travis builds ran out of credit? Builds on Travis are not triggered.
Wondering if people at Inria prioritise only Gitlab or also Github? Because I know that on Gitlab you can deploy your own runners for CI (I don't have any experience on it). And on Github, something similar exists: Adding self-hosted runners. Could be interesting to somehow have some runners to build and test ViSP on exotic configurations (big-endian architecture?, embedded boards?, exotic Linux distributions?).
The idea in a long term timeline would be to be able to catch build issues on exotic configurations and sooner to avoid the works needed before a new ViSP release and with Debian packaging.

@fspindle
Copy link
Contributor

Introduced with PR #982 and #1004

@fspindle fspindle closed this Jan 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants