-
-
Notifications
You must be signed in to change notification settings - Fork 134
Aperture photometry benchmarks
Brigitta Sipocz edited this page Jun 25, 2014
·
5 revisions
Benchmarks were run for all four aperture types as before, on my laptop. The new numbers are:
| Benchmark name | 1st run | 2nd run | 3rd run | 4th run |
|---|---|---|---|---|
| "Small data, single small aperture" | 10.2264 s | 11.7900 s | 10.1574 s | 10.1583 s |
| "Big data, single small aperture" | 10.6584 s | 10.6441 s | 10.3408 s | 10.2445 s |
| "Big data, single big aperture" | 2.8011 s | 2.7938 s | 2.7646 s | 2.6979 s |
| "Small data, multiple small apertures" | 7.6734 s | 6.6951 s | 6.8791 s | 6.5290 s |
| "Big data, multiple small apertures" | 8.5731 s | 6.6724 s | 6.8320 s | 6.5712 s |
| "Big data, multiple big apertures" | 30.3976 s | 27.2106 s | 27.2461 s | 26.6245 s |
| "Big data, multiple small apertures, multiple per object" | 6.4451 s | 5.4585 s | 5.4405 s | 5.4374 s |
Benchmarks were run for all four aperture types, CircularAperture, CircularAnnulus, EllipticalAperture, EllipticalAnnulus. Four runs were done in this comparison, for each branch. It was run on my laptop, thus only good for guidance. Here are the summary tables for the total running times:
| Benchmark name | 1st run | 2nd run | 3rd run | 4th run |
|---|---|---|---|---|
| "Small data, single small aperture" | 11.6258 s | 10.9604 s | 10.6340 s | 10.6611 s |
| "Big data, single small aperture" | 11.3822 s | 10.8941 s | 10.6920 s | 10.5515 s |
| "Big data, single big aperture" | 4.5762 s | 3.9235 s | 4.1338 s | 4.1499 s |
| "Small data, multiple small apertures" | 9.4818 s | 9.6106 s | 9.4838 s | 10.5441 s |
| "Big data, multiple small apertures" | 9.8553 s | 9.7617 s | 10.1435 s | 10.3433 s |
| "Big data, multiple big apertures" | 39.5159 s | 39.6284 s | 39.8242 s | 40.5670 s |
| "Big data, multiple small apertures, multiple per object" | 6.3052 s | 6.2985 s | 6.2413 s | 6.2955 s |
| Benchmark name | 1st run | 2nd run | 3rd run | 4th run |
|---|---|---|---|---|
| "Small data, single small aperture" | 9.9452 s | 10.3705 s | 10.3011 s | 10.2522 s |
| "Big data, single small aperture" | 11.0741 s | 10.3229 s | 9.7764 s | 10.0596 s |
| "Big data, single big aperture" | 4.4999 s | 4.0357 s | 3.9608 s | 3.9521 s |
| "Small data, multiple small apertures" | 9.1032 s | 8.8498 s | 8.7276 s | 8.8531 s |
| "Big data, multiple small apertures" | 9.0083 s | 9.1873 s | 8.7937 s | 9.2207 s |
| "Big data, multiple big apertures" | 39.1573 s | 41.8475 s | 41.5056 s | 38.7509 s |
| "Big data, multiple small apertures, multiple per object" | 7.1534 s | 7.0785 s | 7.7893 s | 7.3863 s |
This is the reference, the new aperture photometry should provide at least this speed...
===============================================================================
Small data, single small aperture (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
0.29342
0.29035
0.27718
0.27693
-------------------------------------------------------------------------------
Real time: 1.1381 s
===============================================================================
Big data, single small aperture (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
0.27986
0.27906
0.28114
0.27852
-------------------------------------------------------------------------------
Real time: 1.1188 s
===============================================================================
Big data, single big aperture (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
1.15869
1.08271
1.30651
1.08762
-------------------------------------------------------------------------------
Real time: 0.0466 s
===============================================================================
Small data, multiple small apertures (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
227.71692
216.22896
215.76095
217.49997
-------------------------------------------------------------------------------
Real time: 0.8775 s
===============================================================================
Big data, multiple small apertures (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
219.69414
219.21515
221.50898
224.71499
-------------------------------------------------------------------------------
Real time: 0.8855 s
===============================================================================
Big data, multiple big apertures (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
108.48308
106.91285
108.60801
107.52988
-------------------------------------------------------------------------------
Real time: 0.4318 s
===============================================================================
Big data, multiple small apertures, multiple per object (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
1651.24488
1678.32494
1712.43191
1669.92998
-------------------------------------------------------------------------------
Real time: 6.7122 s
Speed-wise it looks OK to me now, the new version is 10-20% faster than the previous one. (Note: except the last benchmark, multi object multi aperture, but that is a feature not yet supported and benchmark contains a workaround loop, which is obviously slow).
===============================================================================
Small data, single small aperture (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
0.23080
0.21788
0.22326
0.21778
-------------------------------------------------------------------------------
Real time: 0.8900 s
===============================================================================
Big data, single small aperture (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
0.21943
0.21815
0.23334
0.25873
-------------------------------------------------------------------------------
Real time: 0.9307 s
===============================================================================
Big data, single big aperture (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
1.01769
0.94161
1.04930
0.96459
-------------------------------------------------------------------------------
Real time: 0.0401 s
===============================================================================
Small data, multiple small apertures (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
175.69017
183.07710
167.13381
168.54095
-------------------------------------------------------------------------------
Real time: 0.6947 s
===============================================================================
Big data, multiple small apertures (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
169.42596
168.99300
168.77103
169.86012
-------------------------------------------------------------------------------
Real time: 0.6773 s
===============================================================================
Big data, multiple big apertures (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
92.05103
91.83693
97.15414
94.03992
-------------------------------------------------------------------------------
Real time: 0.3758 s
===============================================================================
Big data, multiple small apertures, multiple per object (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
1800.44007
1765.30290
1770.73002
1764.59408
-------------------------------------------------------------------------------
Real time: 7.1014 s
After commit c013ceca1de9ece4cb1cc8ef3a458ce16fe52124 in PR #68.
===============================================================================
Small data, single small aperture (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
0.23521
0.21828
0.22359
0.21950
-------------------------------------------------------------------------------
Real time: 0.8969 s
===============================================================================
Big data, single small aperture (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
0.21999
0.21925
0.21912
0.21923
-------------------------------------------------------------------------------
Real time: 0.8778 s
===============================================================================
Big data, single big aperture (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
0.99361
0.96290
1.11749
1.21059
-------------------------------------------------------------------------------
Real time: 0.0434 s
===============================================================================
Small data, multiple small apertures (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
10437.15715
10862.54001
10668.58411
10410.96878
-------------------------------------------------------------------------------
Real time: 42.3807 s
===============================================================================
Big data, multiple small apertures (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
10217.87190
10077.44002
10341.56704
10353.68705
-------------------------------------------------------------------------------
Real time: 40.9920 s
===============================================================================
Big data, multiple big apertures (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
232.01203
228.70088
230.95989
220.44110
-------------------------------------------------------------------------------
Real time: 0.9138 s
===============================================================================
Big data, multiple small apertures, multiple per object (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
109862.28395
...
Obviously this won't be committed, there are a few stashed copies only to know what to avoid...
===============================================================================
Small data, single small aperture (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
1.08160
0.93383
0.92625
0.91250
-------------------------------------------------------------------------------
Real time: 3.8544 s
===============================================================================
Big data, single small aperture (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
0.95975
0.91790
0.91985
0.91352
-------------------------------------------------------------------------------
Real time: 3.7113 s
===============================================================================
Big data, single big aperture (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
1891.39588
1875.79451
1456.97291
1517.30680
-------------------------------------------------------------------------------
Real time: 67.4154 s
===============================================================================
Small data, multiple small apertures (milliseconds)
subpixels =
1
5
10
exact
-------------------------------------------------------------------------------
circ
12679.45695
13038.23519
14621.58203
12994.71903
-------------------------------------------------------------------------------
Real time: 53.3345 s
===============================================================================
...