vector-tiles-perfomance-test

[FabianRechsteiner]

Hello,

I'm currently conducting a performance comparison of various vector tile servers. My aim is to test how long it takes to generate and serve a selected tile directly from the PostGIS database on the fly, deliberately with caching disabled.

During an initial test, I noticed that a tile request takes between 4 and 9 seconds. Is this normal, or should tiles be delivered faster? Do I need to adjust any parameters to improve performance? However, I wish to avoid caching.

Thank you for the information.

[cportele]

@FabianRechsteiner

In general, the time to generate a tile depends mainly on the time to execute the queries. So, the time will increase with the number of collections (for tiles for all collections of an API), the number of features in a tile, the number of joins necessary to generate a feature, etc.

It is therefore possible that it takes a few seconds, but in most cases it should only be milliseconds. If it takes a few seconds, this can also be a hint that the tiles include too much data and that you should filter data at smaller scales.

To be more specific, we would need more information about the dataset and the tile provider definitions.

You can increase the log level to get more information. If you set the log levels like this in cfg.yml you will get a TRACE message for every tile for a collection that is generated.

logging:
  level: OFF
  loggers:
    de.ii: INFO
    de.ii.xtraplatform.tiles.app.FeatureEncoderMVT: TRACE

For example:

Tileset UtilityInfrastructurePnt, tile WebMercatorQuad/12/1654/2459 written. Features returned: 90, written: 90, total duration: 71ms, processing: 43ms, feature processing: 7ms, merging: 0ms, encoding: 0ms, size: 7kB.

If you set to DEBUG instead of TRACE you will only see a message for every tile that takes more than a second or that is more than 250kB.

[FabianRechsteiner]

I tried to use your logging parameters in the cfg.yml file, but can't show the logs in the console.

I've only this:

ldproxy  | INFO  [2024-04-04 11:07:14,107]                avprodukt - Service with id 'avprodukt' started successfully.  
ldproxy  | INFO  [2024-04-04 11:07:19,121]                avprodukt - Tile cache seeding started  
ldproxy  | INFO  [2024-04-04 11:07:19,137]                avprodukt - Tile cache seeding finished in 0 milliseconds  
ldproxy  | WARN  [2024-04-04 11:07:51,554]                          - Feature 66499863 in tileset bo_boflaeche_mv has an invalid tile geometry in tile WebMercatorQuad/13/2861/4306. Size in pixels: 0.28798042164766274.  
ldproxy  | WARN  [2024-04-04 11:09:02,183]                          - Feature 66499863 in tileset bo_boflaeche_mv has an invalid tile geometry in tile WebMercatorQuad/13/2861/4306. Size in pixels: 0.28798042164766274.  
ldproxy  | WARN  [2024-04-04 11:09:42,982]                          - Feature 66339349 in tileset bo_boflaeche_mv has an invalid tile geometry in tile WebMercatorQuad/11/715/1075. Size in pixels: 2.8736874530665206. 
postgis  | 2024-04-04 09:12:06.969 UTC [22] LOG:  checkpoint starting: time
postgis  | 2024-04-04 09:12:06.988 UTC [22] LOG:  checkpoint complete: wrote 3 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.005 s, sync=0.003 s, total=0.019 s; sync files=2, longest=0.002 s, average=0.002 s; distance=0 kB, estimate=0 kB; lsn=2/37F71738, redo lsn=2/37F71700

I tried it with docker-image 3.6.3 and latest.

[cportele]

I tried it with 3.6.3 and I see the TRACE messages in the log. From the available information, it is unclear to me, why you do not see the DEBUG or TRACE messages.

@azahnen - Something seems to be broken in the docker hub publication, the latest image has not been updated on recent releases.

[FabianRechsteiner]

Here is my first attempt for the performance test.

I used version 3.6.3. Should I use a different version for the test?

It looks like ldproxy is quite a bit slower than bbox when displaying a lot of objects. For example, test 1 contains the land cover polygons at zoom level 14.

Is that explainable?

What do you think?

Label		Average	Min	Max	Std. Dev.	Error %	Throughput	Received KB/sec	Sent KB/sec	Avg. Bytes
BBOX
bbox_1:Test_1	100	309	0	395	17.53	0.00%	3.22914	324.77	0.46	102990
bbox_1:Test_2	100	498	0	622	39.48	0.00%	2.00156	283.22	1.16	144893
bbox_1:Test_3	100	345	0	587	44.82	0.00%	2.88875	161.29	0.42	57175
bbox_1:Test_4	100	181	0	210	10.98	0.00%	5.49089	244.49	0.79	45595
bbox_1:Test_5	100	109	0	144	8.87	0.00%	9.15332	54.26	1.35	6070
Ldproxy
ldproxy_1:Test_1	100	4166	0	6301	333.4	0.00%	0.23997	16.25	0.05	69339
ldproxy_1:Test_2	100	8602	0	9549	288.86	0.00%	0.11621	9.61	0.09	84718
ldproxy_1:Test_3	100	442	0	533	34.88	0.00%	2.25912	38.07	0.44	17255
ldproxy_1:Test_4	100	3228	0	3571	174.83	0.00%	0.30969	20.81	0.06	68820
ldproxy_1:Test_5	100	2866	0	3385	166.27	0.00%	0.34885	2.33	0.07	6835

[cportele]

It is not possible to comment on the numbers without knowing the data and the tests. If you can provide more information that would be helpful.

In addition, I do not know the BBOX software, can you provide a link?

[FabianRechsteiner]

I think, there is no solution for better performance in ldproxy because ldproxy don't use st_as_MVT to create the vector tiles. Instead the server creates ogc api features and convert them to vector tiles.
I'll document this in my masterthesis and will create a public GitHub Repository where you can read about my performancetests.

bbox is another one of a total of 6 servers that I compare for performance.

thanks for your help

[cportele]

In general, ldproxy cannot use ST_AsMVT as a feature schema can be more complex and involve joins, property transformations, etc. In addition, the specification of the tile provider may include post-processing rules to reduce the tile size (e.g., merge connected polygons).

What could be done for the simple case would be to support an additional tile provider for PostGIS backends that leverages ST_AsMVT.