Subscribe to this thread
Home - General / All posts - Geopackage export speed
gjsa42 post(s)
#13-Jul-18 08:15

Have noticed that exporting a very large layer to geopackage is extremely fast if the layer is dissolved/grouped into few features (as compared to exporting to a shapefile).

However, when the layer is decomposed into millions of individual rows, exporting seems to be row-at-a-time and is much slower than exporting to shapefile: basically, things slow to a crawl.

I have asked a similar question before when exporting into a Postgres DB, and in that case, increasing the transaction size for each background INSERT operation was not possible, so I found a workaround.

Is the same limitation unavoidable when writing many rows into a geopackage ?

Dimitri

5,083 post(s)
#13-Jul-18 08:27

See the discussion in the GPKG topic:

"GPKG is a slow format for large numbers of objects. As a spatial DBMS it is far slower than PostgreSQL. It is fine as an interchange format for larger numbers of objects but only if we understand that after we import from GPKG we will work with the data either within Manifold (superfast) or save the data within a fast DBMS like PostgreSQL."

adamw


8,216 post(s)
#23-Jul-18 08:54

GPKG does, indeed, have various limitations that affect performance. However, in this particular case (exporting vector data) we think we can make things faster. We'll try to do so.

adamw


8,216 post(s)
#25-Jul-18 08:44

A follow-up: try 9.0.167.7.

Manifold User Community Use Agreement Copyright (C) 2007-2017 Manifold Software Limited. All rights reserved.