4

A have a polygongrid on a 1x1m spacing. I have multiple XYZ files also on a 1x1 grid.

I would like to write the Z-value of the xyz-files (is in geometry and in the DB) to the polygongrid if the XY of the pointgrid is within the perimeter of the polygon (in the polygon layer).

On small files i use refFunctions "geomwithin" to do this.

Geomwithin is limited to 100k records, and I have files that easily have more records (points).

Is there a way to override the 100k limit on refFunctions? Or is there a different way to approach this problem?

CaptainAhab
  • 1,002
  • 7
  • 23
  • 2
    You could try editing the plugin source file reffunctions.py which can be found in your plugin directory. Do a simple search/replace and increase the count value for the geomwithin function and any others of interest. Then restart QGIS and hopefully the updated count will take effect. – Joseph Jan 14 '20 at 10:34
  • I'm trying that now. I did a featurecount on the polygonlayer (655327) and set the max in refFunctions.py to 750.000. There was no error message (max 100.000) in the fieldcalculator; it has been crunching for an hour now on 1 cpu. I am just wondering what the story is about the 100k. The documentations states it's maxed out at 100k to prevent loops. I don't understand the loop part. – CaptainAhab Jan 14 '20 at 12:13
  • Can't speak for the author of the plugin but shapefiles aren't really effective to act like large spatial datasets and personally, 100k features in a shapefile seems rather a lot. This is where PostGIS would be ideal. – Joseph Jan 14 '20 at 12:21
  • yes, PostGIS is the next Bastion to conquer. I'm using a geopackage for now. – CaptainAhab Jan 14 '20 at 12:48
  • Actually the issues are all in that plugin, regardless of the format. It uses very inefficient techniques for joining such as looping over every feature for every comparison, and does not utilise any spatial indices. It's ok for tiny layers, unusable for large ones.... – ndawson Jan 14 '20 at 19:35
  • @ndawson I kinda was afraid of that. Is there an alternative? – CaptainAhab Jan 14 '20 at 20:05

1 Answers1

1

I had the same question and I asked it in github. I did it and it works perfectly:

enricofer's answer in Github:

I’m helping Matthias Kuhn in porting refFunctions to core: qgis/QGIS-Enhancement-Proposals#145. These new features will go beyond that limit and will likely land in next QGIS 3.10 so the plugin is to be cosidered discontinued.

Anyway you could bypass the limit editing the file reffunctions.py located in the reffunctions plugin directory replacing in the lines the occurrences of if count < 100000: with a bigger value that fit your needs, for example if count < 400000:. Once modified the file you have to unload and reload the plugin to allow the edits to take effects.

Noura
  • 3,429
  • 3
  • 20
  • 41
César Herrera
  • 162
  • 9