2

I'm doing an intersection between buffers around street centerlines and census block group geoms.

The extent and density of features is fairly large so I'm running into memory limitations consistent with Getting topology or out of memory errors with large dataset intersects and spatial joins in ArcMap.

I was able to avoid the memory error by reducing the buffer size but may need to find an actual solution.

Looking at the output of the successful intersection, the problem with my code is that the intersection is calculating not just the intersection of buffers with census geoms but also the intersection of the buffers with themselves, which raises the complexity enormously.

The self-intersection is pretty evident in the screen grab below. That's a selection of all the intersection output features for a single buffer. While my goal is to have a few peices where it intersects the census geoms, it clearly is being cut up by the other buffers from the same layer.

Has anyone found a way to avoid the self-intersection?

Obviously I could dissolve after calculating the intersection but that doesn't improve the efficiency of the calculation.

I don't see a better option here but maybe I'm not thinking creatively enough.

As a note, the dice tool isn't helpful I don't think because the problem is the number of features not the complexity.

self_intersection

PolyGeo
  • 65,136
  • 29
  • 109
  • 338
Hugh_Kelley
  • 747
  • 1
  • 6
  • 19
  • https://gis.stackexchange.com/questions/371783/zonal-statistics-2-ends-up-causing-a-fatal-error-is-there-an-alternative/371790#371790 – FelixIP Dec 07 '20 at 18:56

1 Answers1

1

You have sort of shot yourself in the foot by buffering all your roads and storing the buffers in a single FeatureClass. You either accept the issue and do the dissolve after processing as you have suggested or filter before you run the intersection.

A simple model iterating over buffers and feeding them one at a time into the intersection tool will generate the data you require. This may be even slower but it wont be generating all those self-intersections.

You could look at parallelizing the problem by using the multiprocessing module in Python, if you take a scripting approach.

A census block I assume will intersect multiple buffers, and double count them.

PolyGeo
  • 65,136
  • 29
  • 109
  • 338
Hornbydd
  • 43,380
  • 5
  • 41
  • 81
  • thanks, filtering would involve creating a new feature class for each buffer not just a selection right? it would be 5000+ feature classes created (and deleted) which would be a pretty tremendous hit to execution time I think. This is confined to a desktop toolbox which i think limits my options in terms of multiprocessing. I don't care about double counting because I'm using population percentages as a contextual attribute for the street. – Hugh_Kelley Dec 07 '20 at 16:18
  • No if you use the feature selection iterator you iterate over say each buffer polygon by ID, these create a "virtual" layer of 1 geometry (the current buffer), nothing is written to any workspace. Have added link in my answer. – Hornbydd Dec 07 '20 at 16:22
  • oh I see. Yeah that would be pretty helpful but, from the docs: "This tool is intended for use in ModelBuilder and not in Python scripting." https://desktop.arcgis.com/en/arcmap/10.3/tools/modelbuilder-toolbox/iterate-feature-selection.htm I'ts probably the model builder version of an arcpy cursor. – Hugh_Kelley Dec 07 '20 at 16:24
  • Yes I'm suggesting a modelbuilder approach, path of least resistance! Otherwise you can script it and then the equivalent to the iterator would be a search cursor. – Hornbydd Dec 07 '20 at 16:25
  • In the time we have had this discussion you could have knocked out the model and started it running. :) – Hornbydd Dec 07 '20 at 16:27
  • not an option for this unfortunately :) – Hugh_Kelley Dec 07 '20 at 16:28