1

I'm executing a python script which does these steps:

  1. iterates through 19 features (makes pairs "each one with each other", order doesn't matter: A-B = B-A)
  2. Creates Closest Facility Layer
  3. Adds location points
  4. Solves Closest Facility route
  5. Exports into single .shp for each pair

I need to get a polyline for each of the pairs of 19 combinations. This will result (if I count correctly) in 19*18/2 = 174 shapefiles. When I run the script, it works fine until about 40th pair. Then it gets slower and slower. I let it run for about 3 hours and I got 142th pair...then ArcMap crashed with an error screen saying all my work is lost. If it ended alright I would then merge all the shapefiles into a single one with 174 features in it.

Is there any way to make the script more efficient or a way how to solve this problem?

This is the script:

 from collections import defaultdict
 import itertools
 import pprint

 import arcpy
 from arcpy import env
 import os

 ddict = defaultdict(list)
 for start,end in itertools.combinations(range(19), 2):
ddict[start].append(end)

try:
arcpy.CheckOutExtension("Network")
env.workspace = r"D:/Dokumenty/GISdata/PID"
env.overwriteOutput = True


inNetworkDataset = r"D:/Dokumenty/GISdata/PID/New/silnice_useky_krnap_ND.nd"
impedanceAttribute = "Length"
accumulateAttributeName = ["Length"]

for key,value in ddict.items():
    for v in value:
        inFacilities = r"PID/New/Obce_body_luzka" # melo by byt key
        inIncidents = r"PID/New/Obce_body_luzka" # melo by byt v
        outNALayerName = "Dvojice_bodu_{}_{}".format(key, v)
        outLayerFile = os.path.join(r"D:/Dokumenty/GISdata/output2", outNALayerName + ".lyr")
        NAResultObject = arcpy.na.MakeClosestFacilityLayer(inNetworkDataset,outNALayerName,impedanceAttribute,"TRAVEL_TO","",1, accumulateAttributeName,"NO_UTURNS")


        outNALayer = NAResultObject.getOutput(0)


        subLayerNames = arcpy.na.GetNAClassNames(outNALayer)
        #Stores the layer names that we will use later
        facilitiesLayerName = subLayerNames["Facilities"]
        incidentsLayerName = subLayerNames["Incidents"]


        import arcpy

        intable = "D:\Dokumenty\GISdata\PID\New\Obce_body_luzka.dbf"

        fields = arcpy.ListFields(intable)

        fieldinfo = arcpy.FieldInfo()


        for field in fields:
            fieldinfo.addField(field.name, field.name, "VISIBLE", "")
        arcpy.MakeTableView_management(intable, "temp_table", "FID = {}".format (key), "", fieldinfo)
        arcpy.MakeTableView_management(intable, "temp_table2", "FID = {}".format (v), "", fieldinfo)
        fieldMappings = arcpy.na.NAClassFieldMappings(outNALayer, facilitiesLayerName, False, fields) 
        arcpy.na.AddLocations(outNALayer, facilitiesLayerName, "temp_table", fieldMappings, "")
        arcpy.na.AddLocations(outNALayer, incidentsLayerName, "temp_table2", "", "")

        arcpy.na.Solve(outNALayer)

                arcpy.management.SaveToLayerFile(outNALayer,outLayerFile,"RELATIVE")
        routesLayer = arcpy.mapping.ListLayers(outNALayer, "routes")[0]
        arcpy.FeatureClassToShapefile_conversion(routesLayer, r"D:/dokumenty/gisdata/output4")

print "Script completed successfully"

except Exception as e:

import traceback, sys
tb = sys.exc_info()[2]
print "An error occured on line %i" % tb.tb_lineno
print str(e)
PolyGeo
  • 65,136
  • 29
  • 109
  • 338
M. Tomas
  • 183
  • 8
  • 2
    If you want efficiency code in C# or VB.net for ArcObjects. You could consider 'pairing' out the processes by writing a batch file or python script then using subprocess.Popen create new threads, if one fails for any reason you've still got the instructions to retry. – Michael Stimson Nov 17 '17 at 02:05
  • 1
    What precisely is the wording of "an error screen saying all my work is lost"? – PolyGeo Nov 17 '17 at 02:10
  • Thank you for the reply. I'm sorry but I can't code in anything but python and only basics. In fact, my friend helped me to write this script. So I don't really understand what you did suggest, I apologize. – M. Tomas Nov 17 '17 at 02:11
  • @PolyGeo It says: ArcGIS Desktop has encountered a serious application error and is unable to continue. If you were in the middle of something, the information you were working on might be lost. – M. Tomas Nov 17 '17 at 02:12
  • Please use the [edit] button beneath your question to revise it with any requested clarifications. Don't forget to take the [tour] which is designed to introduce all users to the site and its protocols. – PolyGeo Nov 17 '17 at 02:14
  • 1
    The subprocess.Popen coding pattern that @MichaelStimson mentions is for Python and used at https://gis.stackexchange.com/a/231120/115 (and in other Q&As here and at [so]). I used it last week for the first time when tracing a Geometric Network repeatedly was bailing. – PolyGeo Nov 17 '17 at 02:19
  • Thank you, I will look into it tomorrow and try to implement it into my script. – M. Tomas Nov 17 '17 at 02:25
  • 1
    A method I have used with great success is to create a script that works for one instance, remove the top few lines, save, then in the 'master' script write to a new python script the missing few lines (InFeatures = , OutFeatures = etc..) then copy in the remaining lines from the working script, close and Popen... with some careful looping I am able to keep the number of processes down to int(os.environ.get('NUMBER_OF_PROCESSORS')) - 1. That is fairly advanced concepts though, perhaps lean on your friend and ask nicely with a gift for these modifications. – Michael Stimson Nov 17 '17 at 02:46
  • I apologize for replying so late. I managed to solve my problem using another method. I saw that my only problem is that arcmap is having perfomance issues while displaying all the layers (symbology and routes) in the map (and in TOC). So I decided that if every created .lyr would be removed from mxd before creating another one, it would solve my problem. And it did. My friend helped me to add these lines into the script: – M. Tomas Nov 25 '17 at 22:32
  • 'code'

    mxd = arcpy.mapping.MapDocument("CURRENT")
    layerList = arcpy.mapping.ListLayers(mxd, "", mxd.activeDataFrame) for layer in layerList: if layer.name == outNALayerName: arcpy.mapping.RemoveLayer(mxd.activeDataFrame, layer)

    arcpy.RefreshTOC()

    – M. Tomas Nov 25 '17 at 22:35

0 Answers0