2

I am trying to export feature classes within a geodatabase to CSV format using "Export Feature Attribute to ASCII" in ArCGIS. I am getting the following error 

Traceback (most recent call last):
  File "c:\program files (x86)\arcgis\desktop10.2\ArcToolbox\Scripts\ExportXYV.py", line 159, in <module>
    export = setupXYV()
  File "c:\program files (x86)\arcgis\desktop10.2\ArcToolbox\Scripts\ExportXYV.py", line 49, in setupXYV
    outFieldNames = outFieldNames)
  File "c:\program files (x86)\arcgis\desktop10.2\ArcToolbox\Scripts\ExportXYV.py", line 129, in exportXYV
    formatValue = LOCALE.format(localeDict[field], value)
  File "C:\Python27\ArcGIS10.2\lib\locale.py", line 196, in format
    return _format(percent, value, grouping, monetary, *additional)
  File "C:\Python27\ArcGIS10.2\lib\locale.py", line 202, in _format
    formatted = percent % value
TypeError: not all arguments converted during string formatting

Here's the python script I am using

import arcpy
from arcpy import env
import os, sys
arcpy.CheckOutExtension("spatial")    
arcpy.env.overwriteOutput = True

#Set environment settings
workspace = r'L:\Simulation\Output\PoolVariables.gdb'


# Set local variables
outWorkspace = r'L:\Simulation\Output\CreateCSV'

feature_classes = []
for dirpath, dirnames, filenames in arcpy.da.Walk(workspace, datatype = "FeatureClass", type = "Point"):
    for filename in filenames:
        feature_classes.append(os.path.join(dirpath, filename))

#print feature_classes

# Loop through feature_classes list

try:
    for fc in feature_classes:
        field_names =[f.name for f in arcpy.ListFields(fc)]
        print field_names
        #arcpy.ExportXYV_stats(fc, field_names, "COMMA",  )
        name = os.path.basename(fc)



    # Extract only the FC basename
        arcpy.ExportXYv_stats(fc, field_names,"COMMA",name, "ADD_FIELD_NAMES")

except:
    print arcpy.GetMessages()

Could someone explain what is happening here?

PolyGeo
  • 65,136
  • 29
  • 109
  • 338
Kenny D
  • 265
  • 1
  • 10
  • The error you mention is not in that code block, it is on line 49 where outFieldNames = outFieldNames Are you trying to just get a complete CSV of each feature class in a geodatabase? There are easier ways.. – Michael Stimson Jun 16 '15 at 21:28
  • Yes, a complete CSV for each feature class in a geodatabase. Could you give some hints please? – Kenny D Jun 16 '15 at 21:31
  • The Point feature classes already have separate coordinates field. Coordinates created by ExportXYv_stats would be redundant in this case. – Kenny D Jun 16 '15 at 21:36
  • How about using Table Select http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//000800000007000000 on the feature class and specify the out_table as name = os.path.join(outWorkspace, os.path.basename(fc) + ".csv") this would replace arcpy.ExportXYv... and you don't need to specify which fields to use. – Michael Stimson Jun 16 '15 at 21:37
  • This saves the table as *.dbf. The issue is also mentioned here – Kenny D Jun 16 '15 at 21:54
  • It's changed since the last time I used it, it used to be if you specified the extension CSV it would know what you wanted (much like CopyRaster)... use Table to Table then, that one seems to work. A feature class implements table so any table tools will still work - but the output is always a table. – Michael Stimson Jun 16 '15 at 21:58
  • Let us continue this discussion in chat. – Kenny D Jun 17 '15 at 17:12

2 Answers2

3

I see the problem... irrespective of what format is chosen a .dbf is written. Well, that has certainly changed since the last time I used those tools. Seeing as you're in python perhaps a little routine could help here:

def WriteCsv(InFC,OutTab):
    print("Exporting %s to %s" % (InFC,OutTab))
    with open(OutTab,'w') as OutFile:
        FieldNames = [x.name for x in arcpy.ListFields(InFC)] # convert fields to names..

        # skip the shape field, it wouldn't print well anyway
        desc = arcpy.Describe(InFC)
        if (desc.datasetType == 'FeatureClass'):
            try:
                FieldNames.remove(desc.shapeFieldName)
            except:
                arcpy.AddWarning("Unable to skip shape field, perhaps it doesn't exist")

        # print the header row to the text file
        OutFile.write(','.join(FieldNames)) # comma format the field names
        OutFile.write('\n')                 # new line

        with arcpy.da.SearchCursor(InFC,FieldNames) as Scur:
            for row in Scur:
                OutFile.write(','.join(map(str,row))) # comma format the row
                OutFile.write('\n')                   # new line

Which would interface with your program like this:

import os, sys, arcpy
# from arcpy import env                 # not needed - specified implicitly in this script
# arcpy.CheckOutExtension("spatial")    # not needed - no arcpy.sa tools used

arcpy.env.overwriteOutput = True

def WriteCsv(InFC,OutTab):
    with open(OutTab,'w') as OutFile:
        FieldNames = [x.name for x in arcpy.ListFields(InFC)] # convert fields to names..
        desc = arcpy.Describe(InFC)
        if (desc.datasetType == 'FeatureClass'):
            try:
                FieldNames.remove(desc.shapeFieldName)
            except:
                arcpy.AddWarning("Unable to skip shape field, perhaps it doesn't exist")
        OutFile.write(','.join(FieldNames)) # comma format the field names
        OutFile.write('\n')                 # new line
        with arcpy.da.SearchCursor(InFC,FieldNames) as Scur:
            for row in Scur:
                OutFile.write(','.join(map(str,row))) # comma format the row
                OutFile.write('\n')                   # new line

#Set environment settings
workspace = r'L:\Simulation\Output\PoolVariables.gdb'

# Set local variables
outWorkspace = r'L:\Simulation\Output\CreateCSV'

feature_classes = []
for dirpath, dirnames, filenames in arcpy.da.Walk(workspace, datatype = "FeatureClass", type = "Point"):
    for filename in filenames:
        feature_classes.append(os.path.join(dirpath, filename))

# Loop through feature_classes list
try:
    for fc in feature_classes:
        name = os.path.join(outWorkspace, os.path.basename(fc) + ".csv") 
        WriteCsv(fc,name)

except:
    print arcpy.GetMessages()

A few pointers:

  • Using '.'.join(map(str,row)) comes from a stack overflow post (thanks Mark Biek), the map is important if the row contains any numbers (like FID) to make the whole list a list of strings.
  • The use of '\n' for the line separator is supported by this stack overflow post (thanks sorin), apparently there isn't a need for os.linesep as python will understand what you're trying to do and insert the correct character anyway... I don't use Mac or Linux so I can't confirm.
Community
  • 1
Michael Stimson
  • 25,566
  • 2
  • 35
  • 74
  • for some reason the script did not loop through the file geodatabase feature classes and created a single, empty csv file. – Kenny D Jun 18 '15 at 04:31
  • That is more likely to be caused by the building of the feature class list, is it iterating the feature classes correctly? Insert a print statement to ensure the list is being populated... also on the first line of the function print("Exporting %s to %s" % (InFC,OutTab)) just to make sure it got there ok and the paths are valid. – Michael Stimson Jun 18 '15 at 04:41
  • The list is complete and (print("Exporting %s to %s" % (InFC,OutTab))) resulted in Exporting L:\Simulation\Output\PoolVariables.gdb\VAR1 to L:\Simulation\Output\CreateCSV\VAR1.csv. The CSV, however, is empty. – Kenny D Jun 18 '15 at 04:54
  • That's odd. The script that I posted (gdb paths changed) ran fine and produced expected results... can you post your updated code please, perhaps something was lost in translation. – Michael Stimson Jun 18 '15 at 05:05
  • There was a typo on my part. The script works like a charm. Is there anyway this can be turned into a toolbox? A lot of folks around here can benefit from this. – Kenny D Jun 18 '15 at 05:27
  • Yes, sort of, it can be put into a toolbox as a tool. Have a read of http://resources.arcgis.com/en/help/main/10.1/index.html#//00150000001r000000 about the basics and change workspace = r'L:\Simulation\Output\PoolVariables.gdb' into workspace = arcpy.GetParameterAsText(0) and outWorkspace = r'L:\Simulation\Output\CreateCSV' into outWorkspace = arcpy.GetParameterAsText(1) to parameterize the variables. If you distribute the script and toolbox then make the path relative to the toolbox and be sure to copy both (or store on a network location). – Michael Stimson Jun 18 '15 at 05:30
  • For some reason the script stopped working after it generated couple of CSVs (last seven files in geodatabase out of 27 files). In this case each csv file is approx 700 MB..would there be any size/ number of files limitation? – Kenny D Jun 18 '15 at 05:44
  • No, text files can be huge (I've had them multi gigabyte), however your virus checker (and other processes) can get in the way if it's set to 'on access scan'. – Michael Stimson Jun 18 '15 at 21:20
  • there was a feature class which was not visible from ArcCatalog while it was printed out in the fc list from python script. I do not know what caused it..so created another geodatabase and copied the fcs there. Ran your script and everything is fine... – Kenny D Jun 18 '15 at 21:32
  • 1
    So it was a corrupt database? In a 'worst case' situation exporting the database to XML and then re-importing to a new database is known to restore otherwise unreadable databases. – Michael Stimson Jun 18 '15 at 21:39
  • This can be a life saver.. – Kenny D Jun 18 '15 at 21:43
3

I am using something like this with great success.

import arcpy, os, csv 
inTables = r"pathToWorkspace"
    for dirpath, dirnames, filenames in arcpy.da.Walk(inTables, datatype="Table", type="ALL"):
        for tableName in filenames:
            print "Appending tables from " + tableName + " to " + newTable
            arcpy.Append_management(os.path.join(dirpath, tableName), newTable, "NO_TEST")


            CSVFile = r"pathToOutputCSV.csv"
            fields = arcpy.ListFields(newTable)
            fieldNames = [field.name for field in fields]

            with open(CSVFile, 'w') as f:
                dw = csv.DictWriter(f,fieldNames)
                dw.writeheader()

            with arcpy.da.SearchCursor(newTable,fieldNames) as cursor:
                for row in cursor:
                    dw.writerow(dict(zip(fieldNames,Utf8EncodeArray(row))))
                print "converted " +  masterTable + " to a CSV file!"
            del row, cursor
cbrannin
  • 105
  • 7