ogr2ogr the result of merging two shapefiles has attributes missing

Question

I am trying to merge two shapefiles from a zip obtained here: http://wsgw.mass.gov/data/gispub/shape/l3parcels/L3_SHP_M351_YARMOUTH.zip

The two shapefiles contain different attributes for the same town in Massachusetts. I want to join the two shapefiles into one consolidated shapefile with all the attribute information from both files.

I can achieve this on QGIS, however something goes wrong when I try to do this on the command line.

On the command when I do this:

$ ls *.shp
    M351OthLeg.shp  M351TaxPar.shp

$ ogr2ogr -f "ESRI Shapefile" consolidated.shp  M351TaxPar.shp
$ ogr2ogr -f "ESRI Shapefile" -update -append consolidated.shp M351OthLeg.shp

It does produce a consolidated file that is larger than the first two:

$ ls -al *.shp
-rwxr-xr-x@ 1 ssikdar  600    249940 Feb 15  2012 M351OthLeg.shp
-rwxr-xr-x@ 1 ssikdar  600  10765676 Feb 15  2012 M351TaxPar.shp
-rw-r--r--  1 ssikdar  600  11015516 Apr 20 21:55 consolidated.shp

But the new file is missing attributes of the second file. When I do an ogrinfo:

$ ogrinfo -so -al consolidated.shp 
INFO: Open of `consolidated.shp'
      using driver `ESRI Shapefile' successful.

...
SHAPE_Leng: Real (19.11)
SHAPE_Area: Real (19.11)
MAP_PAR_ID: String (26.0)
LOC_ID: String (18.0)
POLY_TYPE: String (15.0)
MAP_NO: String (4.0)
SOURCE: String (15.0)
PLAN_ID: String (40.0)
LAST_EDIT: Integer (9.0)
BND_CHK: String (2.0)
NO_MATCH: String (1.0)
TOWN_ID: Integer (4.0)

It doesn't seem to have the attributes of the last file I tried to merge with it.

$ ogrinfo -so -al M351OthLeg.shp
...
MAP_PAR_ID: String (26.0)
LEGAL_TYPE: String (15.0)
LS_BOOK: String (16.0)
LS_PAGE: String (14.0)
REG_ID: String (15.0)
SHAPE_Leng: Real (19.11)
SHAPE_Area: Real (19.11)
TOWN_ID: Integer (4.0)
TAXPAR_ID: String (18.0)

As a test I merged the two shapefile in Quantum GIS. When I do an ogrinfo on that file, the additional attributes are there:

$ ogrinfo -so -al ~/Documents/m351_qgis.shp
...
MAP_PAR_ID: String (254.0)
LEGAL_TYPE: String (254.0)
LS_BOOK: String (254.0)
LS_PAGE: String (254.0)
REG_ID: String (254.0)
SHAPE_Leng: Real (21.5)
SHAPE_Area: Real (21.5)
TOWN_ID: Integer (10.0)
TAXPAR_ID: String (254.0)
LOC_ID: String (254.0)
POLY_TYPE: String (254.0)
MAP_NO: String (254.0)
SOURCE: String (254.0)
PLAN_ID: String (254.0)
LAST_EDIT: Integer (10.0)
BND_CHK: String (254.0)
NO_MATCH: String (254.0)

Is there something I'm missing in my command line arguments that's giving me a different result than QGIS?

Read the manual http://www.gdal.org/ogr2ogr.html and use -addfields. — user30184, Apr 21 '17 at 04:18

score 1 · Accepted Answer · edited Jun 11 '20 at 15:27

The reason why you don't get all the fields is that you're following an approach that is generally suitable for layers with the same schema. Merging two shapefiles with different schemas can be done using the OGR VRT or simply UNION ALL of OGR SQL.

The effect of UNION ALL is to concatenate the rows returned by the right SELECT statement to the rows returned by the left SELECT statement. [...]

The processing of UNION ALL in OGR differs from the SQL standard, in which it accepts that the columns from the various SELECT are not identical. In that case, it will return a super-set of all the fields from each SELECT statement.

So, assuming that you have your shapefiles in the same directory, you can simply execute:

ogr2ogr consolidated.shp M351TaxPar.shp -sql "SELECT * FROM M351TaxPar UNION ALL (SELECT * FROM 'M351OthLeg.shp'.M351OthLeg)"

Thanks! Just to let you know your VRT link in the linked answer points to to 404 I think this is the new link — RidinAGrvyTrain, Apr 21 '17 at 12:50
I've just updated the link, but the link is another one. Thanks the same! — Antonio Falciano, Apr 21 '17 at 13:29

score 1 · Answer 2 · answered Apr 21 '17 at 15:16

1

GDALs ogr2ogr utility has this built in, so in addition to -update and -append also use -addfields

answered Apr 21 '17 at 15:16

SaultDon

10,389
1
43
78

ogr2ogr the result of merging two shapefiles has attributes missing

2 Answers2