6

I'm hoping to analyze google timeline data in R for a final project in a GIS class,

Google can output timeline data as geojson and KML but it looks like the KML file includes a lot more info (type of transportation distance traveled etc) than the JSON file. Additionally, JSON is an option for the entire time lines but to download a single day/week/month it looks like I have to use KML.

I understand from How to efficiently read a kml file into R and Importing KML files to R that I need to specify the layer info as well as the kml file name to readOGR(), what I'm a little confused about is exactly how the layer names are included in a kml file.

Looks like the <name> tag is associated with the layer name, but there are 122 name tags in the file so its clearly not exclusive to layer. Fine.

using layers <- ogrinfo(data_source) gets me

[1] "INFO: Open of `C:\\Users\\Documents\\GIS_Coursework_3\\history-2018-09-28.kml'"
[2] "      using driver `LIBKML' successful."                                                                             
[3] "1:  Location history from 2018-09-28 to 2018-09-28 "

then using Location_History <- readOGR(data_source, "Location history-2018-09-28 to 2018-09-28 ") gives:

Error in ogrInfo(dsn = dsn, layer = layer, encoding = encoding, use_iconv = use_iconv, : 
Multiple incompatible geometries: wkbPoint: 12; wkbLineString: 12

The problem is that there are multiple "sub" layers,

in QGIS i can see that when I open the layer, there is a points layer and a lines layer

sub layers

I dont' see either of these text strings in the KML files anywhere when I open it as a text file.

I could probably just copy those, but it's not particularly useful to me as a one off. I need to get the layer info programmatically, rather than opening QGIS every time.

Is it practical for me to start exploring xml parsing in R?

Is there a package I haven't been able to find that handles this stuff successfully?

Am I missing something obvious about how to read KML layer info?

This is the only feature I've found lacking in R compared to QGIS or ArcGIS. They've been pretty comparable so far which I've found impressive.

PolyGeo
  • 65,136
  • 29
  • 109
  • 338
Hugh_Kelley
  • 747
  • 1
  • 6
  • 19
  • What does rgdal::ogrListLayers("foo.kml") do for you? Have you tried reading with sf::st_read, which can probably cope with multiple geometry types? – Spacedman Dec 21 '18 at 15:26
  • looks like ogrListLayers() returns the "top" layer for lack of a better word, similar to what ogrinfo() returns – Hugh_Kelley Dec 21 '18 at 15:29
  • Is there any way you can supply us with one of these problematic KML files? – Spacedman Dec 21 '18 at 15:33
  • actually st_read() handles it pretty well, no errors or warnings. data looks ok in the sf object it returns... – Hugh_Kelley Dec 21 '18 at 15:35
  • I guess I'm set, rather than sharing the data which is a full day of location data for myself, if you're curious about the kml file you could grab your own from your google timeline in google maps if you have it switched on. – Hugh_Kelley Dec 21 '18 at 15:41
  • Thanks very much for the help! If you post st_read() as an answer I'll accept it. – Hugh_Kelley Dec 21 '18 at 16:03

2 Answers2

7

Functions in rgdal read data into sp class objects, which can only contain one type of spatial object - a set of points, or lines, or polygons. The sf package provides classes for geometry vectors that can have different dimensions geometries within.

Using sf::st_read("file.kml") should return an object with a geometry column, and you can filter lines or points or polygons from this object using the st_geometry_type function.

Spacedman
  • 63,755
  • 5
  • 81
  • 115
  • thanks again, just had a good read about how "tidy" applies to spatial data and why sf is the tidy way to use data because it holds geometries as lists within a column so that each feature is a row. – Hugh_Kelley Dec 21 '18 at 16:22
0

Based on the reading I've done in the past 24 hours, I can't say that what I'm trying to do is impossible in r, but I can say that it's highly impractical. So I'm going to go in a different direction and leave this here for posterity because it seems likely that someone else is going to run into this and maybe I can save them some time.

Context: readOGR, basically requires that you open the kml file in another program first because it requires the layer name that can't be found without parsing the kml somehow and , sort of recursively, there isn't a way to robustly parse kml in r.

Finding 1: using regex to parse xml is a bad idea because the nested/hierarchical nature of the data means that the expressions aren't very regular.

Finding 2, using xmlParser isn't much better. I'm not overly familiar with xml and maybe you can make more of this guide than I could. Take a look at how to parse xml to r data frame. The basic issue I found is that I was seeing information in the kml file in a text editor that xmlParser wasn't including in the lists it returned and it was important information.

Finding 3, someone did attempt a package that that could handle kml files for r, but its not really robust enough to be relied upon. This blog post explains what the person was going for and the code can be found at this github repository

So finally, maybe there's a solution in the works or simply something I didn't find but if you need to load data programatically in r, you should probably stay away from kml unless you're really comfortable with xml or there's a new solution that wasn't available in 2018.

@Spacedman pointed out that sf::st_read() handles the kml well.

PolyGeo
  • 65,136
  • 29
  • 109
  • 338
Hugh_Kelley
  • 747
  • 1
  • 6
  • 19
  • Also, I want to note a last option that I did not explore. It's possible there's a good python package for this, which raises two considerations, 1 is the gradual merging of python and r, and 2 is the current options for running python scripts in r which I'm not familiar with but am aware of. If you really need to parse a kml file in r programmatically, maybe look for a python library that can do it and integrate it into your r script. – Hugh_Kelley Dec 21 '18 at 15:23