Data management practices

Question

Software has code repositories and package managers, so installation and maintenance are done in one line (or click).

What are practices for managing data, instead? Apart from manually organized directories with readme files inside, I found a couple solutions:

drake from Factual, https://github.com/Factual/drake
CKAN from OKFN, https://okfn.org/projects/ckan/
Zotero and other librarian tools, so you save datasets instead of books with them

However, drake is mostly about workflows, not maintenance. And CKAN is server-based solution for which I couldn't find a local "package manager".

What would such a package manager ideally do:

Have you seen anything like this? If not, maybe you'd share your practices of maintaining the data manually?

@albert Why not? Though I found only GitHub LFS and other solutions not integrated in data package managers. — Anton Tarasenko, May 23 '15 at 10:29

score 4 · Answer 1 · edited May 23 '17 at 12:37

4

R, the open source and cross-platform statistics software has something similar where sometimes packages/plugins for R include sample or test data:
- See this SO question: https://stackoverflow.com/questions/12391195/include-data-examples-in-developing-r-packages
- and this info page: http://r-pkgs.had.co.nz/data.html
OKFN (the same folks behind CKAN) has something they called Data Packages (http://dataprotocols.org/data-packages/) which sounds like most of what you're looking for.
- Ecosystem image from Data Packages website embedded below:

edited May 23 '17 at 12:37

Community

answered May 17 '15 at 23:35

Mark Silverberg

2

OKFN also has data package manager dpm (https://github.com/okfn/dpm/), though no good docs for it. – Anton Tarasenko May 21 '15 at 23:21

1 Answers1