8

or: Does selecting a domain when entering a job narrows your future options for domains and hence jobs ?

To make this question as broadly applicable as possible ...

  • profession refers to all kinds of data analysts, from statisticians over machine learner programmers to data miners.
  • imagine you were asked to give an advice to an audience containing both students and professionals of different age classes

Maybe a starting point:

The competitions on Kaggle have shown that outsiders can outperform the models created by company employees (see e.g. here). On the other hand, my (limited) work experience has led me to the conclusion, that understanding how and where the data has been generated is absolutely mandatory to create an abstract environment where something like a Kaggle competition can happen. Furthermore, without domain knowledge, I find it hard to report the results to other layers / departments. Some associate the last skill as key to the "new" profession "Data Science" (see e.g. here or here).

steffen
  • 10,367
  • question has already been flagged in order to make it cw. – steffen Feb 05 '13 at 10:48
  • 1
    Wrt. to the Kaggle example: 1. The "internal benchmark" (which seems to be the basis for the "340% outperforms") doesn't say it is the best model Allstate has. Other competitions use reasonably simple and basic models for benchmarking, that may be the case here as well. 2. No domain knowledge: do not forget the amount of both domain and data analysis knowledge that comes in during preparation of the data set. And: I do not know the profession / application expertise of the winner. – cbeleites unhappy with SX Feb 05 '13 at 16:25

2 Answers2

7

I make an analogy: Solving statistical problems without context is like boxing while blindfolded. You might knock your opponent out but you might bash your hand on the ringpost.

I work mostly with medical and social science researchers. There seems to be a widespread feeling there that the proper model for research is

1) They come up with an idea, gather data, write about it and then 2) They give it to us to "do the statistics".

So, I agree that we need to understand the issues; of course, we don't need as full an understanding of the research as the practitioner has. That is why I (and many other data-people) can work with people in different profession. But, the less we know about a subject, the more we need to interact with the professional to make sure that the results make sense.

One of the many things I like about what I do is that I get to learn a bit about a lot of different subjects.

Peter Flom
  • 119,535
  • 36
  • 175
  • 383
  • 1
    Very nice analogy. Though a bit of statistics in the DoE (randomization, sample size planning) doesn't hurt, neither... And the need for interaction may explode if the overlap in knowledge (and also terminology) is too low. – cbeleites unhappy with SX Feb 05 '13 at 16:03
5

How important is domain knowledge in our profession?

  • Important enough to give distinct names to the domain-oriented data analyses (e.g. -metrics: biometrics, psychometrics, chemometrics, ...)

  • The mix of domain knowledge and statistical knowledge is extremely important in

    • design of experiments, e.g. practical ./. statistical feasibility, domain specific norms, sample size planning
    • guiding data analysis (What kind of transformations or pre-processing are physically/biologically/chemically meaningful? What corrections of the raw data are needed?, criteria for data quality, heuristics)
    • checking whether results can possibly be meaningful/correct
    • interpretation of results
      Here's an example of a domain-specific interpretation of a classifier that was possible only because both data-analytical and spectroscopic knowledge together were at hand (section "Descriptive LDA and spectroscopic interpretation"). Try to imagine the amount of communication that would be needed between a data-analyst without spectroscopic knowledge and a spectroscopist with no idea of LDA to arrive at such an interpretation.
    • In the context of (lack of) reproducibility of published results, there is concern about research conducted as if there were no further knowledge of the field/problem/data, see e.g. E. R: Dougherty: Biomarker development: Prudence, risk, and reproducibility, BioEssays, 2012, 34, 277-279.
      Beck-Bornholt & Dubben would probably argue that incorporating more domain knowledge boosts the prevalence (prior probability) of good scientific ideas.
    • The no free lunch theorem hints into the same direction.

    (I'm a chemist specialized in chemometrics and spectroscopy, and do both measurements and data analysis)

Does selecting a domain when entering a job narrows your future options for domains and hence jobs?

Maybe, but at the same time, you'll be able to claim more expertise in that area and consequently can apply for the specialized jobs (and my experience is that we chemometricians are a much wanted species).

And, in addition, you show that you are able to join work in new domains.

cbeleites unhappy with SX
  • 38,684