Data Science Stack Exchange
Data Science Stack Exchange

Most Popular

more tags
1500 questions
10
votes
1 answer

How do GBM algorithms handle missing data?

How do algorithms GBM algorithms, such as XGBoost or LightGBM handle NaN values? I know that they learn how to replace NaN values with other values but my question is: How do they do it exactly?
user10296606
  • 1,834
  • 5
  • 17
  • 31
10
votes
1 answer

Prediction with non-atomic features

I would like to use non-atomic data, as a feature for a prediction. Suppose I have a Table with these features: - Column 1: Categorical - House - Column 2: Numerical - 23.22 - Column 3: A Vector - [ 12, 22, 32 ] - Column 4: A Tree - [ [ 2323, 2323…
user3798928
  • 101
  • 3
10
votes
1 answer

How much training data does Word2Vec need?

I'd like to compare the difference among the same word mentioned in different sources. That is, how authors differ in their usage of ill-defined words, such as "democracy". A brief plan was Take the books mentioning the term "democracy" as plain…
Anton Tarasenko
  • 711
  • 8
  • 13
10
votes
2 answers

Foreign exchange market forecasting with neural networks

I would like to use ANN to automate trading currencies, preferably USD/EUR or USD/GBP. I know this is hard and may not be straightforward. I have already read some papers and done some experiments but without much luck. I would like to get advice…
user1300
  • 101
  • 4
10
votes
3 answers

Create most "average" cosine similarity observation

For a recommendation system I'm using cosine similarity to compute similarities between items. However, for items with small amounts of data I'd like to bin them under a general "average" category (in the general not mathematical sense). To…
eric chiang
  • 233
  • 2
  • 7
10
votes
4 answers

Why must a CNN have a fixed input size?

Right now I'm studying Convolutional Neural Networks. Why must a CNN have a fixed input size? I know that it is possible to overcome this problem (with fully convolutional neural networks etc...), and I also know that it is due to the fully…
Mattia Surricchio
  • 411
  • 2
  • 5
  • 14
10
votes
1 answer

How flexible is the link between objective function and output layer activation function?

It seems standard in many neural network packages to pair up the objective function to be minimised with the activation function in the output layer. For instance, for a linear output layer used for regression it is standard (and often only choice)…
Neil Slater
  • 28,918
  • 4
  • 80
  • 100
10
votes
3 answers

Handling a regularly increasing feature set

I'm working on a fraud detection system. In this field, new frauds appear regularly, so that new features have to be added to the model on ongoing basis. I wonder what is the best way to handle it (from the development process perspective)? Just…
Maxim Fridental
  • 103
  • 3
10
votes
1 answer

Random Forest VS LightGBM

Random Forest VS LightGBM Can somebody explain in-detailed differences between Random Forest and LightGBM? And how the algorithms work under the hood? As per my understanding from the documentation: LightGBM and RF differ in the way the trees are…
Pluviophile
  • 3,808
  • 13
  • 31
  • 54
10
votes
7 answers

Data science projects explained step by step?

I am looking for a website or book where several practical examples are given step by step, explaining how they choose the relevant features, the model selection procedure, etc...
cpumar
  • 807
  • 1
  • 9
  • 14
10
votes
4 answers

Is (manual) feature extraction outdated?

I recently attended a PhD thesis defence in which one committee members claimed that "manual feature extraction is outdated. Nowadays, we have [deep] machine learning models doing that job for us automatically." Is this statement true? If yes,…
Hagbard
  • 434
  • 5
  • 16
10
votes
2 answers

Is there a method that is opposite of dimensionality reduction?

I am new to the field of machine learning, but have done my share of signal processing. Please let me know if this question has been mislabeled. I have two dimensional data which is defined by at least three variables, with a highly non-linear model…
PhilMacKay
  • 201
  • 1
  • 4
10
votes
4 answers

Math PhD (Nonlinear Programming) switching to Data Science?

I am a math Ph.D. student who is interested in going to the industry as a Data Scientist after graduation. I will briefly give some background on my education before posing my question, so that it is better understood: Maths Coursework: This has…
John D
  • 123
  • 8
10
votes
1 answer

How do I calculate the delta term of a Convolutional Layer, given the delta terms and weights of the previous Convolutional Layer?

I am trying to train an artificial neural network with two convolutional layers (c1, c2) and two hidden layers (c1, c2). I am using the standard backpropagation approach. In the backward pass I calculate the error term of a layer (delta) based on…
cdwoelk
  • 181
  • 2
  • 7
10
votes
3 answers

Network analysis classic datasets

There are several classic datasets for machine learning classification/regression tasks. The most popular are: Iris Flower Data Set; Titanic Data Set; Motor Trend Cars; etc. But does anyone know similar datasets for networks analysis / graph…
sobach
  • 1,119
  • 5
  • 20
1 2 3
…
99 100