Data Science Stack Exchange
Data Science Stack Exchange

Most Popular

more tags
1500 questions
12
votes
1 answer

Finding the inverse of a matrix with neural networks

I have a generic matrix A which is symmetric, positive definite and sparsely populated (it's also quite big, say composed of tens to hundreds thousands rows). I would like to have a neural network learn how to find a matrix B that resembles the…
Sovos
  • 231
  • 2
  • 4
12
votes
3 answers

How to get the probability of belonging to clusters for k-means?

I need to get the probability for each point in my data set. The idea is to compute distance matrix (first column contsins distances to first cluster, second column conteins distances to second cluster and etc). The closest point has probability =…
Толкачёв Иван
  • 317
  • 1
  • 3
  • 8
12
votes
1 answer

Tensorflow neural network TypeError: Fetch argument has invalid type

I am making a simple neural network using tensorflow, with data I collected myself, however, it's not cooperating :P I have run into an error that I can't fix or find the fix for and I would love your help. The errormessage: TypeError: Fetch…
Hasse Iona
  • 453
  • 1
  • 3
  • 13
12
votes
3 answers

Mass convert categorical columns in Pandas (not one-hot encoding)

I have pandas dataframe with tons of categorical columns, which I am planning to use in decision tree with scikit-learn. I need to convert them to numerical values (not one hot vectors). I can do it with LabelEncoder from scikit-learn. The problem…
user1700890
  • 345
  • 1
  • 3
  • 13
12
votes
2 answers

Features of word vectors in Word2Vec

I am trying to do sentiment analysis. In order to convert the words to word vectors, I am using Word2Vec model. Suppose I have all the sentences in a list named 'sentences' and I am passing these sentences to word2vec as follows: model =…
enterML
  • 3,031
  • 9
  • 27
  • 38
12
votes
2 answers

Machine Learning Best Practices for Big Dataset

I am about to graduate from my Master and had learnt about machine learning as well as performed research projects with it. I wonder about the best practices in the industry when performing machine learning tasks with Big Datasets (like 100s GB or…
iLoeng
  • 243
  • 3
  • 5
12
votes
2 answers

Neural Networks - Loss and Accuracy correlation

I'm a bit confused by the coexistence of Loss and Accuracy metrics in Neural Networks. Both are supposed to render the "exactness" of the comparison of $y$ and $\hat{y}$, aren't they? So isn't the application of the two redundant in the training…
Hendrik
  • 8,587
  • 17
  • 42
  • 55
12
votes
5 answers

How to overcome training example's different lengths when working with Word Embeddings (word2vec)

I'm working on Sentiment Analysis over tweets using word2vec as word representation. I have trained my word2vec model. But when I'm going to train my classifier, I'm facing the issue that every tweet has different length and the classifier…
antorqs
  • 221
  • 2
  • 5
12
votes
8 answers

Definition of a model in machine learning

This definition does not quite apply since we are not always assuming an underlying distribution. So what is a model really? Can a Gradient Boosted Model (GBM) with specified hyperparameters be considered a model? Is a model a collection of rules?
organic agave
  • 221
  • 1
  • 2
  • 4
12
votes
1 answer

Hashing Trick - what actually happens

When ML algorithms, e.g. Vowpal Wabbit or some of the factorization machines winning click through rate competitions (Kaggle), mention that features are 'hashed', what does that actually mean for the model? Lets say there is a variable that…
B_Miner
  • 702
  • 1
  • 7
  • 20
12
votes
3 answers

Which supervised learning algorithms are available for matching?

I'm working on a non-profit where we try to help potential university applicants by matching them with alumni that want to share their experience/wisdom and, at the moment, it is happening manually. So I'll have two tables, one with students and one…
k1nd3rm4x1
  • 123
  • 1
  • 1
  • 5
12
votes
3 answers

Why use cosine similarity instead of scaling the vectors when calculating the similarity of vectors?

I'm watching a NLP video on Coursera. It's discussing how to calculate the similarity of two vectors. First it discusses calculating the Euclidean distance, then it discusses the cosine similarity. It says that cosine similarity makes more sense…
Allure
  • 275
  • 2
  • 7
12
votes
2 answers

Document classification using convolutional neural network

I'm trying to use CNN (convolutional neural network) to classify documents. CNN for short text/sentences has been studied in many papers. However, it seems that no papers have used CNN for long text or document. My problem is that there are too many…
lenhhoxung
  • 231
  • 2
  • 8
12
votes
1 answer

Which of the 180 algorithms in R's caret package are feasible?

R's caret package works with 180 models. The author warns that some of the package may be intractably slow or less accurate than top-choice models. The author is not wrong about this. I've tried to train Boruta and evtree models and had to give up…
Hack-R
  • 1,919
  • 1
  • 21
  • 34
12
votes
2 answers

Opensource tools for help in mining stream of leader board scores

Consider a stream containing tuples (user, new_score) representing users' scores in an online game. The stream could have 100-1,000 new elements per second. The game has 200K to 300K unique players. I would like to have some standing queries like:…
Tahir Akhtar
  • 315
  • 2
  • 9
1 2 3
…
99 100