Data Science Stack Exchange
Data Science Stack Exchange

Most Popular

more tags
1500 questions
12
votes
1 answer

What's the relationship between an SVM and hinge loss?

My colleague and I are trying to wrap our heads around the difference between logistic regression and an SVM. Clearly they are optimizing different objective functions. Is an SVM as simple as saying it's a discriminative classifier that simply…
Simon
  • 1,026
  • 10
  • 10
12
votes
2 answers

Creating new columns by iterating over rows in pandas dataframe

I have a pandas data frame (X11) like this: In actual I have 99 columns up to dx99 dx1 dx2 dx3 dx4 0 25041 40391 5856 0 1 25041 40391 25081 5856 2 25041 40391 42822 0 3 25061 40391 0 0 4 25041 …
Sanoj
  • 251
  • 1
  • 2
  • 6
12
votes
2 answers

Solving a system of equations with sparse data

I am attempting to solve a set of equations which has 40 independent variables (x1, ..., x40) and one dependent variable (y). The total number of equations (number of rows) is ~300, and I want to solve for the set of 40 coefficients that minimizes…
mike1886
  • 933
  • 9
  • 17
12
votes
3 answers

How does a query into a huge database return with negligible latency?

For example, when searching something in Google, results return nigh-instantly. I understand that Google sorts and indexes pages with algorithms etc., but I imagine it infeasible for the results of every single possible query to be indexed (and…
resgh
  • 231
  • 1
  • 7
12
votes
4 answers

Transformer model: Why are word embeddings scaled before adding positional encodings?

While going over a Tensorflow tutorial for the Transformer model I realized that their implementation of the Encoder layer (and the Decoder) scales word embeddings by sqrt of embedding dimension before adding positional encodings. Notice that this…
Milad Shahidi
  • 403
  • 4
  • 8
12
votes
3 answers

Predicting next medical condition from past conditions in claims data

I am currently working with a large set of health insurance claims data that includes some laboratory and pharmacy claims. The most consistent information in the data set, however, is made up of diagnosis (ICD-9CM) and procedure codes (CPT, HCSPCS,…
Jamie
  • 123
  • 7
12
votes
1 answer

ngram and RNN prediction rate wrt word index

I tried to plot the rate of correct predictions (for the top 1 shortlist) with relation to the word's position in sentence : I was expecting to see a plateau sooner on the ngram setup since it needless context. However, one thing I wasn't expecting…
Arkantus
  • 157
  • 3
12
votes
3 answers

How do I set/get heap size for Spark (via Python notebook)

I'm using Spark (1.5.1) from an IPython notebook on a macbook pro. After installing Spark and Anaconda, I start IPython from a terminal by executing: IPYTHON_OPTS="notebook" pyspark. This opens a webpage listing all my IPython notebooks. I can…
Kai
  • 303
  • 1
  • 2
  • 10
12
votes
4 answers

Prohibitive size of random forest when saved to disk

When saved to disk using cPickle: https://stackoverflow.com/questions/20662023/save-python-random-forest-model-to-file, my random forest is 6.57 GB. with open('rforest.cpickle', 'wb') as f: cPickle.dump(rforest, f) I want to use the forest…
compguy24
  • 263
  • 2
  • 7
12
votes
7 answers

Multi-country model or single model

I am working on a ML model to be deployed in a product operating in many countries. The issue that I am having is the following: should I train one model and serve it for all countries? train a model per country and serve each model in its…
David Masip
  • 6,051
  • 2
  • 24
  • 61
12
votes
3 answers

What needs to be done to make n_jobs work properly on sklearn? in particular on ElasticNetCV?

The constructor of sklearn.linear_model.ElasticNetCV takesn_jobs as an argument. Quoting the documentation here n_jobs: int, default=None Number of CPUs to use during the cross validation. None means 1 unless in a joblib.parallel_backend context.…
OldSchool
  • 261
  • 1
  • 2
  • 8
12
votes
1 answer

Balanced Accuracy vs. F1 Score

I've read plenty of online posts with clear explanations about the difference between accuracy and F1 score in a binary classification context. However, when I came across the concept of balanced accuracy, explained e.g. in the following image…
Ric S
  • 257
  • 2
  • 11
12
votes
4 answers

Can the F1 score be equal to zero?

As it is mentioned in the F1 score Wikipedia, 'F1 score reaches its best value at 1 (perfect precision and recall) and worst at 0'. What is the worst condition that was mentioned? Even if we consider the case of: either precision or recall is…
akhil penta
  • 221
  • 1
  • 2
  • 4
12
votes
5 answers

Please review my sketch of the Machine Learning process

It's amazingly difficult to find an outline of the end-to-end machine learning process. As a total beginner, this lack of information is frustrating, so I decided to try scraping together my own process by looking at a lot of tutorials that all do…
rocksNwaves
  • 309
  • 1
  • 10
12
votes
5 answers

LSTM or other RNN package for R

I saw some impressive result from LSTM models producing Shakespeare like texts. I was wondering if an LSTM package exists for R. I googled for it but only found packages for Python and Julia. (maybe there are some performance issue which explains…
Viktor
  • 850
  • 1
  • 6
  • 17
1 2 3
…
99 100