Data Science Stack Exchange
Data Science Stack Exchange

Most Popular

more tags
1500 questions
13
votes
1 answer

Expected 2D array, got scalar array instead

Can anyone help me with this error. I did the following code but it does not work and I am getting the following error: ValueError: Expected 2D array, got scalar array instead: array=6.5. Reshape your data either using array.reshape(-1, 1) if your…
Siri1993
  • 131
  • 1
  • 1
  • 3
13
votes
2 answers

Finding outliers in Image dataset

I have been working on an image classification tasks for which I am extracting the image frames from the video stream collected for different classes. I have already trained an image classification model (using transfer learning) however due to the…
deepguy
  • 1,441
  • 8
  • 18
  • 39
13
votes
2 answers

Cross-validation: K-fold vs Repeated random sub-sampling

I wonder which type of model cross-validation to choose for classification problem: K-fold or random sub-sampling (bootstrap sampling)? My best guess is to use 2/3 of the data set (which is ~1000 items) for training and 1/3 for validation. In this…
IgorS
  • 5,474
  • 11
  • 31
  • 43
13
votes
1 answer

What happens when we train a linear SVM on non-linearly separable data?

What happens when we train a basic support vector machine (linear kernel and no soft-margin) on non-linearly separable data? The optimisation problem is not feasible, so what does the minimisation algorithm return?
SVM
  • 131
  • 1
  • 4
13
votes
2 answers

Perform k-means clustering over multiple columns

I am trying to perform k-means clustering on multiple columns. My data set is composed of 4 numerical columns and 1 categorical column. I already researched previous questions but the answers are not satisfactory. I know how to perform the algorithm…
Lola
  • 141
  • 1
  • 1
  • 3
13
votes
5 answers

How to Write Multiple Data Frames in an Excel Sheet

I have multiple data frames with same column names. I want to write them together to an excel sheet stacked vertically on top of each other. And between each, there will be a text occupying a row. This is what I have in mind. I tried the…
Della
  • 325
  • 1
  • 3
  • 9
13
votes
2 answers

Preference Matching Algorithm

There's this side project I'm working on where I need to structure a solution to the following problem. I have two groups of people (clients). Group A intends to buy and group B intends to sell a determined product X. The product has a series of…
R.D
  • 151
  • 5
13
votes
2 answers

Variable input/output length for Transformer

I was reading the paper "Attention is all you need" (https://arxiv.org/pdf/1706.03762.pdf ) and came across this site http://jalammar.github.io/illustrated-transformer/ which provided a great breakdown of the architecture of the Transformer.…
Sean Lee
  • 251
  • 2
  • 8
13
votes
1 answer

What is the difference between global and universal compression methods?

I understand that compression methods may be split into two main sets: global local The first set works regardless of the data being processed, i.e., they do not rely on any characteristic of the data, and thus need not to perform any…
Rubens
  • 4,107
  • 5
  • 23
  • 42
13
votes
6 answers

Deep network not able to learn imbalanced data beyond the dominant class

I have data with 5 output classes. The training data has the following no of samples for these 5 classes: [706326, 32211, 2856, 3050, 901] I am using the following keras (tf.keras) code: class_weights =…
dbm
  • 251
  • 1
  • 2
  • 7
13
votes
1 answer

Why doesn't class weight resolve the imbalanced classification problem?

I know that in imbalanced classification, the classifier tends to predict all the test labels as larger class label, but if we use class weight in loss function, it would be reasonable to expect the problem to be solved. So why we need some…
user137927
  • 379
  • 1
  • 3
  • 11
13
votes
2 answers

How do I get the feature importace for a MLPClassifier?

I use the MLPClassifier from scikit learn. I have about 20 features. Is there a scikit method to get the feature importance? I found clf.feature_importances_ but it seems that it only exists for decision trees.
jochen6677
  • 591
  • 2
  • 4
  • 9
13
votes
2 answers

What makes a Tree-Structured Parzen Estimator "tree-structured?"

From what I understand the Tree-Structured Parzen Estimator (TPE) creates two probability models based on hyperparameters that exceed the performance of some threshold and hyperparameters that don't. What I don't fully understand is why TPE is…
AAC
  • 509
  • 2
  • 5
  • 13
13
votes
1 answer

How to make my Neural Netwok run on GPU instead of CPU

I have installed Anaconda3 and have installed latest versions of Keras and Tensorflow. Running this command : from tensorflow.python.client import device_lib print(device_lib.list_local_devices()) I find the Notebook is running in CPU: [name:…
Deni Avinash
  • 133
  • 1
  • 1
  • 5
13
votes
2 answers

Conda Verification Failed

I was trying to install jupyter package for anaconda in my current environment but constantly getting the following error. Preparing transaction: done Verifying transaction: failed CondaVerificationError: The package for ipython located at …
Desmond
  • 307
  • 1
  • 2
  • 8
1 2 3
…
99 100