how to normalize data 'with a sample range from -1 to 1 and a mean value of 0'?

Question

I am trying to pre-process data following a statement in a paper.

They said

for the normalization, each dataset is normalized on a per channel basis with a sample range from -1 to 1 and a mean value of zero.

$X_{rescaled} = a+(b-a)\frac{X - min(X)}{max(X)-min(X)}$, following this, it can rescale data $X$ into an arbitrary interval $[a \ b]$, but it can't promise the mean is zero.

Probably I didn't get the statement here, anyone has any idea about this?

Answer 1

I believe the intended application of this preprocessing technique is to first standardize the data with mean zero then change its range to $[-1,1]$ for a neural network input. To quote this referece :

You will get better initializations if the data are centered near zero and if most of the data are distributed over an interval of roughly [-1,1]

However, in all cases, applying first a standardization technique to get zero mean and then apply a new data normalization to map your features to a certain range is meaningless... it is equivalent to normalizing only $X$ since the standardization step does not change the min/max values (check this question).

What I would suggest is only standardization to zero mean and unit variance like so

$$ X_{i}{scaled} = \frac{X_{i} - \mu_{X_i}}{\sigma_{X_i}} $$

where $\mu_{X_i}$ and $\sigma_{X_i}$ are computed for every feature $i$. Then if you really want to have a range of $[-1,1]$ and your data is Gaussian, you can simply filter them accordingly which according to a Gaussian distribution should give you about $68\%$ of your data. In the reference that I quote, they say 'most of the data' which in this case means $68\%$ if it is Gaussian. But this seems to me like an overkill since you loose too much data.

On the other hand, if your data is not Gaussian then clipping values will result in an even greater loss of data (and information). So one should choose a better mapping (e.g. logarithmic, box-cox, and so on...) to make data Gaussian-like first, then standardize to zero mean and unit variance, then clip to $[-1,1]$ if necessary. An example using a simple logarithmic transformation for non Gaussian features:

$$ X_i = log(X_i + \epsilon) $$ then $$ X_i = \frac{X_{i} - \mu_{X_i}}{\sigma_{X_i}} $$

See scikit-learn's implementation of standard scaler here and their implementation of non-linear data transformers here.

Update: I found this scikit-learn page that discusses tips to tackle deep learning data preprocessing very useful. They do indeed mention that it is recommended to either scale data to $[-1,1]$ or standardize it to have zero mean and unit variance. Applying both is meaningless as shown in this answer that @Tim so kindly provided.

Answer 2

The following code in R was my attempt of to solve the problem of finding an algorithm to convert a vector of values to a new vector of values with mean of 0 and range of [-1, 1].

After one pass, it doesn't quite work. (The result will have the specified range, but the mean will be off-zero, though usually not by too much).

However, it appears that if it's applied iteratively (feeding the results back through the algorithm), it will reach the desired result within a certain tolerance.

I doubt this transformation is practically useful. And one could probably come up with a simpler process to arrive at a similar result.

Briefly, it centers A on mean of 0, and then divides this new A_center into those values above and below 0 (A1 and A2), adds a value of precisely 0 to each, and then applies a linear transformation to each of these to fit a range of [-1, 0] and [0, 1].

### RUN THIS ONCE, WITH A AS THE INPUT VECTOR ###
A = c(1,2,2,3,3,3,4,4,4,4,5,5,5,5,5,8,8,8,8,8,8,8,8,8,8,8)
A_trans = A
RUN THIS ITERATIVELY UNTIL THE RESULTS ARE WITHIN TOLERANCE
A = A_trans
hist(A)
A_center = A - mean(A)
A1 = c(0, A_center [A_center < 0])
A2 = c(0, A_center [A_center > 0])
A1_scaled = ((A1 - min(A1) * (0 - -1))/(max(A1) - min(A1)) + -1)
A2_scaled = ((A2 - min(A2) * (1 - 0))/(max(A2) - min(A2)) + 0)
A_trans = c(A1_scaled[2:length(A1_scaled)], A2_scaled[2:length(A2_scaled)])
hist(A_trans)
Sum=data.frame(
  N    = c(length(A),length(A_center),length(A_trans)),
  Mean = c(round(mean(A), 2), round(mean(A_center), 2), round(mean(A_trans), 2)),
  Min  = c(round(min(A), 2), round(min(A_center), 2), round(min(A_trans), 2)),
  Max  = c(round(max(A), 2), round(max(A_center), 2), round(max(A_trans), 2)),
  CountLess0 = c(sum(A < 0),sum(A_center < 0),sum(A_trans < 0)),
  CountGreater0 = c(sum(A > 0),sum(A_center > 0),sum(A_trans > 0)),
  row.names=c("A", "A_center", "A_trans")
)
Sum