How should I stack up optical flow along axes to pass it to a neural network

Question

I have extracted the optical flow along x and y axes. I want to pass them into a ConvNet. The thing I cannot understand is whether these should be two different input channels or should I combine them in some way, like stacking them, adding them or averaging them

Paper- Two-Stream Convolutional Networks for Action Recognition in Videos

score 1 · Answer 1 · answered Apr 17 '18 at 02:14

1

Stack them so that it is an input with two channels. This is the standard approach.

answered Apr 17 '18 at 02:14

shimao

26,092

Do you mean the optical flow images will have 6 channels, 3 for each ? – userlatebloomer Apr 17 '18 at 08:25
No, one channel is for the $x$ component of the flow, one channel is for the $y$ component of the flow. There are two channels in total. – shimao Apr 17 '18 at 08:58

How should I stack up optical flow along axes to pass it to a neural network

1 Answers1