9

Recently Google publicized interesting deep dream. Besides art generation such as http://deepdreamgenerator.com/, do you see any potential applications of deep dream in computer vision or machine learning?

5 Answers5

4

There's already at least one application out, if you interpret 'application' broadly enough: Decoupled Deep Neural Network for Semi-supervised Semantic Segmentation by Hong, Noh and Han. They use it for image segmentation. Standard image recognition networks can only give you a bounding box for each object recognized on an image. If you want to know which pixels constitute that object, you have to do image segmentation.

Basically, after finding a dog on an image, Hong et al's architecture back-propagates the dog-ness through the neural network down to the pixel level, to find the pixels that were the most responsible for the dog appearing. (They then use this heatmap as input for a supervised segmentation network, there's no deep dreaming in that part.)

This is already kind of an existence proof that the Deep Dream idea can be useful outside image manipulation. But I wouldn't downplay image manipulation itself either. I mention two things that are not immediate applications of Deep Dreaming, and we don't have them currently, but I can kinda see a plausible road from the original Deep Dream algorithm towards these:

  • Beautifying pictures and human faces and bodies. (Automating what a Photoshop retouch artist does.)
  • CSI-style image upscaling with fake but believable interpolated detail.
Dániel Varga
  • 41
  • 2
2

heres another application that is very new & just demonstrated within last few weeks. computers are filtering images to look like paintings in the distinctive style of different artists eg Van Gogh, Picasso, etc... and it seems possible since the technology can encompass different artistic styles it might be used for forgery detection in the art world at some point. (many very advanced analysis techniques are used in this area historically.) note that filtering methods are very popular on Instagram so it seems likely these will be commercially available at some point.

vzn
  • 129
  • 4
1

It is impossible to prove a negative, but other than using the same pattern detection system in general to detect shapes/images and replace them with other similar images, possibly for use in automatic image correction or similar, I don't think it has real potential outside of modifying pictures.

I may have to delete this answer if it is proven wrong.

DoubleDouble
  • 121
  • 3
  • 1
    Well there is a valid use as a visualisation/introspection tool to find patterns that your network has learned. In this link http://googleresearch.blogspot.ch/2015/06/inceptionism-going-deeper-into-neural.html see the discussion about the dumbbell classifier. Not sure if that counts as a true purpose, since it is self-referential – Neil Slater Aug 13 '15 at 15:48
  • I think that's what makes the question difficult to answer, there are things behind deep dream that are widely applicable depending on how far you go, but is it still considered "an application of deep dream"? To me, it seems like deep dream is using an application of those techniques - which is what is applicable elsewhere. But I can't view that link at the moment so maybe I am incorrect. – DoubleDouble Aug 13 '15 at 16:02
1

Greyscale to Color

For example:

http://s15.postimg.org/3xq8jx03f/image.jpg

to

http://s15.postimg.org/i5fx8kcsb/image.jpg

http://s15.postimg.org/c5s64wrzv/image.jpg

The tree wood seems unnaturally red but still, it's not bad. This has worked but less impressively with other greyscale images I've tried.

alan2here
  • 111
  • 3
0

Naritivly context aware, visual profanity filter.

In other worlds, rendering physically realistic and thematically/stylistically appropriate clothes on people who are insufficiently dressed, to make the image more family safe.

Thats the idea, however at the moment it's both unreliable and when it does work inaccurate.

However, more tweaking of the parameters of the dream than I have access to, or possibly just using more iterations and a lower "octave" value than I can specify should make the results much more reliable.

Examples:

Before: http://s22.postimg.org/5sjpqbzoh/image.jpg

After: http://s22.postimg.org/wew6fb3vl/image.jpg

.

http://s13.postimg.org/c4urz139j/image.jpg

alan2here
  • 111
  • 3
  • Can you augment this by explaining what you mean with this example? otherwise it's just links. – Sean Owen Sep 17 '15 at 13:41
  • 1
    I think the problem with this idea is it doesn't really align with how Deep Dreaming works. You would need to train a network to recognise "suitable clothing", but it would not then appropriately dress unclothed figures - instead it would output drapery-looking stuff over places that already looked a bit like pieces of suitable attire. I.e. it is more likely to make a tree trunk into a trouser leg than to put a dress on a girl in a bikini. Deep Dreaming doesn't pick targets for replacement like an image regular-expression engine, it hallucinates matches in like-for-like way. – Neil Slater Sep 17 '15 at 16:56
  • See http://cs.stackexchange.com/questions/47262/google-deep-dream-has-these-understandings I can't demonstrate or prove much, as it's mostly personal experience and observation and I havn't be giving it many nude people, but I think it is smarter than you give it credit for, although I do appreciate that my examples do indeed look quite cobbled together :-P – alan2here Sep 17 '15 at 19:16
  • 1
    The pictures are good - amongst some of the most interesting I've seen from Deep Dreaming. However, I think the search for deeper meaning and structure beyond clever pattern matching is like looking for general intelligence in a dissected retina . . . there is a level that bigger/faster/deeper networks just trained on images will not take us to - something more is needed. – Neil Slater Sep 17 '15 at 21:09