Snapchat Filters vs. Art Museum

I’m sure most of us have seen our friends posting hilarious gender-swapped versions of themselves all over social media, thanks to a new filter by Snapchat.

Now I have never used Snapchat before — I just don’t know why I’d use it, personally — and I don’t usually go to museums. But that’s all only because I’m not usually a trend follower. On the other hand, I thought that registering a Snapchat account and taking it to an art museum might be fun, just to test the limits of the filter on “selfies” from a few hundred years ago. So I headed over to Stanford’s Cantor Arts Center to find out.

A couple of the paintings had somewhat impressive results. It worked particularly well on male-to-female conversions where the only modifications really needed were some face cleaning, face shaping, and long hair.

It failed miserably though when it comes to beards, and understandably so, as a computer vision engineer myself — this is an in-painting problem, where you’d take the beard out and want to paint a chin and neck in its place. While there have been some impressive recent results in both still image [0, 1], and video [2] in-painting, and I’m sure some of these might perform well on these images if one were to take the beard out of the image, I doubt Snapchat had the resources to incorporate such methods into a phone app. Nevertheless the results are interesting, to say the least.

Female to male conversions seem to amount largely to eyebrow thickening and roughening up the face a bit. Snapchat’s filter evidently doesn’t really know to modify anything but the face though.

Enough with the paintings. Art museums also usually have 3D sculptures, to which Snapchat mostly decided to give them some real human hair without modifying much else. Here are a couple of Ming Dynasty sculptures from China and their gender-swapped alter egos. (An interesting side note: the Buddhist deity Guanyin — not pictured here — evolved from male to female over the centuries and passage through various cultures.)

Lastly here are a few other statues, and and as you can see it really didn’t do well with that last African face mask.

On one hand, we do see that Snapchat’s filter has learned some gender stereotypes (the most obvious one here being that “women have long hair, men have short hair”), but some of the results are comical in its inability to in-paint beards or deal with more abstract interpretations of the human body. Maybe future versions of the filter will improve on this, or maybe the fad will pass and we’ll see other interesting identity-changing filters, including age, hairstyle, or outfit changing.