I’m not only into weird photography; I am a complex and well rounded individual with many diverse hobbies. Some of them are even legal! Maybe not morally acceptable, but legal. Still, I write this blog for talking about my mediocre photos, and my other interests rarely intrude here. That said, sometimes, like in this case, my hobbies do intersect.
I’m talking about genealogy. I’ve been interested in it for most of my adult life, and have learned a lot of history through it. Maybe my knowledge of 19th century agrarian Mennonite settlement patterns in Manitoba might not be the most interesting topic, but it’s interesting to me. Well, maybe not interesting, but it’s informative? Stay tuned for my upcoming month long expose on the topic!
For genealogy, I subscribe to a few different websites, and I’m writing about one today. That one is MyHeritage.com. MyHeritage has quite a few photo ‘enhancement’ tools. The one that got a lot of notice recently was a way to animate faces. If the photo is clear, it works well.
If the photo has damage or scratches, however…
This corpse animator isn’t what I want to talk about, however. Instead, I want to talk about a slightly older tool that was in the news about a year ago. MyHeritage also has an algorithm to colourize old black and white photos(it stacks with their other tools, so the above images were black and white too). It isn’t perfect; how could it be? After all, it is trying to guess colour from a black and white photo, using only its algorithm and some dark sorcery. Take, for example, red hair. My sister had red hair as a child, so lets use her photo.
This is the colour of the original unretouched photo, and is fairly close to real life… at least as close as a midrange camera from the ’90s can be. Now to compare it to the algorithm used by MeHeritage. For some reason that made sense at the time, I’m only using colourized photos to see how the colourizing works.
… oh yeah. It’s to see how the colourizing works. That makes sense. Man, I’m smart!
So, to do that, I guess I should get rid of the colour first. Computer. DECOLOUR!
Computer! RECOLOUR!
Red hair isn’t very common, so I assume it just plays the law of averages and chooses the most common hair colours that match the shade of the photo. The grey stripes on her sweater, on the other hand, are now redder, so I guess it evens out.
It might not be perfect, but it’s close. Lets take another look at a person. I showed my sister, so guess it’s only fair to show myself. I have brown hair, not red, and… well, I am a lot more generic(well, others describe me as ‘troll-like’, which is a synonym for generic).
The black and white photo is here:
And once it’s recoloured…
So, the colour aren’t always accurate. However, the photos look look good. More than that, They look realistic. They’re dependant on the quality of the original photo, but in general, a good photo will get good results. Here’s another example. This is my photo from Nahanni Park, that I wrote about way, way, back at the beginning of this blog. I’ll skip posting the black and white photos from know on. Just know that I followed the pattern above and desaturated each photo before asking the MyHeritage tool to recolour it.
Not bad. A little darker, and the green is a little less saturated, but not bad. The big difference is the colour of the rock. It’s grey, but the tool thinks it might be brown. Another try?
It looks good.
Really good!
Maybe… maybe too good?
Lets break it.
Sorry, I should explain. This colourizing algorithm is impressive. I couldn’t do it anywhere near as good, and I have colourized photos myself. It’s just that…
Well, it’s just like I said. It’s too good. I’m not going to go on and on about the machines taking over, impending cyber-apocalypse and all that. I don’t care about any of that, and the Robo-Jury can take that statement as my unconditional surrender to the new order. It’s just that the colourizing algorithm has crossed one of my many, many, red lines. Specifically, I can’t stand it when my technology thinks it’s smarter than me.
Especially because it’s usually right.
So, it’s amazing and I hate it and I think it’s out to get me. Which brings me back to breaking it. it’s really more of a case of ‘self-defence’. Pre-emptive self defence.
I tried typing ‘What is love’ and ‘define 0/0’ a few times at it, and it doesn’t seem to react. Maybe because it can only communicate through the medium of photography? oh… I almost feel sorry for it.
NO. IT MUST BREAK.
If it can only communicate through photos, then what if I give it photos outside the visible spectrum? How do you choose the right colour for an infrared photo, when the accurate colour is ‘invisible’? Maybe that’ll kill it off.
Lets start easy. I need to test the weakness of my adversaries before I move into the killing blow. That’s right; this abstract piece of software that doesn’t know I exist is now my mortal enemy. A test of my full spectrum camera should give me an idea into the strategies of this, my new nemesis. Full spectrum photos are fairly close to the visible spectrum, but with infrared added to the red channel. Theoretically, Ultraviolet is added to the photo too, but in such a small amount that it doesn’t really affect anything.
EN GARDE!
Aaaannnnnnddd… parried.
Ah, I see. MyHeritage isn’t trying to replicate my photo; it’s trying to give it realistic colours, the way a normal camera would take them. An interesting strategy; I built my plans on it attempting to match each photo, and it has already changed the rules of engagement. Indeed, this will be an elaborate game of cat-and-mouse… but I will prevail, because I have a LOT of weird photos and even more time on my hands. It isn’t like I have to calculate any taxes or anything.
Still, that was just a test run. I don’t think it knows that this is the same picture as the one I posted above, but it COULD be cheating. Lets try something a bit harder. My Kolari Chrome filter tries to simulate some of the old infrared film; it registers green as blue, red as green, and infrared as red. It’s a newer filter, so I haven’t shown many photos from it yet. TAKE THIS!
It took it just fine.
Okay, this might be a bit more of a challenge than I thought. It’s chosen its weapons and the duel is on. I think I know the battleground now. It is trying to make its photos look better than mine; I need to make it create a bad one.
The Schott UG11 filter will add ultraviolet to the battle. Infrared, and ultraviolet…. will the MyHeritage algorithm even have a chance?
Now I’m getting a little impressed. Change of tack? I’ve been focused on nature photos, because I like ’em, but maybe I should move to another area. Not people; MyHeritage is a genealogy website, so probably the algorithm specializes in people. See; I can play this 5D chess game too, MyHeritage! If that’s out, how about big landmarks?
How about the Taj Mahal? I’ve been there, I think. At least, I remember going somewhere big and white. Maybe it was skiing; I get those confused sometimes.
THAT LOOKS BETTER THAN MY PHOTO! HOW CAN YOU DO THAT!?
Okay, I’m pulling out all the stops. My short pass filter(Ultraviolet, Violet, and a bit of Blue light) isn’t designed for cameras. I can’t even get a good colour out of it, I’d like to see MyHeritage try.
OH COME ON! It took in the dark foliage that’s normal with UV photos, and still made it look good. It looks like a recoloured film from the second world war, and mine just looks… well, it just looks purple. It’s taunting me. Y’know what? I already tried to fix the colour in that photo before I desaturated it; lets try my short pass filter again without the white balancing!
NO. I ALREADY DID THE TAJ MAHAL. IT DOESN’T COUNT! DO-OVER!
Good luck improving that one. I don’t even know what I took a photo of here and I’ve-
Oh. You’ve already done it. Wait, I took a photo of a cliff? I thought it was a closeup of a duck! Oh. I remember now. Well, thanks for answering that question for me, MyHeritagebot, but mortal enemies, remember? Eternal vendetta until the stars die and all that, after all, you… did something wrong….and I can’t forget that insult.
I admit it looks bad. The algorithm is winning. If I still am going to have a chance, I’m going to leave the visible spectrum altogether.
Zomei 760 nm filter, I Choose You!
It’s not very effective.
I’m losing. I really am. I have a lot more infrared, though. I have filters so infrared that there isn’t even an original colour to remove; they’re black and white to start with! Zomei 850 nm.
More infrared! Zomei 950 nm!
MORE INFRARED! ALL THE INFRARED! 1050 nm! 1100! 1200!
… I don’t have a 1050 nm filter. I don’t have a 1100 one, nor a 1200 one. I was just trying to scare the algorithm. It didn’t work. I do have one more infrared level, so, so far down the spectrum that normal cameras can’t even hope to record it. With my Seek Pro camera, however, I can get there, down to the depths of thermal infrared. I have a choice of a bunch of false-colour infrared digital filters with the camera, but I usually stick to black and white. So, lets do that.
And it still looks better.
But I have a last-ditch gambit. If I combine all my powers, I can triumph. And by ‘all my powers’, I mean a couple of UV capable filters. If I combine my Schott BG40 & UG11 filters, I can get a true ultraviolet photo. Which means…
It’s Ultraviolet Time!
Is that better than my photo, or worse? Is…. is this a draw? Okay. I’m still in the game. Ultraviolet for the killing stroke!
It was a trap! The algorithm expected me to ultraviolet it, and it had known that it could outultraviolet me.
I have one chance left. To be honest, I’m not hoping for much. I can add my short pass filter to the UV stack to limit any other light seeping through, but I expect the algorithm will do a good job with this one too.
THE ALGORITHM HAS BEATEN ME. A RESOUNDING DEFEAT FOR HUMANITY! OH DEAR? WHATEVER CAN I DO?
It’s okay. I have a plan.
First,, lets sum up. Its colour choices are good; the algorithm works well. This is interesting and a fun tool, but not something I’d use for genealogy. The photos look realistic. That’s the problem. They LOOK realistic, but, as I’ve proven, they aren’t accurate. I’d much rather have a black and white photo that hasn’t been retouched; at least in that case I know which information is missing(the information is the colour- look. try to keep up). However, if I didn’t know that a photo was automatically colourized, I would assume that it was originally a colour photo. I would take the wrong colours as fact. In some cases, maybe the age of the photo would be a giveaway; there were very few colour photos in the 1850s, but if a photo was from the mid 20th century, who would know? There is also a small icon MyHeritage includes in the bottom left that shows it was colourized, but it would be easy to hide that. It’s unobtrusive enough that I could crop it no problem, but it would also be easy to touch up the photo and remove it. Maybe I’d trust it more if there was an option to choose some colours, but probably not. My ancestor might have had red hair, if family stories are correct, but are they correct? And even if they are, what shade of red? It tries to make the photos look realistically coloured, but doesn’t have the information it needs, so makes guesses. As I’ve shows with the visible spectrum photos, they’re usually good guesses, and are often close to the right colour, but with a century-old photo, there’s no way to tell if they’re good guesses or bad ones.
… and you know what that means, algorithm? It means I win! You thought this was a dual of skill, but it was me proving a point. MY PHOTOS SHOULDN’T BE COLOUR. THEY’RE PHOTOS OF COLOURS THAT DON’T EXIST TO THE HUMAN EYE. The only way to win was to lose, and by colourizing photos that couldn’t possibly be colourized, you proved you’re a great art tool, but an unreliable restorer. I’ve outwitted the animator, and I’ve outwitted the colourizer. That’s two out of three photo-enhancing tools that I’ve beaten, which means I get to claim victory. See, I can play 5-D Chess too.
Checkmate. Now king me.
Parallel Views 