hudsontitle

NOT EVERY PICTURE TELLS A STORY

Don’t be scared, but there may be strangers in your house.

Anyone lucky enough to inherit their family’s photo album must have wondered about some of the folks in there. Are there unlabeled Victorian-era portraits where people look as stiff as statues? Maybe there are snapshots from a century ago of relatives posing with seemingly close friends – but could they be distant relations you’d like to know about? There may also be missing persons. Why do none of the photos with great-aunt Tilda include her husband Cornelius?

This is a quick trip through a collection of pictures left by a Santa Rosa family from around the turn of the century. Or rather, it’s about forty images that were donated to the Sonoma County Library, which scanned them and placed them online. Not all of the set is interesting – about half are nameless, rigid-necked Victorians – but some are quite unusual and deserve attention. The final section of this article concerns the more serious question about what could be done to restore information missing from thousands of historical photos in our library’s archive and elsewhere.

The first puzzle is figuring out which family we’re looking at. The library’s descriptions usually mention “William H. Hudson” or “part of a collection of Hudson family photographs” but that’s a weak clue, as there were several unrelated Hudson clans in the North Bay at the time and three included a William H. living in Santa Rosa, St. Helena and Petaluma. A few years ago the Sonoma County Historical Society wrote a piece on this and concluded a Santa Rosa businessman was the right guy.1

But in the library collection three different men are identified as William H. Are their photos mislabeled or were these really the trio who shared the same name? And which one is the businessman we seek? Fortunately, he was vain enough to buy an entry in one of the local history “mug books” so we have an accurate reference of what he looked like, at least in his senior years.2

william-toumeyRIGHT: William Hall Hudson (1855-1927)

That history book offers a detailed bio or you can read a condensed version at his entry on Find a Grave (although it presently uses photos of wrong Williams). Here I’m skipping most of the details except for those that apply to the photographs.

Overlooked was that William and his wife Percie had very private lives. Rarely were they mentioned in any of the local newspapers. Aside from a nice announcement in the paper where he once worked as a printer, their marriage was elsewhere a two-line notice in the Vital Statistic columns. There were no birth announcements for their children, even though she was part of a large and prominent Healdsburg family.

Their quiet profile extended to the family album. There’s no wedding photo, nor baby pictures, nor portraits of their son in Army uniform as he went off to fight in WWI, nor picture of the son with his wife or of their children. (Think about that for a moment: Grandparents without a single photograph of their grandkids?!) I want to assume such images did exist and at some point a family member raided the album, but the Hudson’s seeming desire to live behind closed doors does not lend great confidence.

"Portrait of an unidentified couple with their backs to the camera" (Courtesy Sonoma County Library)
“Portrait of an unidentified couple with their backs to the camera” (Courtesy Sonoma County Library)

The iconic picture in the library’s Hudson set portrays a man and woman with their backs turned. The Sonoma Historian newsletter printed it in 2010 as a “mystery photo” and it can also be spotted on social media, where it’s usually tagged as being funny. It may indeed be a gag photo; we have no information about it or if the couple is Percie and William.

We can date it to around the late 1880s because of the “Souvenir Studio” credit.3 Photographer Jim Piggott seemed to be a light-hearted fellow who might encourage goofiness; his advertising motto was “A man may have A BAD DISPOSITION Nevertheless he may conceal it by having his photos taken at the Souvenir Studio.”

Also in that period the Hudsons lost a child – the mug book biography stated one of their two kids died in infancy. We can guess that happened in 1886 or 1887 because an item in the May 1886 Democrat noted Percie had returned to Santa Rosa from a long visit in the Midwest with her “little daughter” Ethel. Per the Hudson’s pattern of reclusiveness, there was no death notice, as there had been no birth announcement.4

This photo is likely a “mourning portrait” taken to memorialize Ethel’s passing. Americans in the Victorian era were somewhat more restrained than their Brit cousins in taking creepy post-mortem photographs of their dead and poses similar to this were an alternative. The symbolic objective was to conceal faces of the mourners because they could not bear to be seen, as they were wracked with indescribable grief. (Some were quite artistic, with the eyes and upper face hidden in noir-like shadow or the head sharply turned away from the camera.)

"Portrait of Willam H. Hudson, about 1863" (Courtesy Sonoma County Library)
“Portrait of Willam H. Hudson, about 1863” (Courtesy Sonoma County Library)

Seven portraits in the library’s Hudson family collection are supposedly William as a child, but like the adult portraits they appear to be two (or more?) boys; colors of the eyes and hair vary and the ears may jut out (or not). In one bizarre offering the boy’s hair is crudely drawn in along with his eyebrows, which were placed so high on his forehead he could pass for a Vulcan.

Live long and prosper, sketchy Earthboy
Live long and prosper, sketchy Earthboy

Another quirky example of Victoriana can be spotted in the portrait that’s assumed to be William at age six or seven. Notice the curtain is unusually draped behind his right arm and over the corner of the chair. That was a typical trick used to conceal a mother or other adult for what’s now called “hidden mother photography.”

As the Wikipedia page explains, it was difficult to keep small children completely still during the long exposure times required by early cameras, yet the Victorians didn’t want the distraction of including an adult in the picture. Their solution was to hide the mother under a blanket or behind something, such as the curtain seen here. Presumably the mother is clutching the boy’s hidden left hand and chanting, “holdstillholdstillholdstill…”. Hidden mothers were most commonly used while photographing infants, but perhaps William was an unusually squirmy child.

"William H. Hudson and daughter, Santa Rosa, California, about 1900" (Courtesy Sonoma County Library)
“William H. Hudson and daughter, Santa Rosa, California, about 1900” (Courtesy Sonoma County Library)

The library caption on this image is incorrect, as his only daughter died in infancy. We don’t know who she is, but comparing this man to the portrait in the history mug book shows this is indeed William, and is the only bonafide picture of him in the Hudson family collection.

Today we would call William an entrepreneur, as during most of his adult life he owned and operated different businesses at the same time. Trained as a printer he became the proprietor of hotels in Ukiah and Windsor. His main income came from plants that bottled soda pop and mineral water, which he ran in Mendocino and Southern California before buying the Santa Rosa Bottling Works in 1887.

The girl’s fan advertises “Sassafras Sour” which was probably among the brands of root beer he made under license. Other carbonated soft drinks included “Cresta Blanca” (not the wine), “Ly-nola” (described as a ‘fruit beverage’), “Oyster Cocktail” (stored at room temperature for how long?), “Whistle” (maybe the best name ever for a soda) and “Dr. Swett’s” (maybe the worst).

William upgraded his bottling plant on West Third and the Press Democrat ran little items about new state-of-the-art carbonation machines and such. He expanded his territory and opened a branch store in Occidental before selling the business and retiring in 1924.

"Gold Lion whiskey--perfection--the Old Crow whiskey" (Courtesy Sonoma County Library)
“Gold Lion whiskey–perfection–the Old Crow whiskey” (Courtesy Sonoma County Library)

“Dear Dr. Freud; please take a look at this photograph. Seriously, WTF? Sincerely, William, Santa Rosa Ca.” [parody]

This advertisement cabinet card is the oddest artifact in the Hudson collection and, I’ll wager, the champion oddity in the library’s entire photo archive. Nothing is known about it except that the back reads, “‘Compliments of Tom Spencer, Livingston & Co.’ for Gold Lion whiskey, 1234 Deer Run”.

William likely had this because of another of his companies. From 1892 to 1901 he had the largest retail and wholesale liquor store in Santa Rosa, first on Exchange Avenue then moved to lower Fourth St. Yet curiously, he later acted as if it never existed – that long-running business went unmentioned in his history book bio and in the obituaries.

"Mrs. A. H. Bates, Capt. M. V. Bates" (Courtesy Sonoma County Library)
“Mrs. A. H. Bates, Capt. M. V. Bates” (Courtesy Sonoma County Library)

The Hudson album also included a souvenir card from a circus sideshow with Anna and Martin Van Buren Bates, the “tallest couple in the world” according to a blurb printed on the back. The man of “ordinary” height next to them was about six feet tall. The Bates were part of the W. W. Cole circus which played Santa Rosa on October 6, 1880.

It was not uncommon to include such photos in a family album. My own family’s collection had a portrait from about that same time of an adult couple with dwarfism, but no caption explaining who they were or if they were relations. (This caused high anxiety in my cousin when we were kids, as she was slightly shorter than her schoolmates and convinced herself she had stopped growing.)

This keepsake cabinet card and the whiskey ad above make the point the albums weren’t just archives of dead ancestor portraits. They were storybooks. Particularly in the age before radio and TV, the family album was an important source of entertainment – a display of assorted interesting people to talk about. Other identified (and presumably unrelated) portraits connected to the Hudsons included a Petaluma minister and the vice president of the United States under Grover Cleveland.

An album might be pulled from the shelf when relatives or old friends stopped by to visit and the pictures unlocked memories: Perhaps someone would say, “Oh, here’s grandma Gus. Remember how blue were her eyes?” The visitors might chip in with their own stories and for a few nostalgic moments the clock turned back as she was recalled tenderly (or not). And then everyone had a good laugh over the card of the girl with riding crop in her teeth on the next page.

"Unidentified male member of the Hudson family, about 1880" (Courtesy Sonoma County Library)
“Unidentified male member of the Hudson family, about 1880” (Courtesy Sonoma County Library)

The photo of this unknown workman is (in my opinion) the most intriguing of the Hudson set. There must be quite a tale behind it; Victorian-American tradition was to wear somber “burial clothes” and pose in the style of formal classical painted portraits. Everyone else in the Hudson album was a model of respectability to be honored for a life well lived.

It’s not just that he’s dressed as a heavy laborer and looks like he’s taking a break from ditch digging. His hands are rough, dirty and he’s holding them in a way to draw attention to them. Having that big cigar clenched in his teeth makes him seem all the more defiant over having to sit for a portrait. “Ya’ nagged me for a damned picture and here it is. Hope yer’ happy.”

Some other unknown portraits from the Hudson collection in the Sonoma County Library
Some other unknown portraits from the Hudson collection in the Sonoma County Library

We don’t know the name of our stogie-chompin’ pal, nor do we know when he was photographed. Among the Hudson set there are eighteen others of unknown people from the family or supposedly connected to them. Of the 31,000+ images in the Sonoma County Library Photograph Collection, I imagine there are easily a thousand with one or more unidentified figures. And that’s just for our little slice of heaven – multiply that by all the holdings in university and library archives and posts on social media nationally. A truly staggering number of pictures show people who have no names and likewise no stories.

With a family album like the Hudsons where nearly half of the entries have no-names, the temptation to speculate can be difficult to resist, even for experienced genealogists. That middle-aged couple photographed in Iowa, where the Hudsons lived when William was a child – it must be safe to assume they are his parents, right? Well, no; they could easily be relatives from far-flung branches on the family tree or just friends.

Is there a solution to this identification problem? I believe there is – or at least, a means to ID lots of these mystery people.

It should come as no surprise that late 19th century studio photographers made their real money on selling copies, not taking the picture itself. Customers were sold packs of cabinet cards like the ones seen above to be given away to family and friends (prices ranged around 4/$1.00 and 12/$2.00). This means there could be surviving duplicates of these cards floating around in other online collections. And some of those surely have details about who’s in them.

Much has been written about AI companies scraping the internet for text that could be reused for apps such as ChatGPT. Lesser known is that they also collect image data on the web and reverse image search apps, such as Google Lens, can find other copies of the same portrait. In theory.

For better or worse you can’t do that, but not for any technical reasons. In Google Lens any image search for a person comes with an alert which reads, “results for people are limited”. Experiments using images from the Hudson collection results in no exact matches but plenty of lookalike portraits, almost all from commercial sources such as eBay, Etsy and antique vendors where they are available for purchase. Yet we know Google has indexed the Sonoma County Library pages because it finds the captions from the photos. What’s going on here?

Google and its ilk are justly concerned image search tools can be used to easily violate privacy – there’s no foolproof way to determine whether the picture of someone’s face came from a century-old source or was taken sans permission using a phone five minutes ago. The risk of all manner of wrongdoing is indefensibly high.

But as you can probably imagine, there’s also plenty of complaining on social media that this approach is overbroad. Not only does it restrict legit research such as genealogy, it also can enable crime by helping identity thieves keep themselves concealed.

Unless I’m missing some fine legal hairsplitting, it seems to me the solution to the researcher’s dilemma is to recognize that exact matches for images now in the public domain are fair game. That’s how Google and Internet Archive handle books; currently anything produced before 1929 (the 95 year limit of copyright) is available for full download without restrictions. Similarly, you should be able to find other copies of your ancestor’s Victorian era portrait if Google et. al. know where they are. Or to put it another way, the search engines with this particular information do not have a legitimate reason to hide it from you.

There’s no dispute Google Lens is the 800 lb. gorilla in the world of reverse image search, so this is an issue only that company can solve. Perhaps they should split off (what surely is) the small niche of historic image searching to a different body, such as an academic institution or Ancestry. They could even drop the facial recognition module – ain’t nobody’s going to use a portrait of your g-grandma to catfish lonely men on dating sites.

Linking all the image archives on the internet with simple matching could be an incredibly powerful tool. Historians may discover photos which were described at the time but have been long considered lost; genealogists could find unknown branches of their family. And we even might learn that all the strange business packed into that whisky ad was just symbolism connected to silly rituals performed by a group like the Elks’ club in the day.


1 “Lots of Names, Few Photo IDs” Sonoma Historian 2010 #4, pg. 14-15
2 Honoria Tuomey, History of Sonoma County, California Vol. II, 1926, pg. 548-549
J. K. Piggott’s studio at the corner of Fourth and B street first advertised in 1888 and he sold it a year later, then repurchased it in 1890 before selling it again the next year. So the photo had to be taken 1888-1891 except for a gap between Nov. 1889 and Aug. 1890.
4 Confusing matters further, there was another Santa Rosa girl named Ethel Hudson, born in 1879. She was the daughter of daughter of Henry W. Hudson and on at least one occasion the Democrat confused H. W. Hudson with our W. H. Hudson. It appears there might be a link between those two Hudson families. The mother (Mrs. H. W. Hudson) of 1879 Ethel was part of the Northcutt family. Percie was the aunt of a Miss Lou Northcutt – see reference in sources below. Tracing it further, however, is beyond my genealogical skill set.

 

All images above from the Sonoma County Library Photograph Collection have been slightly modified, usually to brighten the photo and improve contrast. The entry with Hudson and the girl was flipped horizontally so the text on her fan would be legible

 

sources
 

MARRIED. PALMER-HUDSON — In Healdsburg, July 10, 1881, at the residence of the bride’s parents, by Rev. S. A. Taft, Miss Percie Palmer of Healdsburg to Wm. H. Hudson of Ukiah. With the above happy announcement came the largest package of the best-made wedding cake, and we join the many friends in wishing the young couple much joy. The groom is a nephew of T. W. Hudson, Esq., of this city. Arriving in this State a few years ago in his teens, he aided us a few times in printing the Flag; bis course since then has been characterized by enterprise and industry till now, by his own exertiona he has become a partner in a good business in Ukiah, as well as the possessor of one of the fair young ladies of Healdsburg. How many of our young men have made a better record? We congratulate both of the young people and have unbounded confidence in their success and happiness.
– Russian River Flag, July 14 1881

Mrs. W. H. Hudson and little daughter Ethel returned from Columbia, Missouri, where they been visting for some time past. Mrs. Hudson’s niece, Miss Lou Northcut, accompanied them, and will spend the summer with her aunt in this city.

– Daily Democrat, May 14 1886

HEART ATTACK HELD CAUSE OF HUDSON DEATH
Retired Businessman, 71, Drops Dead in Street; Final Rites Thursday

William H. Hudson, 71, for 40 years a resident here and founder of the Santa Rosa Bottling Works, dropped dead in Third street within two blocks of his home yesterday morning while on his way uptown. Heart trouble is blamed for his sudden death, although he is said to have been in good health and is reported to have remarked to members of his family before leaving home how well he was feeling. He was picked up by passersby, who saw him fall near the cannery. Although Coroner Fred Young could not be reached last night, it is probable there will be a formal investigation, the date for which has not been set.

Funeral services will be held Thursday afternoon at 2 o’clock from the Welti chapel on Fourth street, with the Rev. J. Allan Price of the Baptist Church officiating. Burial services will be held in Odd Fellow’s cemetery by Santa Rosa Eagles, of which Hudson member. Hudson was born in Missouri on December 24, 1885 [sic], and when a child went to Iowa and then to Nebraska with his parents, where he learned the printer’s trade. On first coming to California in 1882 he entered the hotel business at Ukiah, later establishing a bottling works in Mendocino.

Five years later he transferred the business to Visalia and in 1887 he moved to Santa Rosa, where resumed business. He retired in 1924 and since that time has not been actively engaged in business. He married Miss Percie Palmer. daughter of Mr. and Mrs. J. S. Palmer, Healdsburg pioneers, on June 10, 1881. Clyde V. Hudson is the only surviving son of the two children born to the couple.

Besides his son is survived by two sisters, Mrs. Theodicia Schroder and Mrs. Allie Tolson of Long Beach.

– Press Democrat 1 November 1927

Read More

3ew-smaller

COLOR ME THIS

I actually gasped at seeing the enhanced image – I had no idea the technology had advanced so far.

Over the years I’ve dabbled with applying colorization and other image processing effects to old pictures. Anyone interested in history or genealogy has likely done the same; probably all photo editing apps have at least a few enhancement tools built-in, and some of the more powerful online versions use AI to guess at the contents of the image for automatically adding color to black and white photos.

Results were rarely satisfying. Colorized images looked washed-out and the color choices could be laughably wrong. And because the underlying software was developed using modern photographs taken in color, processed black and white images might even lose quality – old portraits often have a shallow depth of field, for example, and apps may “fix” that by sharpening up the background.

Then on a whim, I recently took a photo from a 1920 Press Democrat and uploaded it to an AI website. I didn’t expect much improvement; my experience was that the software would probably despeckle the picture but not materially improve it. Still, I wanted the best possible image since the woman was key to the article I was researching.

To repeat myself: I actually gasped.

LEFT: Original photo from the Press Democrat, June 30, 1920. RIGHT: Hotpot enhanced image (see article).
LEFT: Original photo from the Press Democrat, June 30, 1920. RIGHT: Hotpot enhanced image (see article).

There was so much signal noise in the PD original it was difficult to read the woman’s expression; was she glaring angrily at the photographer? Did she look tired, or even sick? But with one click of the mouse, out of that hazy static emerged a clear and sharply-focused image of a woman’s face – with eyelashes, even! (This is the same woman shown in the graphic above the title.)

I took that portrait and ran it through a different AI-based program for colorization. The result was acceptable, but now she had the pale, sepia skin tone often seen in colorized photos.

Curious if colors in the photo could be improved further, I opened it in a graphics editor. Using just two menu options, I was able to give her a suitably realistic complexion. 1 Even though the portrait was still far from perfect, you could almost imagine it was a selfie made today – as long as you don’t squint too closely.

LEFT: Hotpot enhanced image colorized by Palette. RIGHT: The colorized version further enhanced using GIMP (see footnote 1).
LEFT: Hotpot enhanced image colorized by Palette. RIGHT: The colorized version further enhanced using GIMP (see footnote 1).

WHAT WAS USED FOR TESTING

To test colorization, my criteria was that the apps had to be available to everyone. This meant that they had to be on the internet, free to use, and work automatically without technical knowledge or experience.

Except as noted, all images in this article were generated using Hotpot to first sharpen and improve the black and white photo, followed by applying the “Fix Face” enhancer if the image was a portrait. Colorization was done using Palette which offers a variety of filters which can be applied to the image. Both Hotpot and Palette require payment for processing larger images.

Other colorization apps tested include:
DeepAI
DeOldify
Img2Go
Hotpot (Colorize)
Fotor
PixelCut

The entire process took less than ten minutes, and most of that was spent experimenting with menu options in the editor. All of the tools were free. Anyone can do this.

Automatic colorization is really quite a breakthrough, particularly when you consider it combines two remarkable achievements that were both considered futuristic stuff just a few years ago.

The first step is object recognition – how the computer recognizes without any prompting from the user that the image is the face of a woman, as opposed to the picture of a truck, a cat, or something else. It wasn’t until 2001 before anyone demonstrated that could be done in real time.

Gentle Reader should recall from the previous article how AI chatbots were “trained” using text scraped from the internet. Image processing AI programs are also trained on datasets, such as Stanford’s ImageNet which now contains over fourteen million images organized by descriptions.2 That enabled a giant leap forward in being able to correctly identify various parts of an image as well as building a library of color information about the objects.

Although all colorizing AI apps start by tapping into the same basic training about what’s in the image, what they do with that data can produce widely different results, although all default to very conservative color choices. The differences lie in how much – and how well – the colors can be tweaked to produce a result pleasing to your eye.

(RIGHT: Same location today)

In the first colorized image below the Img2Go app recognized there was lots of foliage, which the AI decided is always the exact same shade of green, no matter what kind of tree. The street and sidewalk are likewise greenish. (Another app lightly tinted both the sky and street blue.) As a final gripe, Img2Go super-sharpened the entire image without asking.

Colors were far more realistic in the lower image, particularly the palm trees. The sidewalk and (unpaved) street look right and although the sky is light overcast, it’s not so dark as to make the shadows look out of place. But despite those improvements, it’s still easy to see this photo has been colorized.

The present corner of Mendocino Avenue and 7th Street looking north towards College Avenue, c. 1905. (Photo courtesy Larry Lepeere collection)
The present corner of Mendocino Avenue and 7th Street looking north towards College Avenue, c. 1905. (Photo courtesy Larry Lepeere collection)
Colorized image by Img2Go
Colorized image by Img2Go
Colorized image by Palette
Colorized image by Palette

The sad truth is that no matter what is done, these images will likely always look colorized. Future advances in AI will most likely make better color choices, but there are limits because of issues with the quality of the original photograph itself.

Most black and white photos taken before circa 1930 used film that pushed contrast higher than a human eye would see; it was also not very sensitive to red light and over sensitive to blue (this article explains more). As a result, what was blue in real life appears lighter in a vintage black and white photo while red areas look much darker, sometimes appearing almost black. So when looking at a picture taken during the “Golden Age of Photography,” keep in mind it’s actually quite distorted and information has been irretrievably lost. Ideally some of this info could be provided in object descriptors, such as “Photographed 1921 using Kodak Verichrome (orthochromatic).” Ha, ha, no.

Making matters worse, the current crop of AI colorization apps train using modern photos. Pictures are desaturated and then the image generator tries to predict the correct colors, testing accuracy by comparing them to what’s found in the original (here’s an article with examples). Needless to say, a modern photo with its colors removed still looks like a modern photo because it keeps all the same tones. As I understand it, this is a reason why colorized vintage images often look brownish and washed-out – the color model is simply wrong. It’s like trying to translate a novel written in Spanish by using an Italian dictionary; the surprise is the trick can work as well as it does.

Yet it seems to me the people building these apps don’t understand users want to colorize OLD photos, and not, you know, re-colorize a color picture they just took with their iPhone. All of the colorizing apps I’ve tested using their automatic option performed no better than fair on vintage images. All, that is, except for one: Palette.

Palette is far from perfect and like all the other apps it has bugs, particularly sometimes not coloring within the lines. It offers twenty preset filters which can be applied to an image; some are “meh” like its competitors and some may be flawed – but often there are two or three that are gasp-worthy.

Developer Emil Wallner seems to have an understanding of vintage film – both color and black and white – that others lack. The results often have warmer tones and bolder colors, both which can look more historically accurate. Palette also looks at small gradations in the black and white image to offer dramatic color choices.

Below is an auto decorated for the 1910 Rose Carnival (more details). Other colorizers painted the floral decoration completely green – as did the default Palette filter. But in the original image the flowers over the hood and fender are slightly lighter. Per the old film being more sensitive to blue, Palette painted them that color. This surely must be a bug, right? Roses ain’t blue! Yet it turns out Palette’s color choice isn’t bad at all, because those aren’t roses, or even actual flowers. They’re fakes made out of colored paper. We know that because of an item in the Press Democrat from two years earlier, where the Floats Committee declared parade entries could no longer combine artificial floribunda with the real thing. Since the headlights and other parts of the auto are clearly paper, the rest of the decorations must be as well.

1910 image colorized by Palette. (Photo courtesy Rockwell family archives)
1910 image colorized by Palette. (Photo courtesy Rockwell family archives)

The next image is from around 1915 and shows a woman cuddling her kitty on the front steps – a poignant snapshot that has long been a personal favorite. I’m guessing the year to be c. 1915 based on the size of the Yucca, which was probably planted after the house was built in 1910 (the home is still there).

This is a deceptively difficult image to get right. All of the AI colorizers (including Palette’s “base” version) barely apply any tint at all, rendering it in such pale browns I sometimes struggled to see any color. But again, Palette found small measurable differences in the gray levels that suggest the front of the house had five shades. As with the “blue roses,” we can’t know if Palette’s colors are what actually appeared, but they are historically appropriate. This image also demonstrates Palette’s bug of not always identifying object borders correctly – look at the color shift on the window frames.

Circa 1915 image enhanced by Hotpot then colorized by Palette. (Photo courtesy Larry Lepeere collection)
Circa 1915 image enhanced by Hotpot then colorized by Palette. (Photo courtesy Larry Lepeere collection)

Some images defy colorization, at least using any of the AI-based apps currently available. This rare 1890 photo of Santa Rosa House, the famous old hotel at the corner of First and Main, is only partially colorized, with many people left in black and white. Palette performed marginally better than others, but this is a common problem with all AI colorizers when there’s not enough data to recognize an object. To be useable this photograph would need to be rescanned at a much higher resolution and have additional preprocessing enhancements. The horses look good, though.

1890 image enhanced by Hotpot then colorized by Palette. (Photo courtesy Sonoma County Library)
1890 image enhanced by Hotpot then colorized by Palette. (Photo courtesy Sonoma County Library)

The final image is a 1885 portrait of Julio Carrillo. Palette did its usual fine job with color but like the portrait of the young woman discussed at the top, the real improvement is Hotpot’s “Fix Face” option, which adds dimension to his skin and beard.

But the garbled image of the woman was significantly improved while the overall quality of Julio’s portrait was not. In the original his eyes look more thoughtful; Hotpot very slightly lowered his eyelids, which is only apparent when you do a blink test.

1885 original image and a version enhanced by Hotpot then colorized by Palette. (Photo courtesy Sonoma County Library)
1885 original image and a version enhanced by Hotpot then colorized by Palette. (Photo courtesy Sonoma County Library)

So here’s the $64,000 Question: Should we be colorizing at all?

The “pro” side falls back to the same few arguments: People today expect color; photographers were hand-coloring back in the 19th century and no one objected; colorization can bring out hidden details. And let’s face it, being able to colorize a picture with the click of a mouse or tap on a screen is just pretty damn cool.

The “con” side begins by pointing out the apps are still doing a lot of guesswork. Yes, Palette has an remarkable ability to mimic old film and apply historically accurate colors, but it has no way of knowing what some of the original colors were – on the auto flowers, yellow would have been as good a choice as blue. Why is the woman’s hair a different color than her eyebrows? And although her forehead did not appear strikingly darker in the poor quality newspaper original, Hotpot turned it into a change of skin tone (or lighting?) which was further exaggerated by Palette. It feels like the same issues I raised with AI chatbots, which too often answer basic history questions partially or completely wrong. The technology may work satisfactorily 90% of the time, but when – and if – we will clear the hurdle of that last ten percent always seems a few years away.

And then there’s the briar patch of ethical issues, both because of the tech and those who use it. There’s been much discussion in articles and online forums about skin color automatically being lightened, and whether or not this is because the majority of faces in those mammoth image datasets are Caucasian. There are also examples of AI colorizers darkening the skin of Black people so much that facial features are nearly lost.

Colorization always has been somewhat controversial (here’s an article mostly about pre-AI objections) but there was an uproar in 2021 after an artist altered photos of Cambodian genocide victims, sometimes adding smiles to their mugshots as well as colorizing them. In the wake of that incident a “Colorizer’s Code of Conduct” was written, which I encourage everyone who’s planning to try AI colorization to read. It’s short and hits all the important points.

Even if you’re simply colorizing your grandparent’s wedding portrait to post on FaceBook, it’s still important to acknowledge the image has been significantly altered and an original black and white version exists – that thread of history must be unbroken. And who knows? Perhaps someday a member of your family will want to re-colorize it when we have AI apps that work more reliably.

 


1 The final image enhancement was done using GIMP, a free Photoshop-like app that will run on all desktop computers. Three options from the “Colors” menu were used:
Auto > Color Enhance (this maximizes saturation)
Hue-Saturation reduces max saturation (-40)
Brightness-Contrast, adjusts brightness (+75) and contrast (-10)

2 ImageNet initially received a huge boost in the number of images and descriptors in its dataset by incorporating data from Flickr, one of the earliest photo social media sites that allowed users to add tags describing the picture. Descriptions are also being crowdsourced, which has furthered the problems of troublemakers adding malicious tags, including racist slurs and disinformation to the descriptors.

Read More