Meet the Upscalers

When you’re a kid, sometimes a TV show or film can leave an outsize impression on your subconscious. And, sometimes, revisiting that material can be disappointing when you’ve grown, especially in the harsh glare of a HDTV screen. But fans across the world are trying to use technology to help bring the past up to the same standard as the present. Meet the Upscalers.

There are plenty of examples of professional projects that have tried to preserve and improve older material. CBS spent big to remaster Star Trek with new CGI effects and Star Trek: The Next Generation for the Blu-ray market. In the UK, the BBC has, for decades, employed a group of professional Doctor Who fans to clean up and restore the series’ classic episodes.

But there are just as many projects that exist without the studios’ blessing, completed by rogue fans looking to do their favorite media justice.

Of all the projects to revive an older property, the work done to Star Wars has been the most contentious. Before work began on the prequel trilogy, George Lucas sought to finish the original Star Wars films as he had intended. He spent $10 million on a lovingly detailed restoration of the films from the original negatives. And, when complete, Lucas chose to amend and edit the film in ways that angered fans.

He added CGI sequences that he’d abandoned in the original version for cost and time reasons. He filled empty backgrounds with digital performers and creatures, including some which dominated the frame. And he changed a few character moments, leading to much dismay from fans who saw the original scenes play out differently. Since then, there had been pressure on Lucas to release the “original” versions of the films from the cleaned-up print.

And in 2006, he did. Included as bonus extras on the 30th anniversary special edition DVD box set, fans could see the original trilogy. But, to dissuade fans from liking the originals too much, Lucas didn’t use the cleaned-up print made for his special editions. Instead, the DVDs were a grimy transfer from the 1993 LaserDisc release that had a fixed letterbox matte. Essentially, Lucas said that fans could have the originals, but only in a quality that would make a VHS blush.

One filmmaker has spoken out about post-release tinkering. In 1988, they wrote to a congressional committee decrying post-release editing. “People who alter or destroy works of art and our cultural heritage for profit are barbarians,” they wrote. They railed on “engineers with their computers” committing “defacements” to treasured movies as they “add or subtract material to the philosophical taste of the copyright holder.” The person who wrote that, of course, was a director and producer by the name of George Lucas.

A number of fans have since tried to recreate the original Star Wars by re-editing the available material. Most notable, Petr “Harmy” Harmacek, a visual artist by trade, who made The Despecialized Editions. He says that Lucas’ attempt to obscure the originals is “an act of cultural vandalism” that he set about attempting to fix. That required him to collate material from the DVD releases, as well as bootlegged film prints, which he then assembled. Sometimes, that required rotoscoping and animating sequences by hand to create a perfect version.

Separate to Harmacek’s work is 4K77, a project by a fan group known as the Silver Screen Crew to create a 4K version of the original release of Star Wars (with no episode number, or subtitle) from 1977. “Rob,” who wishes to remain anonymous, is a programmer from the US who has been working on the project for several years. At this point, 4K77 and 4K83 -- a 4K version of 1983’s Return of the Jedi -- have been released, with work ongoing on The Empire Strikes Back for 4K80.

Rob’s story begins several years earlier, when he was on a forum discussing how best to try to preserve Star Wars. “One day,” he said, “someone said ‘why don’t we just buy a print and scan it ourselves?’” Easier said than done, since you can’t sell prints on the open market, and they commonly return to the studios for archival or destruction after their theatrical release.

At the time, it seemed impossible, but one member found some loose reels from The Empire Strikes Back. Unfortunately, no reputable film scanning company would get involved since it would both be legally dubious and anger the studios. Worse is that the equipment needed to scan the film would cost far more than a ragtag bunch of Star Wars fans could afford.

Rob said that the team found someone who had “built a [film] scanner in his basement, which used parts from a VHS player to move film past a digital camera.” The machine wasn’t practical to scan a full film, it would have “taken years” but it worked as a proof of concept. Thankfully, fashion and changes in technology would make their job a lot easier.

“When [cinemas] went digital in 2014, they were just throwing [35mm projectors] away,” said Rob. And so the team bought “five or six” projectors and began working on ways to retrofit them to become an ersatz film scanner. The crew originally hooked up an HD camera to the projector’s output, via telescopic lens, to take images of every frame. They then swapped that out for a BlackMagic 4K camera connected to a film dubber from the 1980s.

After digitization, they stretched the raw footage to its correct aspect ratio then color-matched with an existing (clean) version of the film. Then, with a lot of scripting and some Photoshop hacks, they overlaid the film frame with frames from the Blu-ray release. “I would use the difference filter to see what the differences were, which is basically dirt and dust,” said Rob. The prints used for 4K77 are “obviously damaged and are a couple of generations away from the negatives.”

This by-hand grain removal, however, meant that it took six months to clean up just half of the first reel. So Rob turned to Phoenix, a video restoration package, that automated some of the process, albeit not to his tastes. He said that the software is far too zealous, removing key details from the screen -- including stars -- in its quest to eliminate dirt and grain. After the Phoenix pass, however, he manually checks every frame to restore lost detail.

Rob says that, spending an hour a day, he can correct up to 1,000 frames a day -- there are around 180,000 frames in the film altogether. “The pictures keep getting better and better,” said Rob, “and we’re almost as good as the official versions.” He’s under no illusion, though: Whatever the team achieves, the clean original prints in a vault at LucasFilm are far superior.

The team’s work on the other two films are feast and famine, feast for the 1983 movie, famine for the earlier. “We were really lucky with Jedi,” said Rob, “we managed to find a pristine print that had almost no dirt and damage on it.” The only things that the crew feels the need to tinker with are the colors, to try to get them as close to how it would have looked on release.

Regarding Empire, the team has contacted a private collector who says that they have a pristine film print. Those reels are currently in store and will, eventually, be scanned and handed over to the team for remastering and upscaling.

And, for a fan project that uses a four-decade film scan that was then algorithmically improved, 4K77 looks pretty brilliant. Thankfully, Fox, LucasFilm and now Disney have chosen not to go after the project, which is distributed for free. “They’ve been very tolerant of us and other people like us who have taken Star Wars and fan-edited it,” said Rob. He added that it’s not worth the time, given that the fan community obsessing over the original version is so small.

Because Star Wars was shot on film and screened theatrically, it’s at least possible to get prints of the film. It’s these artifacts that enable fans to at least attempt to restore the original to its former glory. Television shows aren’t often as accessible, with material usually limited to what is available in the home video market.

In 2018, I wrote about the sad story of Babylon 5, shot in anticipation of the forthcoming HDTV revolution. But neglect by Warner Bros. and self-defeating cost-cutting means that the only available version looks dreadful. High-quality originals are locked in a vault and although they wouldn’t be too expensive to fix, the material will languish forever more.

Fans aren’t deterred, however, and have been empowered by how cheap and accessible technology has become. Whereas, once, professional restoration equipment cost hundreds of thousands of dollars, home machines can now do it, and do it well. Hell, it’s possible these days to edit video on a smartphone, and most laptops go to the next level and can crunch video files.

Advances in AI have enabled better upscaling technology than previously available, too, especially for enthusiasts. ESRGAN, for instance, is a Generative Adversarial Network, code using two competing neural networks to improve resolution in images. And it has found itself popular in the video game community to increase the resolution of assets in older games.

But the most notable tool is Gigapixel AI by Topaz Labs, software that upscales images from their original sources to 4K resolution. Gigapixel was famously used to upscale a film from 1896 -- L’Arrivee d’un train en gare de La Ciotat -- to UHD. Because Gigapixel was originally for improving still images, the films have to be sliced into their individual frames. But sufficiently motivated video editors have done so -- and with staggering results.

Stefan, who uses the alias “Capt Robau,” started using Gigapixel, and then the company’s newer video tool, to upscale textures for a version of Half-Life entitled ReSrced. But he quickly started trying the tech out on the cult TV shows of his childhood, including Star Trek: Deep Space Nine, Voyager, Babylon 5, Buffy The Vampire Slayer and Earth: Final Conflict. 

“This is definitely a passion project of mine,” said Robau, which began when he experimented with upscaling textures for Final Fantasy VII. His interest quickly pivoted toward older TV shows, which are notorious for their (lack of) visual quality. “With HD displays, you really start noticing the difference between the standard-definition TV shows of the ‘90s and modern HD offerings,” he said.”

Robau said that Gigapixel was not as pin-sharp as his handmade upscaling attempts, but that it was far faster. “It’s not close to an actual remaster,” he said, but it shows “what can be done even without access” to the original masters and professional equipment. And for shows that have been put out without much care and attention, it is possible for folks to improve them at home.

And others have been working on their own variations on this theme, uploading short clips of their efforts to YouTube. It’s likely that these small teases are the most we’re ever going to see for a number of these projects. After all, if someone started uploading whole episodes, you can imagine that the rights holders would soon start complaining.

You may be wondering what drives people to spend so many off-duty hours resurrecting this material. The one recurring theme is the hook of childhood and the emotional bonds that are formed in our formative years. 4K77’s Rob said that he first saw Star Wars in cinemas as a toddler, but his lightbulb moment wouldn’t come until he saw the film on TV in 1982. He said that the TV broadcast prompted him to “go out and buy all the toys that my cousins had been telling me were so great for so long.”

Michael Moreau, who experimented with Gigapixel to fix Babylon 5 almost a year ago, feels there’s a big connection. “Babylon 5 came out around the time my dad and I were living on our own after my parents split up,” he said. The pair watched the show on a barely working TV to start with and kept watching, on progressively better TVs, through its run. “It was something that my dad and I bonded over at a difficult time,” he added.

And CaptRobau says he has fond memories of “watching all of Star Trek, Stargate, etc, on CRTs” in his early teens. But, alas, this material, often filmed with a relatively low budget and at speed, rarely looks as good on HDTVs. “When you return [as an adult],” he said, “you notice the low quality, jagged edges etc.” He added that his projects are “an attempt to get my older self to be able to see those things as flawlessly as I did when I was younger.”

It’s the problem with adulthood that, often, real life is never presented in as good a resolution as our memories. And perhaps all of these are just attempts to bring real life up to the same standard as our nostalgia.