When Hugo the Troll was first introduced almost 20 years ago, no one really expected him to become a pioneer. But he did.

At the beginning, Hugo’s presenter used the Animation Mask System (AMS). This was basically an ice hockey helmet with sensors that monitored face movements. What was important about Hugo’s early capabilities was that the action was real time. He could talk to viewers at home.

In the course of time the AMS system was replaced by the Virtual Interactive Animation System (VIAS) which brings life to one or more virtual characters in a virtual studio. A game controller, a microphone and a dedicated graphical user interface (GUI) are used to control the characters, virtual cameras, lights, and other elements of the virtual scene. This set-up increases flexibility. For example it can be operated either by a single person or by an actor who controls the character inputs while a producer switches between the different cameras. The result is that the VIAS application renders high quality video and audio that is ready for broadcast.

Developing real time 3D characters

The VIAS System can also enable a virtual TV host or multiple virtual characters to interact with each other. The system can be used in a live situation as well as offline to create cost effective interstitials (gaps) or program bumpers.

“The latest challenge is making Hugo and other characters appear in three dimensions without losing the real time element,” says Martin Ciborowski, Team Leader of Video Based Entertainment for NDS Denmark. His team includes an animator, two 3D graphic artists, four programmers and Ciborowski who functions as the art director.

“We develop 3D interactive TV characters that are animated in real time. The 3D interactive technology delivers characters that look real and act real,” Ciborowski says. But that’s not all. “Broadcasters can now choose to have a 3D version of one of their brands as host of a TV show,” Ciborowski explains.

“Most of the early real time 3D applications were for children’s programming,” Ciborowski says, “but we’ve moved way beyond that. What we’re doing now is looking for ideas and products that support 3D interactivity on TV.” For example, his team is currently working on a poker application. “The idea is to have a lot of people playing poker,” he says. Real people. “Well, they’re actually 3D interactive TV characters who look like and respond like human beings in real time.”

Lifelike Internet poker

The 3D poker application looks like a game of poker on the Internet, he says. “The difference is that in this case each character is represented by a 3D avatar. We have hosts and poker experts in a virtual studio,” he says. “They can bring up the action at any table. It’s a streaming poker application.”

“What’s important to us is that we’re constantly looking for new technologies that enable us to interact with lifelike 3D characters in real time. These are the characters that we create.”

In other words, Ciborowski and his team are bringing 3D characters to life.

One of the possibilities Ciborowski’s team is currently working on is having a camera that can track the characters’ activities and motions. “This would enable us to translate the motions into the motions of a 3D character that looks real.”

3D Talk Show Repartee

Who would be interested in such a character? He explains that on a late night talk show, like Jay Leno, producers currently use two dimensional characters when they want to present a parody of someone in the news.

“What we’re working on is having an actual 3D character on a TV talk show host’s desk. The host could carry on a conversation with a character that looks real.”

In order to create such a character, Ciborowski’s team has to translate and adopt body language, facial expressions and even emotions for a variety of characters. “Development of these characters is a constant challenge,” he says.

“A possible next step is implementing a game pad that can be swung by the player,” Ciborowski says. This will allow people to play golf or baseball at home.

There’s more. “In order to enhance an actor’s actions, we’re investigating the possibility of using 3D cameras that show different parts of the body and actually recognize hand and head movements,” he says.

“Basically all of these potential applications require bringing the actor (or host) closer to the 3D character in real time,” he says. “This means that in the near future we could actually have 3D characters taking part in a game or a meeting.”

Endless Possibilities

The 3D engine that runs these applications can also be used for both PC and console games, Ciborowski adds. “This will allow us to develop an almost endless variety of games. At present we’re using the 3D engine to take control of 3D characters.”

As an example, good old Hugo is now in a game called “Cloud Cruiser.” “We have Hugo standing on a hover board. During the game he negotiates a track through the clouds, picking up coins as he goes.”

The way the game works is that players call into a live TV show and use buttons on their telephone to have Hugo jump between different tracks while avoiding obstacles.

“At NDS Denmark we are developing games and interactive applications for set-top boxes that are as interesting and exciting as commercial games developed for games consoles,” Ciborowski says.

“One of the challenges we face is taking 3D games and porting them to an STB so that the video content is actually stored on the subscriber’s DVR,“ he explains. This means viewers can choose if they want to play a game live (via their phone) or in their home, whenever they want, when it is streamed to their DVR box.

“We’re looking in many different directions,” he says. “The fact is, the possibilities are endless when it comes to developing 3D applications,” he says. “The real test is delivering real time animation to the subscribers.”

For more information:

NDS Interactive TV