Fisherman’s Song 渔父词

Interactive Installation,Ancient ChinesePoem

Developer Tools : TouchDesigner/IntelRealSense/MadMapper/AE

Fisherman’s Song is an Individual Project

“How can the poetic meaning of ancient Chinese poetry,
which transcends literal translation, be conveyed in interactive visual form?”

About Fisherman’s Song

“Fisherman’s Song” is a multidimensional art installation that creates a space for audiences to experience the beautiful story and deep meaning behind an ancient Chinese poem.

This interactive and immersive piece unfolds as users traverse the space.
The selected poem - “Fisherman’s Song” (渔父词) by Zhu Dunru(朱敦儒), describes the poet’s life experience after deciding to quit his life as a dignitary and live a secluded life next in the forest.

In the poem, he scorns the greed and desire of people he encountered as a dignitary and expresses his desire for the lifestyle he had always dreamed of, a life connected to nature and the elements.

The Poem

好事近渔父词 Fisherman’s Song

摇首出红尘，醒醉更无时节。活计绿蓑青笠，惯披霜冲雪。

晚来风定钓丝闲，上下是新月。千里水天一色，看孤鸿明灭。

Shaking my head, cast of the shackles of the red dust, And now at my own time will I get drunk, or sober if I wish

With my bamboo hat, my palm-coat, And more often than not I tread the hoarfrost and rush into the snow.

The wind calm at dusk, my fishing line resumes its serenity; In Both the skies and waters is seen a new moon bright.

Boundless waters and skies merging into an entity, A lonely wild pagoda is now in sight, now out of sight

ITP 2019 Thesis Presentation - Fisherman’s Song

Technology / Developing tools

/TouchDesigner

The main software I use to create the visual effects and interactivity of the project is TouchDesigner.
In order to create a fully immersive experience, I decided to use In-depth camera instead of Kinect, which is physically smaller and invisible in dark.

/Intel RealSense Camera

The in-depth tracking of the position of the audience will trigger different visual effects.

it is easy to hide and unnoticeable when install in the ceiling. but it also has clear and strong depth tracking even in the dark environment.

Use /Touch Designer for /Depth Tracking

There is no equipment required for the audience to interact with the visual, which is a fully immersive experience for everyone. There are four different projectors involved in the project.

I use Map Mapper control each of them projecting different visual effects simultaneously, in order to create the timeline of the story behind the poem.

Using the depth image from the camera, black and white image that represents the distance, I used blob tracking to know where the people are in the space, their position and movement will control the progression of the story, ,

and here is an example that the snow will slow down when people walking in the stage and the speed will constantly changing based on their location.

The location of the audience will decide the speed of the snow

Design inspiration

The ancient Chinese architecture style - Suzhou Garden, inspired me the most with its' unique design principles.

1. the garden originated from the ancient Chinese intellectuals' desire to harmonize with nature while cultivating their temperament.

2. the garden contains a series of separate but interconnected parts, to be discovered and enjoyed gradually. The staging design of "Fisherman's Song" use digital visual work tell the story of the poet's life in nature, which is a harmonious fusion between modern technology with nature and ancient culture.

3. Fisherman's Song follows the layout of the Suzhou Garden which allows audiences to have a personal experience and understanding of the poem.

Stone and Water- Solid and Fluid, Tough and Tenderness

Container of Nature - The gate of the garden is a painting frame of the scene behind it

Environment Design based on Suzhou Garden

The garden is a discovery space. everything is hidden. people needs to follow the path and uncover it’s beauty.

So I decided to create a explorable and hidden space, with the visual inspired by the Chinese ink painting style projected , in order to captured the spirit of the period

Material Study and Fabrication

I tested over ten different types of materials and I decided to use layers of transparency silks.

The transparency of silks invites the audience keep exploring the hidden space, with the visual inspired by the Chinese ink painting style, i try to captured the spirit of the period

In order to make sure the silk has no wrinkles while hanging, and can capture the projection well, I created the thin hanger to hang the silk, also to divide the space of my the environment. audience could follow the path I lay with silks, And explore the story behind the poem in their own way.

“上下是新月，看孤鸿明灭 In Both the skies and waters is seen a new moon bright.”

Chinese Silk and Bamboo hook to create the scene of the pagoda fly through the moon.

Bamboo Hook, Chinese Silk and Projection Mapping