Software draws paintings of musicians’ brains

NZTrio Somi Kim (piano), Amalia Hall (violin) and Ashley Brown (cello) will wear EEG headsets to transmit their brainwaves. PHOTO: SUPPLIED

Auckland artist Simon Ingram seeks to extend and reinvent the idea of what painting is, using technology. He tells Rebecca Fox about how he came to put EEG headsets on musicians.

Ever wondered what a musician might be thinking as they play beautiful music? Well, you will soon be able to see.

The NZTrio, a contemporary classical music group, will perform in Dunedin while wearing EEG headsets connected to software which drives a painting machine translating their mind’s electrical waves into painted lines on a canvas.

It is an extension of artist Simon Ingram’s series of works looking at how painting can be extended in a contemporary way using technology.

"I’m quite curious."

While Ingram has worn the headset while listening to music before, it will be the first time he has put the headsets on the actual musicians.

He admits it can be a slightly uncomfortable idea for some.

"The idea of the brain being plotted as a painting is slightly creepy."

Artist Simon Ingram readies his painting machine.

The idea for the concept came from earlier work Ingram had been doing over the years looking at ways painting could respond to the technology of today using radio astronomy, mathematics, mechanical and electronic means.

"Like a lot of people of my generation, I got interested in technology. In the ’90s there was the birth of the internet and ways in which ubiquity of computers and software re-shaped what I’d learnt in the art sphere.

"So by the early 2000s I was thinking a lot about a technological version of painting."

The idea of machines being used in painting has been an idea in art history for long time.

"It’s an art historical idea - a lot of people have done painting machines [in the] last 20 years."

He started with works using radio antenna that could capture cosmic waves from space. The early machines were autonomous.

"After that I really became interested in flipping it and looking inwards rather than outwards looking up at the sky, looking inwards to the brain."

Ingram, a senior lecturer at Elam School of Fine Art, also became interested in exploring environmental concerns after discovering that plotting radio waves of different frequencies could tell him about the Earth’s atmosphere and how energy from space came into the atmosphere.

In 2013 he discovered Dunedin firm Kamahi Electronics and approached Edwin Nieman with the idea of what he wanted to do. The company made a prototype machine which Ingram has gone on to extend and develop further. The machine moves a paint brush to dip it in a pot of paint and draw lines on a canvas.

"I’m always looking to work with technically minded people, employ them or pay for services. I’m not a scientist, I’ve always been an amateur scientist. I’m curious about exploring science as a medium."

To drive the machine Ingram needed special software so he called in John-Paul Pochin, who developed software which could drive the machine using the data received from an off-the-shelf EEG headset. It generated a line that wanders around the canvas avoiding the linear path it previously traced.

"The EEG headset is another kind of radio receiver which picks up energy in the brain while radio astronomy picks up electrical signals from space. This work is more intimate as it looks at what is inside the skull."

As the robot traces a path around a canvas, the beta and alpha waves given off by a person are interpreted to help determine the path the line traces, whether it goes left or right or how long the lines are.

In 2018 he presented Monadic Device, where Ingram wore the headset for five days having his own mental activity recorded as he read, wrote and chatted to people in Sydney Contemporary. As a result his machine painted lines that wandered across the canvas.

"The space was full of people. They came along, asked questions."

Simon Ingram tests out his brain-driven painting device which will be driven in Dunedin by the brainwaves of the NZTrio. PHOTO: JACQUIE MANNNG

He also presented it in Wellington, this time asking other people of varying backgrounds and interests to wear the EEG headset while they were doing whatever they wanted to do whether it was read, knit or play chess. They then got to take home the painting based on their brainwaves.

"It was an experimental thing."

Always keen on connecting with musicians he also took to the stage with Orchestra Wellington to "perform" a composition by Alex Taylor, conducted by Marc Taddei.

"The machine is there on stage with the pianos and cellos."

Then in 2020 he collaborated with Indie band Moniker at Wellington’s City Gallery.

"I listened to their music as they played and we made a video recording of that."

The performance with the NZTrio takes these events a step further, exceeding the original intention of the works but becoming "broader and richer" with the group’s playing. He hopes the work of art created in the process will be able to stay in Dunedin.

"It’s not something I’ve done before, recording electrical activity on the brain while a musician plays."

The works are also a nod to the concerns about digital technology invading people’s private lives.

"It is exploring ideas of those fearful robots will take over jobs, concerns Mark Zuckerberg is looking at your private details... it’s trying to push those buttons, to bring that idea to life, that aspect of our inner worlds being made public."

Working at Auckland University also enabled him to collaborate on art-science projects with engineering and computer science students.

He has also developed Terrestrial Assemblages, an ecological working group of people from different disciplines interested in the climate crisis.

"I’ve learnt a lot from making the machines and working with other people so this enables me to work in this way."