You are here

Third time lucky? Software to change the voice's timbre...

All about the tools and techniques involved in capturing sound, in the studio or on location.

Moderator: Moderators

Third time lucky? Software to change the voice's timbre...

Postby VOLOVIA » Thu Jun 10, 2021 7:54 am

I tried to start a discussion on this topic twice already and failed to raise a single comment, thus I try one more time and let it lie. This is surprising since I believe it represents a fascinating subject:

In an era of AI, machine learning, deep fakes, etc. but also very successful profiling of amps, cabs, effects, etc.

AND, vocal manipulation of the pitch including within a choir or embedded in a track (Melodyne etc.),.. (the holy grail of post-mixing editing)...

the actual changing of the vocal timbre has eluded all attempts. I have tried the various Waves plug-in, the built-in in Cubase and LogicPro X, and they all fail miserably. The formant change option in Logic for instance does nothing to change the timbre significantly and only at extreme settings you can hear it, but it becomes highly artificial and robotic. Waves' with its 'throat' parameters etc. promised much but delivers nothing but aliens.

So, where is the technology to modify the vocal harmonics convincingly to change the timbre (not just transpose or tune, which is done rather successfully already)? What are the technical obstacles stopping a well-tuned, sung vocal take be modified to sound closer to a soul singer let's say, or different gender? How far are we from being able to morph my vocal take (providing it is of professional standard) into something like Barry White, say?

I'd rather not discuss the 'moral' implications of such technology, or even waste time in sly comments such as "learn how to sing properly", and keep the discussion on the technological challenges. Maybe the technology is already out there, mature, but I am not aware of it. If so, links, please.

What would I use it for? Not much to change my own voice, which represents me for what it's worth, but I would love to have Stevie Wonder and Madonna "doing" my BVs...
User avatar
VOLOVIA
Regular
Posts: 405
Joined: Thu Oct 01, 2009 12:00 am
Location: London
VOLOVIA - FACEBOOK - TWITTER Songwriter/guitarist

Re: Third time lucky? Software to change the voice's timbre...

Postby Wonks » Thu Jun 10, 2021 8:19 am

I think you are misusing the word ‘timbre’ here, as all the formant adjusting software I’ve used certainly noticeably affects the timbre of the voice.

What it doesn't do is affect the overall character of the voice and make you sound like someone significantly different.

What you are talking about is software that will morph your voice into sounding just like someone else’s voice. I doubt that software exists at the moment.

I’d guess that at best you’d either have to start with a voice very similar to the original you want to mimic, or you’d need a comprehensive recording of that voice saying every word in the dictionary, and each word in several different ways. You’d still need very complicated software that works out how to select the right word, the closest version of the target voice pronunciation and the ability to morph that sound to the way and speed you say it.
User avatar
Wonks
Jedi Poster
Posts: 11611
Joined: Thu May 29, 2003 12:00 am
Location: Reading, UK
Correcting mistakes on the internet since 1853

Re: Third time lucky? Software to change the voice's timbre...

Postby Arpangel » Thu Jun 10, 2021 8:23 am

The human voice is so complex, to modify it, you’d have to do it in such a convincing way, because the brain is tuned to pick up the human voice more than anything, and it will recognise anything that doesn’t sound natural.
I can see a picture of someone from years ago, and not recognise them, but if I hear a voice, it’s instant recognition.
Pitch correction and transposition is still not very good, and regards tonal changes, I don’t know of anything at all, if you want a natural sounding, convincing change.
User avatar
Arpangel
Jedi Poster
Posts: 7751
Joined: Sat Jul 12, 2003 12:00 am
That would be an ecumenical matter.

Re: Third time lucky? Software to change the voice's timbre...

Postby Mixedup » Thu Jun 10, 2021 9:21 am

I suspect that the answer to what you want lies in a sort of AI sample instrument, since you'd need to be able to tell the system what sort of character you want at any given point — much like you tell a sample instrument what sort of playing style you want (can't just say 'classical', as you need to be much more specific...). The tech exists – as in the demo fakes from Adobe and other years ago, where they took Obama and others and created new speeches; there's no reason in principle that such a system couldn't learn two voices and use one to 'drive' the other. But singing styles and voices involves way more critical parameters than speech. It would take a LOT of learning of both the singer and the 'donor' voice. The reason they used Obama etc is that there are hours and hours of public speaking to use for the learning. Even with a prolific artist there's much less, and that's even before you start to consider the particular style/delivery. So I reckon you'd have to get session singers in for the learning sessions... Yamaha's Vocaloid is about half way there, I suppose. But it will not give you what you seek. I' sure there are research projects out there, and I'm sure it's coming... just not yet. Meanwhile, there are a gazillion great online session vocalists/musicians you could turn to if you're unhappy with your own voice, not to mention singers looking for musical collaborators...
User avatar
Mixedup
Jedi Poster
Posts: 4443
Joined: Wed Sep 03, 2003 12:00 am
Location: Cambridgeshire, UK

Re: Third time lucky? Software to change the voice's timbre...

Postby Mixedup » Thu Jun 10, 2021 9:26 am

To add one point... I think this is far more complex than you imagine. Consider mic modelling. There are a handful of companies doing this at all successfully, and none are perfect in all applications. Those that are best employ a specific mic — a known quantity — and impose characteristics on the result. Ie a one:many relationship. To do something like that successfully for any singer, it would be a many:many relationship, with all the additional complexity of there being many more variables for each 'one'.
User avatar
Mixedup
Jedi Poster
Posts: 4443
Joined: Wed Sep 03, 2003 12:00 am
Location: Cambridgeshire, UK

Re: Third time lucky? Software to change the voice's timbre...

Postby blinddrew » Thu Jun 10, 2021 11:46 am

In contrast to the previous posters, I reckon we're probably not that far off.
Text to voice generators like Adobe's can work off about two minutes recorded speech to generate new content, but obviously they're not having to work with mumbled words, interesting accent changes and the tonal and dynamic ranges of typical singing.
I think this will be driven by the movie and games industries (foreign language over-dubbing being the obvious use), but music won't be far behind.
User avatar
blinddrew
Jedi Poster
Posts: 14837
Joined: Sun Jul 05, 2015 12:00 am
Location: York
Ignore the post count, I have no idea what I'm doing...

Re: Third time lucky? Software to change the voice's timbre...

Postby James Perrett » Thu Jun 10, 2021 3:39 pm

I already find it difficult to work out whether some of the voices I hear in our son's programming are real or fake. The speech synthesiser in Scratch is extremely good and I don't think it will be long before it can sing as well.
User avatar
James Perrett
Moderator
Posts: 10773
Joined: Mon Sep 10, 2001 12:00 am
Location: The wilds of Hampshire
JRP Music - Audio Mastering and Restoration. JRP Music Facebook Page

Re: Third time lucky? Software to change the voice's timbre...

Postby VOLOVIA » Thu Jun 10, 2021 5:27 pm

James Perrett wrote:I already find it difficult to work out whether some of the voices I hear in our son's programming are real or fake. The speech synthesiser in Scratch is extremely good and I don't think it will be long before it can sing as well.

Hi guys, thanks for your input. As I mentioned before, I don't need to replace my voice. I think some people might even find it palatable! But yes, I have been bombarded on Facebook by a company selling 'voice-over software' and I gave it a try. Started with English and it sounded rather good. However, there is no mother like the mother tongue, thus I tried it in Italian, and still sounded very, very natural (TXT to Speech).

The next step will be modulating the voice according to notes... But this is NOT what I was looking for, a "machine that sings for me". I was talking about a 'filter box' that can alter simple parameters like the volume of the throat canal or the pipe length. Again, I remember a plug-in from Waves having the 'right' parameters but sounding useless, not better than the bog-standard Logic's "formant change" option. In my technical innocence, again, I would have thought that a sung phrase nowadays could be easily manipulated to sound like a different voice. I am obviously wrong. Thanks!
User avatar
VOLOVIA
Regular
Posts: 405
Joined: Thu Oct 01, 2009 12:00 am
Location: London
VOLOVIA - FACEBOOK - TWITTER Songwriter/guitarist

Re: Third time lucky? Software to change the voice's timbre...

Postby blinddrew » Thu Jun 10, 2021 5:33 pm

I wonder if it's another candidate for the paradox of robotics, where things that are simple and menial for humans (cleaning a toilet, mimicking someone else's voice) are actually really difficult for robots?
User avatar
blinddrew
Jedi Poster
Posts: 14837
Joined: Sun Jul 05, 2015 12:00 am
Location: York
Ignore the post count, I have no idea what I'm doing...

Re: Third time lucky? Software to change the voice's timbre...

Postby Wonks » Thu Jun 10, 2021 5:50 pm

There's this Antares/Autotune product which is along the lines of what you were looking for, but I've no idea what it's like. But you can try it for free. https://www.antarestech.com/product/throat/
User avatar
Wonks
Jedi Poster
Posts: 11611
Joined: Thu May 29, 2003 12:00 am
Location: Reading, UK
Correcting mistakes on the internet since 1853

Re: Third time lucky? Software to change the voice's timbre...

Postby Drongoloid » Thu Jun 10, 2021 6:38 pm

VOLOVIA wrote:
I'd rather not discuss the 'moral' implications of such technology

I'd say the moral implications are the only important ones in this discussion. How nuclear fission works is of interest to but a handfull but it's implications are of interest to all of us.
Drongoloid
Regular
Posts: 53
Joined: Fri Jul 08, 2005 12:00 am
Location: bedsit in baldock

Re: Third time lucky? Software to change the voice's timbre...

Postby blinddrew » Thu Jun 10, 2021 11:16 pm

Drongoloid wrote:
VOLOVIA wrote:
I'd rather not discuss the 'moral' implications of such technology

I'd say the moral implications are the only important ones in this discussion. How nuclear fission works is of interest to but a handfull but it's implications are of interest to all of us.
Well that would be a potentially interesting discussion, and one that we could start a thread on, but it feels a trifle rude to go directly against the wishes of the OP. ;)
User avatar
blinddrew
Jedi Poster
Posts: 14837
Joined: Sun Jul 05, 2015 12:00 am
Location: York
Ignore the post count, I have no idea what I'm doing...

Re: Third time lucky? Software to change the voice's timbre...

Postby VOLOVIA » Fri Jun 11, 2021 4:57 am

blinddrew wrote:
Drongoloid wrote:
VOLOVIA wrote:
I'd rather not discuss the 'moral' implications of such technology

I'd say the moral implications are the only important ones in this discussion. How nuclear fission works is of interest to but a handfull but it's implications are of interest to all of us.
Well that would be a potentially interesting discussion, and one that we could start a thread on, but it feels a trifle rude to go directly against the wishes of the OP. ;)

Feel free! morals/morality/philosophy is my 4th hobby of choice... glad to read opinions on the matter. Sure, the technology as we are allows any sung performance to be pitched near to ideal as one wishes, the so-called "anyone can sing in tune nowadays". Sure.
Singing is not just "notes-in-tune", but also requires delivery, call it passion/meaning if you liked. At this point the next step would be to turn the great guitar solo I played "perfectly" on a cheap studio guitar into a PSR-59-strat-Mesa- blah blah tone monster. This is kind of possible already, as you all know.
Now, do the same with a voice, in tune, full of expression, but sung with a 'thin voice'. Hey, presto, pre-set 24 Stevie Wonder Young Boy II turns the character of the voice, the harmonics, into a great sound. Like turning a studio flute tone into a solid-silver marvel.
Would this be ethical? Artistic? Worthwhile? Soulless?
User avatar
VOLOVIA
Regular
Posts: 405
Joined: Thu Oct 01, 2009 12:00 am
Location: London
VOLOVIA - FACEBOOK - TWITTER Songwriter/guitarist

Re: Third time lucky? Software to change the voice's timbre...

Postby Forum Admin » Fri Jun 11, 2021 8:38 am

I'm sure iZotope will now have read your post and be working on their forthcoming VoiceMimic app! :) If Jon Culshaw and other great vocal impressionists can mimic, I'm sure AI will soon be able to.

Maybe iZotope DeadRingers is a better name? :beamup:

I recall SOS reviewing the TC-Helicon VoicePrism hardware processor in 2001, which claimed to be capable of changing your sonic personality. SOS review here:
www.soundonsound.com/reviews/tc-helicon-voiceprism-plus

As an aside, has anyone tried Murf AI for voice-overs?
User avatar
Forum Admin
Moderator
Posts: 3570
Joined: Mon Jul 21, 2003 11:00 pm
Location: A studio deep in the fenlands of Cambridgeshire, UK
SEARCH 12,613 SOS Reviews/Techniques articles: www.soundonsound.com/search

Re: Third time lucky? Software to change the voice's timbre...

Postby James Perrett » Fri Jun 11, 2021 10:37 pm

Forum Admin wrote:As an aside, has anyone tried Murf AI for voice-overs?

I had a quick listen to their demos and I'm not convinced that they're much better than the ones you could create for free in Scratch although they seem to offer a wider range of voices.

https://en.scratch-wiki.info/wiki/Text_ ... _Extension

Looks like the Scratch extension is powered by Amazon Polly

https://aws.amazon.com/polly/

which has both free and paid for tiers and a wider variety of voices.
User avatar
James Perrett
Moderator
Posts: 10773
Joined: Mon Sep 10, 2001 12:00 am
Location: The wilds of Hampshire
JRP Music - Audio Mastering and Restoration. JRP Music Facebook Page

Next