I tried to revive my lost UTAU I made when I was 12 using AI

[Reiz]

Backstory:
In 2015, when I was 12, I made my first UTAU set for release. His name was "Itsudan Tokime" and was a CV voicebank. I finished his voicebank but I couldn't release him because my friend was still working on his art. (tbh if ai art was a thing back then i wouldn't have had that problem) He was surprisingly a "high-quality" voicebank despite having recorded him on my iPhone 4s.

Unfortunately, my laptop gave out and I inevitably lost all of his files since I was dumb enough not to upload a backup. Therefore, Tokime ceased to exist. And because I was only 12, my voice went through puberty and I would no longer be able to remake him.

All that exists now of his voicebank on the internet are 2 covers:

Q
Anti Beat

Fast forward to 2024, I became interested in making AI voice models and covers and had the idea of producing an AI of my old UTAU using the 2 recordings I made back then. Using weights.gg, I fed it isolated recordings and spent a little more than a dollar to have it trained.

The results are far from perfect, but I'm actually impressed with how it turned out. He doesn't have as much power and clarity as how my UTAU was. But his tone and pronounciation is close enough to sound like it.



While I probably won't make any covers using this AI voice alone, I might make it sing CV/VCV samples and use those to remake an utau from scratch.

 

[Kiyoteru]

Projects like these are exactly what AI vocal processing tools are useful for. Good luck with making new voicebanks and have fun!

 

[Reiz]

Update:

I remade Tokime's voicebank!

He doesn't sound.... toooo bad, given the imperfections of AI processing. Basically, I made one long recording of Tsukishiro Hakupo's CV samples then fed into the voice model. I cut every sample individually and otoed them, creating a voicebank. With the way Hakupo's VP pronounces certain syllables he's bound to sound a lot different (I do plan on experimenting either with other UTAUs or if I have the time and energy, my own voice lol), and since I pretty spedrun the otoing, that aspect isn't perfect either. He still lacks clarity due to the AI replication of it compared to the actual voicebank in 2015. Nevertheless, I'm surprised with how it turned out.

Here's a comparison video:

 

[Kiyoteru]

If you need another source for base samples, Namine Ritsu has an extremely permissive terms of use for the UTAU voicebanks. You'd be free to make whatever you like with them, and even release the final voicebank publicly.