MelodiaSynth - Web UTAU Frontend in development

MelodiaSynth - Web Vocal Synth in development



What is MelodiaSynth?
MelodiaSynth is a WIP cloud-based alternative to the UTAU front-end program.

Why does this exist?
UTAU's front-end was developed with frameworks that will soon no longer be supported by Microsoft. The lack of updates from UTAU's original developer is worrysome, as there is no guarantee that the software will continue to work in future versions of Windows. Melodia Sequencer is an attempt to replicate, improve and extend UTAU's functionality using new technologies, while eliminating problems that plagued the original software, particularly locale.

What are the advantages of MelodiaSynth?
Melodia runs on a browser, so it is completely cross-platform. Since the rendering is done in the cloud, the user does not need to change their computer's locale in order to create, edit or render projects. Furthermore, the rendering is done in parallel, taking less time to finish than a default installation of UTAU.

What are the disadvantages of MelodiaSynth?
The rendered output needs to be downloaded from the server, so slower internet connections might result in slower render times. However, due to MelodiaSynth having a lower render time than UTAU, this is compensated to some extent. Also, as the software is still in development, not all features have been implemented. Needless to say that it does require an internet connection.

What browsers are supported?
The project is being tested against the latest versions of Chrome, Firefox and Opera. Although untested, it should work on Safari.
Microsoft Edge, Internet Explorer and all mobile browsers are not supported and may or may not work with limited functionality.

Where can I get it?
Melodia can be accessed for testing at https://melodiasynth.com/

What voicebanks are available?
Once you create an account, you must upload a CV voicebank at https://melodiasynth.com/creator.
Click on the + button on the top left corner, type the voicebank name and drag all .wav/.ini files to the upload area. Wait for all files to be uploaded and return to the sequencer.

Is the source code available?
The front-end sequencer code is available at [removed]

Media

Spoiler: Videos

Spoiler: Old Post

The goal of this project is to create a web-based UTAU front-end that can be used in any browser, regardless of locale settings or operating system.
No more downloading, installing, changing locale settings, opening virtual machines or fighting against resamplers under wine! Everything will be easily accessible through a web browser and the rendering will be done asynchronously in the cloud. It aims to maintain full compatibility with current resamplers and voicebanks, as well as UST files.

Eventually it will allow users to upload voicebanks to their accounts and easily share them with other users.

As of now, the project is still in its infancy, being tested as a proof-of-concept.
You can add, edit, remove and move individual notes or regions vertically. Rests are invisible and handled internally to emulate a rest-less system. Zooming in and out has been implemented, and support for time signatures other than 3/4 and 4/4 is possible. Switching between tracks seems to be working.

I'm temporarily hosting it at http://redelee.ddns.net:25565/ for back-end testing. It is using Defoko's voicebank (with kana and romaji alias). Feel free to check it out if you're curious, although I cannot guarantee that it will be up at the time you're reading this. Feedback is welcome!

 

This is so cool! I can't give much in the way of feedback rn, but good luck!

 

Another person doing God's work, which is really just any development work at all in the utau community

 

One step closer. One step closeeeer!

 

I hope thie editor is compatible with my screen reader... who knows

 

I noticed if you zoomed in too much you'll be unable to scroll sideways to the notes that are laiter in the track. It seems the bar in the bottom can't be smaller than about 1/2-ish.

Managed somehow to create a ghost note on the secound track, unfortunatly I don't know how I did it.

Spoiler: Pic

The note should be somewhere between the two rests in the picture. I have checked all the way from C1 to B7, the note cant be seen for some reason. The note size is 120.

Got defoko to sing. But she won't sing notes that are smaller than 172 in the track.

I'm exited to see how this will grow/develop.

 

Just a small note to any mac users, it won't work on Safari! Use Google Chrome instead!

I messed around in it and I really like the feel, though I have to say the pop up being the only way to add lyrics is a bit tedious. I know you probably have thought about that already and will work on it in the future, I just wanted to voice what I felt. Defoko sang fine for me though!

 

Gosh, if it's fully finished, I hope it's compatible with the latest Windows OS. I might try it later. =)

 

It's so amazing <3 Finally something close to what I've been wishing for (and been unable to do by myself since it always seemed to me to be even tricky to do for veteran and hardcore coding users )

Also noticing it doesn't likely work well with Firefox at the moment =O though it may be on my end because I'm facing lags again with incoming thunderstorms ... Anyway good luck on the project and can't wait for further advancements or updates ! 4(>u<)9 (finally ! >=D)


Edit : Ok so now it's working fine and I really like it ~ <3 If I can find a way to save and export the wav sample i'll upload it ^^ With my usual ritual whenever it's been a while since I4ve used a bank nor I'm using a brand new one haha ~ ♥

 

It's really nice so far, but the "Editing Note" dialogue feels like its a tiny bit bigger than it should be. It spills off the page and I have to scroll down to hit the ok button.

 

Oh, no wonder why it didn't work for me (lol) Need to try open it with Chrome in the next time :"D dang !

 

Yeah it works fine on Chrome

 

I will try tomorrow, it's all ready 11 PM here, so... need to sleep :"v

--- Double Post Merged, May 23, 2017 ---

@Khento IDK if it's technically possible but it would be cool if people who has vision limitations could use that site too.

Alessandra is blind. To use a browser etc they need to a screen reader (and because UTAU has no complability to that, they cannot use UTAU themselves)

 

Unfortunately, it currently isn't compatible, since the UI is essentially an image that is redrawn every single frame. I'm not sure how screen readers could interact with that. I'm looking into a solution though. Because I'm unfamiliar with screen readers, if you have any suggestions as to how you'd expect them to work with the website please don't hesitate to post them.

Ah, the scrollbar. I haven't implemented dinamic resizing of the grid container yet, so the scrollbar has a fixed length.
As for the ghost note, maybe it was placed out of range, but it's difficult to know what happened without a log... I'll definitely add a report bug button ASAP.
The reason she doesn't sing small notes is because I haven't finished the backend code to properly implement tempo, note length, overlap, preutterance, etc. I'll try to fix that soon.

I'm testing against Chrome 58, Firefox 53, IE11 and Microsoft Edge. Chrome and Firefox seem to be working fine, while Microsoft Edge doesn't highlight the selected tool, but still renders. And of course, there's always that guy that doesn't work with anything. I'm probably not going to support IE, since its a pain to do anything with it. At any rate, I should include a loading message.
Also, I'm not a veteran nor a hardcore coding user (I'll leave that title for resampler developers ). I just think that the best way to start a project is to just do it. Planning is important, sure, but if you keep planning and planning you'll end up with a project stuffed with a thousand ideas that no one has the courage to implement. I think that starting small and moving forward step-by-step makes the project look much more approachable. Also, this project is on GitHub. Pull requests are welcome (although at this point the code must be too messy for anyone to read it).

I might have to slow down for a week since exams are coming. Hopefully I'll have more time to work on it after that.

 

It's ok, take your time ;w;/ and yeah, hence why I've edited my message as soon as it managed to work on it

Anyway, the debut step is promising ~ ♥

 

It's really cool to share your progress with us! You might have this already, but I'd like to suggest posting a bug list and/or a change-log so we can stay updated and organized. Apologies if this is already up.

 

Finally, it can read otos!

Check out this demo song, the tracks no longer get skewed over time!
Just go to http://redelee.ddns.net:25565/ , open the sidebar, click Import and copy and paste the text below. Press play and wait for around a minute (upload speeds are bad, sorry). The sidebar will open to indicate that it is rendering

Spoiler: Demo Song

{"name":"Tsugai Kogarashi","tracks":[{"trackName":"Meiko","voiceId":0,"resamplerId":0,"wavtoolId":0,"notes":[{"length":1440,"lyric":"R","noteNum":60},{"length":240,"lyric":"ka","noteNum":58},{"length":240,"lyric":"wa","noteNum":61},{"length":720,"lyric":"i","noteNum":63},{"length":240,"lyric":"ta","noteNum":63},{"length":240,"lyric":"ko","noteNum":61},{"length":240,"lyric":"ga","noteNum":58},{"length":240,"lyric":"ra","noteNum":61},{"length":240,"lyric":"shi","noteNum":58},{"length":240,"lyric":"so","noteNum":56},{"length":240,"lyric":"yo","noteNum":58},{"length":240,"lyric":"so","noteNum":56},{"length":240,"lyric":"yo","noteNum":58},{"length":480,"lyric":"to","noteNum":51},{"length":2400,"lyric":"R","noteNum":60},{"length":240,"lyric":"hi","noteNum":58},{"length":240,"lyric":"ra","noteNum":58},{"length":240,"lyric":"hi","noteNum":58},{"length":240,"lyric":"ra","noteNum":58},{"length":480,"lyric":"to","noteNum":58},{"length":240,"lyric":"a","noteNum":58},{"length":240,"lyric":"i","noteNum":61},{"length":720,"lyric":"ma","noteNum":63},{"length":240,"lyric":"mi","noteNum":58},{"length":240,"lyric":"e","noteNum":61},{"length":240,"lyric":"ru","noteNum":58},{"length":240,"lyric":"hi","noteNum":56},{"length":240,"lyric":"o","noteNum":58},{"length":240,"lyric":"ma","noteNum":56},{"length":240,"lyric":"chi","noteNum":54},{"length":240,"lyric":"na","noteNum":56},{"length":240,"lyric":"ga","noteNum":58},{"length":480,"lyric":"ra","noteNum":51},{"length":240,"lyric":"to","noteNum":58},{"length":240,"lyric":"ki","noteNum":63},{"length":720,"lyric":"o","noteNum":63},{"length":240,"lyric":"ka","noteNum":58},{"length":240,"lyric":"zo","noteNum":61},{"length":240,"lyric":"e","noteNum":58},{"length":240,"lyric":"a","noteNum":56},{"length":240,"lyric":"ru","noteNum":58},{"length":720,"lyric":"ku","noteNum":63},{"length":240,"lyric":"R","noteNum":60},{"length":480,"lyric":"tsu","noteNum":63},{"length":480,"lyric":"zu","noteNum":68},{"length":480,"lyric":"ru","noteNum":70},{"length":240,"lyric":"ko","noteNum":68},{"length":240,"lyric":"to","noteNum":66},{"length":240,"lyric":"no","noteNum":68},{"length":240,"lyric":"ha","noteNum":66},{"length":240,"lyric":"ni","noteNum":73},{"length":240,"lyric":"i","noteNum":68},{"length":480,"lyric":"ro","noteNum":70},{"length":240,"lyric":"do","noteNum":70},{"length":240,"lyric":"ra","noteNum":73},{"length":480,"lyric":"re","noteNum":70},{"length":240,"lyric":"a","noteNum":73},{"length":240,"lyric":"ka","noteNum":77},{"length":240,"lyric":"ku","noteNum":78},{"length":240,"lyric":"i","noteNum":77},{"length":240,"lyric":"ro","noteNum":75},{"length":240,"lyric":"me","noteNum":73},{"length":240,"lyric":"ki","noteNum":70},{"length":240,"lyric":"se","noteNum":68},{"length":240,"lyric":"tsu","noteNum":73},{"length":240,"lyric":"na","noteNum":68},{"length":480,"lyric":"ni","noteNum":70},{"length":240,"lyric":"o","noteNum":68},{"length":240,"lyric":"do","noteNum":63},{"length":240,"lyric":"ru","noteNum":70},{"length":240,"lyric":"R","noteNum":60,"tempo":null},{"length":240,"lyric":"ku","noteNum":58,"tempo":null},{"length":240,"lyric":"re","noteNum":65,"tempo":null},{"length":480,"lyric":"ha","noteNum":66,"tempo":null},{"length":240,"lyric":"i","noteNum":65,"tempo":null},{"length":240,"lyric":"chi","noteNum":63,"tempo":null},{"length":480,"lyric":"ma","noteNum":65,"tempo":null},{"length":240,"lyric":"i","noteNum":66,"tempo":null},{"length":240,"lyric":"te","noteNum":68,"tempo":null},{"length":240,"lyric":"no","noteNum":70,"tempo":null},{"length":240,"lyric":"o","noteNum":68,"tempo":null},{"length":240,"lyric":"hi","noteNum":63,"tempo":null},{"length":240,"lyric":"ra","noteNum":61,"tempo":null},{"length":480,"lyric":"ni","noteNum":58,"tempo":null},{"length":240,"lyric":"su","noteNum":56,"tempo":null},{"length":240,"lyric":"be","noteNum":58,"tempo":null},{"length":960,"lyric":"ri","noteNum":63,"tempo":null},{"length":240,"lyric":"R","noteNum":60,"tempo":null},{"length":240,"lyric":"ka","noteNum":63,"tempo":null},{"length":240,"lyric":"ta","noteNum":68,"tempo":null},{"length":240,"lyric":"ru","noteNum":63,"tempo":null},{"length":1200,"lyric":"wa","noteNum":70,"tempo":null}],"selectionRegionStart":92,"selectionRegionEnd":92},{"trackName":"Kaito","voiceId":1,"resamplerId":0,"wavtoolId":0,"notes":[{"length":3840,"lyric":"R","noteNum":60},{"length":240,"lyric":"so","noteNum":51},{"length":240,"lyric":"yo","noteNum":51},{"length":240,"lyric":"so","noteNum":51},{"length":240,"lyric":"yo","noteNum":51},{"length":480,"lyric":"to","noteNum":46},{"length":240,"lyric":"ka","noteNum":58},{"length":240,"lyric":"wa","noteNum":61},{"length":720,"lyric":"i","noteNum":63},{"length":240,"lyric":"ta","noteNum":63},{"length":240,"lyric":"ko","noteNum":61},{"length":240,"lyric":"no","noteNum":58},{"length":240,"lyric":"ha","noteNum":61},{"length":240,"lyric":"wa","noteNum":63},{"length":240,"lyric":"hi","noteNum":65},{"length":240,"lyric":"ra","noteNum":66},{"length":240,"lyric":"hi","noteNum":65},{"length":240,"lyric":"ra","noteNum":66},{"length":480,"lyric":"to","noteNum":63},{"length":2400,"lyric":"R","noteNum":60},{"length":240,"lyric":"ma","noteNum":51},{"length":240,"lyric":"chi","noteNum":51},{"length":240,"lyric":"na","noteNum":51},{"length":240,"lyric":"ga","noteNum":51},{"length":480,"lyric":"ra","noteNum":46},{"length":240,"lyric":"to","noteNum":51},{"length":240,"lyric":"ki","noteNum":58},{"length":720,"lyric":"o","noteNum":59},{"length":240,"lyric":"ka","noteNum":51},{"length":240,"lyric":"zo","noteNum":56},{"length":240,"lyric":"e","noteNum":53},{"length":240,"lyric":"a","noteNum":53},{"length":240,"lyric":"ru","noteNum":53},{"length":720,"lyric":"ku","noteNum":58},{"length":4560,"lyric":"R","noteNum":60},{"length":240,"lyric":"a","noteNum":68},{"length":240,"lyric":"ka","noteNum":68},{"length":240,"lyric":"ku","noteNum":71},{"length":240,"lyric":"i","noteNum":71},{"length":240,"lyric":"ro","noteNum":71},{"length":240,"lyric":"me","noteNum":66},{"length":240,"lyric":"ki","noteNum":61},{"length":240,"lyric":"se","noteNum":61},{"length":240,"lyric":"tsu","noteNum":68},{"length":240,"lyric":"na","noteNum":61},{"length":480,"lyric":"ni","noteNum":63},{"length":240,"lyric":"o","noteNum":63},{"length":240,"lyric":"do","noteNum":58},{"length":240,"lyric":"ru","noteNum":61},{"length":240,"lyric":"R","noteNum":60,"tempo":null},{"length":240,"lyric":"ku","noteNum":51,"tempo":null},{"length":240,"lyric":"re","noteNum":58,"tempo":null},{"length":480,"lyric":"ha","noteNum":59,"tempo":null},{"length":240,"lyric":"i","noteNum":59,"tempo":null},{"length":240,"lyric":"chi","noteNum":59,"tempo":null},{"length":480,"lyric":"ma","noteNum":61,"tempo":null},{"length":240,"lyric":"i","noteNum":61,"tempo":null},{"length":240,"lyric":"te","noteNum":65,"tempo":null},{"length":240,"lyric":"no","noteNum":63,"tempo":null},{"length":240,"lyric":"o","noteNum":63,"tempo":null},{"length":240,"lyric":"hi","noteNum":58,"tempo":null},{"length":240,"lyric":"ra","noteNum":58,"tempo":null},{"length":480,"lyric":"ni","noteNum":53,"tempo":null},{"length":240,"lyric":"su","noteNum":53,"tempo":null},{"length":240,"lyric":"be","noteNum":49,"tempo":null},{"length":960,"lyric":"ri","noteNum":59,"tempo":null},{"length":240,"lyric":"R","noteNum":60,"tempo":null},{"length":240,"lyric":"ka","noteNum":59,"tempo":null},{"length":240,"lyric":"ta","noteNum":59,"tempo":null},{"length":240,"lyric":"ru","noteNum":59,"tempo":null},{"length":1200,"lyric":"wa","noteNum":65,"tempo":null}],"selectionRegionStart":33,"selectionRegionEnd":33}],"tempo":100}

EDIT September 3rd, 2017: It is no longer possible to import tracks through this method

 

Sounds awesome!

 

I have an idea, why don't you hold a little contest for a CV bank that will be included as one of the default banks? I'd love to enter that, does this ust otos just like utau?

 

Please let me know when you implement voicebank name and icon, so I can provide you with an image! As for the name, you can put either "MPT" or "Lalita Laakkonen". I believe it's currently Voicebank ID #1?