Well, have you reconsidered kind of keeping your voice natural when recording? Sometimes if you exaggerate the tone, results can be oddly experimental.
You have to ensure when recording the voicebank, every one of the recordings isn't one pitch higher or lower.
Note you do NOT have to sing to record the voicebank. I used my neutral and deeper tone for the two VCV voicebanks which both are in range of my voice. Having to record a range that is not suitable for your voice can also result in experimental or failed recordings.
I know you may doubt your voice but let me be clear, don't force your voice to record something that it can't do. Vocal practices can help reach a range on what you need but that's another can of worms.
Elsewise, if the possible case of oto.ini and frequencies, you may want to recheck if they are fixed or there are some that sounded off from recording.
That's about it from my side. I can't give actual advice with such problems as it is subjective to what is the cause. It can be the UST (tuning, pitch, etc), the recordings of the voicebank, oto.ini, frequency or how you use it based on your description.