So it happens that while training the SSRN, I realized something went wrong because the image in the TensorBoard was empty and the train and valid loss increased. I totally didn’t know what went wrong but I’ve decided to create a new virtualenv and download all the dependencies again and update them. It seems to work now. And This is exactly what happened when I started training SSRN for the first time, and nothing was trained. I didn’t remember why it worked afterward, but now I know.
And since it’s not really a good idea to have over 10k files in a directory, I’ve decided to pack them into a single file for increase I/O performance. You can imagine that it’s not really nice to constantly reading small files and even though I’m using SSD the 4K read is bottle-necking constantly. Plus, the biggest concern is to not upload over 20k files to my google drive every time I’ve changed something in the audio source. And google drive kind of hates it and decided to ban me from using google drive if I did some intense I/O operation on a single folder for a day. Now it should make everything easier.
Oh well, I am still obsessed with tweaking the voice model. But time to start really finishing the whole web interface/app while training the model.