A picture and more definition about what you want to learn would be helpful. There are a lot of threads about modeling of threads and what you'll see is that generally it is recommended to simplify threads to face to face bonded contacts or use bolt connectors. When it comes to modeling of threads your approach is correct but I suspect will be challenging to solve. We did a few projects where we did 2d axisymmetric analyses to evaluate threads and that is what I would recommend as a starting point for you as well. 3d will be quite challenging.
I suspect the ability of the non-linear solver to figure out the contact condition is being screwed-up by the presence of beaucoup singularities. at the throat of the threads.