The primary unknown in a thermal analysis is temperature, which is a scalar quantity, so I think your solution is converging.
The heat flux is like stress in a static analysis. As you say, it has to be a singularity and the results there are not representative, what it does not mean that the rest of the solution is no valid.
How much do you need to remesh the model it is up to you. Computer resources avalaible, time and the problem to be studied.are the things you have to keep in mind. In your case, the difference is around 1% what is pretty neat
You could keep remeshing the model, but finer in the areas of interest.
Thanks very much for your answer. Yes I did a few more tests and settled on a mesh I feel happy with.