This is a very common occurance. Simulation sees the stress there as being infinite or close to it so refining the mesh actually causes the reported stress to climb. A coarser mesh will most likely give smaller stresses in that area but they will probably still not be totally accurate. Without seeing you model or the mesh I cannot really say if there is a way that you can easily get around this but it is a meshing issue. Have you tried the Curvature Based Mesher? Since it meshes the part based on curved geometry sometimes the mesh will be cleaner and give better results.
Looks like bad elements. You should plot aspect ratio and jacobian and see if red shows up in the same location. Looks like it will. You just need to make smaller less distorted elements. Try a mesh control.
The geometry does look right for a stress singularity (accelerated increasing stress with smaller mesh size)
Lots of possible reasons: overconstrained boundaries, bad mesh, high aspect ration, element size w.r.t. neighboring elements. Use mesh controls to ensure good mesh in key areas.
Hi Peter: Prob just a "bad" set of elements, as others have remarked. You can plot "Element" stress to show the averaged value of the stress throughout the elements and probably identify the exact element(s) causing the issue - put mesh controls there. Also create a mesh plot of the elements' Aspect Ratio, and another of "Jacobian". You really do not want elements AR > 5 (in areas of interest). The Jacobian shows you how "folded-over" the element might be (numbers < 40 are supposed to be acceptable). Here are a couple screen shots showing AR and Jacobian from a course I developed.
Thanks for the follow up. I really appreciate the thoughts and different ways to approach the issue. I took another path, I suppressed the feature from the model and re-ran the simulation. It ran normally and the results are reasonable.