It may have been built from the z axis but it was then exported and imported prior to making the user defined coordinate system. That may have caused a misalignment of the temporary axis and the bore face. Did the solid Import? If so, I might try filling in the imported bore and cutting the bore again in solidworks.
Ha ha, it's only 14 months late, but if you construct a point at the center of the circle/arc of the hole, then add a coordinate system there, use the coordinate system after selecting the surface of the hole to apply the bearing load. With the two items selected, enter the direction and value.
Creating the coordinate system is necessary, for whatever reason, but when I do that on a similar problem it works. Without it, you get the same answer you explained above.