Have you confirmed that the only thing selected after the "SelectByID2" statement is the face where you want the sketch?
Have you checked the value of boolstatus? Just as Josh said, it probably didn't select the face. If you created a sketch on that face before, you should be able to get the face object in another way. Maybe you can even use the persistent ID and re-use that one.
Using the coordinates isn't a very robust way of selecting something. From the SelectByID2 help:
"If you do not know the object name or the object type, pass empty strings for the Name and Type parameters. The selection routine makes the best attempt to select the correct object."
There's some more on selecting faces there:
Coordinates - yeah. In this case, it's a simple test always starting from the same part file, so I don't think that's what tripping things up.
You're right about the status. In the full addin, my call to:
status = doc.Extension.SelectByID2("", "FACE",
xyzuv, xyzuv, xyzuv, addToSelection, mSelMarkForFace, null, 0);
was returning false. Looks like my call to InsertSketch was picking up the existing sketch because it was the last thing on the Feature stack. Some kind of default behavior maybe.
My PMPage had no selection box with filter swSelFACES, so I added one. Magic! The selection started working. Even better, I get a new sketch right on top of the existing one, like I was hoping for. Perfect. I made the selection box not visible, since what the user does is pick a sketch segment from the existing sketch (which goes into a different selection box with different 'mark'), and I make a new sketch for the new lines derived from the selection.
One last thing. Calling SelectByID2(..) with addToSelection = false seems to be not enough to clear old selections. I have to call swDoc.ClearSelection2(true) before that in order for things to work.
So, things are working for now. Lovely day in this minefield. =/
Ah, some fresh info that sheds some light on the best way to do this: the user has selected a sketch segment!
SelectByID is very rarely the best way to go about selecting things with code, even though that's what the recorder does. Don't listen to the recorder. You rarely know what the ID is, and you can rarely be confident that there's actually something located at the coordinates you're passing in.
If your user is selecting a sketch or a sketch segment, you can get the sketch plane referenced from the sketch, then select it with Select2 or 3 or whatever it's up to now. That way, it doesn't matter what condition your base model is in. As long as the user has selected a sketch segment, your code can make sure it's a 2D sketch, then select the sketch plane and insert a new sketch.