Welcome to the forum. With your drawing open, go to Tools > Options > Document Properties > Dimensions and set them there. I am at home now and don't have SW here so I can't post a screen shot. I also don't remember if it's set there in Dimensions or if you need to expand Dimensions and set the font for each dimension type (Linear, Angular,Diameter, etc.).
When you get it set to the proper size, you might want to save your drawing template so you don't have to change it every time you start a new drawing.