I downloaded your file and got the same result. I see that your motion study does not have any mate to control the distance that the gear travels over time. I added a vertical plane at the start of the rack, then added a distance mate to the axis of the gear. In the motion study I set that distance to be 2.5" at start and 10" at end.
Testus WLTdesign.mp4 226.3 KB
Wouldn't the original motor function at 6RPM control the distance over time? A distance mates between planes would have to override that. Or perhaps I just need the starting location more defined...