I don't use Visualize, however, from your description is sounds like the video ram might be getting maxed out. Although I would think Hybrid rendering would prevent this.
Most likely you have tried rendering without GPU, (CPU rendering only) but just in case you have not ...
Thanks for the insight; you are correct.
Running Visualize in GPU or Hybrid mode will result in using the VRAM (video memory) on the graphics card to process your model in Visualize. You can change this setting in Tools > Options > 3D Viewport tab. Uncheck 'Automatic' and you can change the hardware using for Visualize:
Your Quadro 4000 has only 2GB of VRAM, but will likely render faster than your CPU.
Your CPU has 12GB of ram, so you can open up larger models, but will be slower to render than your GPU.
Rule of thumb: Visualize can open 5 million polygons per 1GB of VRAM/RAM. So your Quadro 4000 can import a model up to 10 million polygons (2GB x 5 million), whereas running in CPU modeyou can import a model up to 60 million polygons (12GB x 5 million).
You can see how many polygons your model is in Visualize in the HUD (Heads Up Display) at the bottom of the screen:
Also try GPU only.
There is another thread regarding decals and only one type of mapping is supported.
Finally, upgrade from SP0 to SP3. Lots of enhancements/fixes with each SP.
So I re-opened that project to try it in GPU mode only and it still crashes the video card driver and fails. I was trying to run it in Hybrid mode as it seems to be the best option. When I ran it as CPU only it was estimating taking more than 4 times longer to run. I did close out of everything before running the test and the project has less than 900,000 polygons. I did create a duplicate of the project and got it to work just was curious as to why it would crash the video card every time and not render.