14 Replies Latest reply on Oct 8, 2010 9:32 AM by Ron Reiners

    Benchmark for Flow Sim

    Rich Bayless

      Hello,

       

      I was wondering, what is the best method to measure the performance of hardware when using Flow Simulation?  What have people done for benchmarking?

       

      I saw the Solidworks benchmark results at http://spreadsheets.google.com/pub?key=pz7wTpIkC7LA28ybEyxyTPw using the standard punch holder from Anna Wood's site http://www.solidmuse.com/solidworks-benchmark-punch-holder.html.

       

      The punchholder test is good for Solidworks performance and stresses the CPU primarily, with relatively low memory usage.

       

      Flow Sim uses lot's of memory for a high cell count model and uses the CPU's, although not fully for a single simulation.

       

      I would imagine that selecting one of the tutorial models would be a good start.  Set up a configuration with boundary conditions, heat source, radiation, porous media, etc., and set refinement to achieve a specified cell count.  Then clone that configuration several times so you will be able to run one model at a time, two at a time, four at a time using batch solve.

       

      Any one interested in trying something along these lines?

       

      Cheers,  Rich.

        • Re: Benchmark for Flow Sim
          Kristofer Skyttner

          Good idea. Would be very intresting.

           

          Should a bechmark try to include as many calculation-elements (radiation, porous media, etc) or should you limit it to the most commonly used (whatever they are..)?

           

          It could be a risk of low participation rate if it takes to long to solve the benchmark, but it would be interesting to benchmark as much of Flowsim as possible. It would also be good to have a big distribution portal as Anna woods homepage to get up the participation rate.

           

           

          //Kristofer

          • Re: Benchmark for Flow Sim
            Kristofer Skyttner

            Due to the lack of responses, here comes another more concrete proposal. How about using a tutorial-file that comes with Flow Sim? The CPU-cooler example (Tutorial Advanced 6 - CPU Cooler) could be a reasonable test case for a benchmark. In addition to rotation and heat conduction one could add gravity and radiation, then almost all options are tested.

             

            It's anyhow a good tutorial.

             

            asm.jpg

             

            (Image from ...\SolidWorks Corp\SolidWorks Flow Simulation\lang\english\Docs\Tutorial.pdf)

             

            cut1.jpg

             

            (Image from ...\SolidWorks Corp\SolidWorks Flow Simulation\lang\english\Docs\Tutorial.pdf)

              • Re: Benchmark for Flow Sim
                Anna Wood

                I am not a simulation person.  If someone creates an assembly setup for a flow simulation I will be happy to host it in my website.  I can create a web form for results and a new Google spreadsheet.  It needs to be complex enough to stress a good system.  This will allow the benchmark to live for several generations of hardware.  Chhers, Anna

                  • Re: Benchmark for Flow Sim
                    Rich Bayless

                    Hello Anna,

                     

                    Thank you for the offer to host the results.  That would be a large barrier otherwise.

                     

                    Hello Kristofer,

                     

                    That tutorial looks like a great place to start.  I'll see about running it next week as time permits.

                     

                    Have a great weekend!

                     

                    Rich.

                    • Re: Benchmark for Flow Sim
                      Bill McEachern

                      Attached is a model that I supplied to Ian Hogg which he used in his performance benchmarks for Flow sim when they introduced the vectorized solver. It is a straight incompressible problem with no heat transfer. It is based on a NACA paper for submerged inlets (NACA Ducts). There are lots of papers on submerged inlets and I can't recall which one exactly I used but it is one of the attached I think.

                       

                      I have not looked at this file in a while but there should be studies set up to run. The goal here was to generate and visualize the vortex system that makes the things work in the first place which requires a decent flow solution to pull it off.

                       

                      If you need any help feel free to ping me a note.

                      Bill

                        • Re: Benchmark for Flow Sim
                          Anna Wood

                          I will give it a look.  Will ping you with questions.  Thanks, Anna

                            • Re: Benchmark for Flow Sim
                              David Paulson

                              Anna,

                               

                              In your benchmarks, how far back do you go in versions of SolidWorks?  With FEA and Flow Simulation multiple processors can have a huge impact.  Flow enabled multiple core processing with v2009.  It would be beneficial if the benchmarks went back to at least v2008.  FEA and Flow are applications where the efficiency of the code using many processors are more important than the speed of the processor.  It would be very nice to know if the code is becoming more efficient with parallel processing hardware.

                                • Re: Benchmark for Flow Sim
                                  Anna Wood

                                  David,  It all depends on what version file Bill set-up.  If it is a SW2008 file we will start there.... Otherwise, it will be a later version.  Cheers, Anna

                                    • Re: Benchmark for Flow Sim
                                      David Paulson

                                      Anna,

                                       

                                      Multi core applications deal with more variables than do single core applications such as Solid

                                      Works.  With the benchmark it is desired that you can evaluate not only the hardware and the number of cores applied to the solution, but also the software version and it's ability to utilize as many cores as possible.  This is where the benchmark can become a meaningless number if you cannot test one hardware configuration against the last five versions of Flow or Simulation.  And conversely test one version against a number of hardware platforms.

                                       

                                      The cost of upgrading to the latest vesrion of any simulation of flow program can be more $$ than upgrading the hardware.  Upgrading the hardware is a somewhat known result.  Upgrading the software is more of an unknown, and probably more expensive.  I can add another CPU to my system fto bring it up to 8 cores for about $500-600, whreas it might cost me $4-5,000 to upgrade Flow 2009 to Flow 2011.

                                       

                                      So, Bill, I hope that you can provide the model in the latest five SW versions.

                                        • Re: Benchmark for Flow Sim
                                          Bill McEachern

                                          It is what it is. I do not recall what version it is but if you have older versions of Flow feel free to export a parasolid and load it into the oldest version you have and rebuild the study - it would not be difficult as it only has 3 BC's and a mesh control, a couple of goals and a few plots. I would be happy to supply the set up info if you can not load the file. It would then be possible to up load into later versions directly. I do not have any older versions as I have 2010 on any of my current systems. Feel free to contribute though..

                                           

                                          On the performance bench marks the code can be run with a specified number of cores/processors which would provide scaling efficiency metrics - on whatever version is being run. In the case where you have enough RAM and need multiple studies you can get near perfect scaling (this is an assumption on my part as I have not confirmed it but seems likely - it would be nice to know for sure) with the code by running multiple jobs on individual cores. Scaling efficiency is also dependent on what physics is enable in the model. Some physics is more amenable to vectorization than others - gas dynamics comes to mind. When Ian did the previous work he did run an array of physics and an array of mesh densities for the four or five problems he used. Maybe Ian could post the summary slide of that work to supply some sort of base line - maybe even the other models he used. I do not think anything proprietary was used - training models I think. Ian?

                                            • Re: Benchmark for Flow Sim
                                              Ian Hogg

                                              Hi Bill (et al),

                                               

                                              The training models would be a good baseline since they touch on the various facets of the product. They can also be downloaded from:

                                               

                                              http://www.solidworks.com/sw/support/808_ENU_HTML.htm

                                               

                                              What's nice is that they have from 2008 onwards. The only problem is that some models have changed over the years, so you may not find all of them in the earlier versions. But as you said, Parasolid out/in is always a fallback.

                                               

                                              When I first did the multicore benchmarking for 2009, I used the following training models (plus a few others):

                                               

                                              ModelFlow type
                                              Lesson5 level 1 meshTransient Conjugate Heat  Transfer
                                              Lesson5 level 1 meshTransient Conjugate Heat  Transfer
                                              Lesson 3Steady State Incompressible Flow
                                              Lesson 12Steady State Rotating Region
                                              Lesson 6Transient incompressible flow
                                              Lesson 3 autorefinedSteady State Incompressible Flow
                                              Lesson 11Supersonic  Flow
                                              Lesson 4Steady State Incompressible Flow
                                              Lesson5 level 4 meshTransient Conjugate Heat  Transfer
                                              Lesson5 level 4 meshTransient Conjugate Heat  Transfer
                                              Lesson 7Steady State Conjugate Heat Transfer
                                              Frying PanSteady State Conjugate Heat Transfer
                                              Power SupplySteady State Conjugate Heat Transfer
                                              NACA Duct testSteady State Incompressible Flow

                                               

                                              The frying pan and power supply are corporate demos.

                                               

                                              On a side note, as far as flow simulation performance goes, product version is going to have a secondary influence in most cases. Every now and again there are improvements to the code for specific problem types, but overall the biggest bang for buck is multicore and cpu speed. Those are the primary factors. Will you see much difference between 2008 and 2009 run on a single CPU? not likely.

                                               

                                              From the 2009 testing, 2 cores gave a significant reduction in solution time over all classes of problems. 4 & 8 cores, helped further, but it was diminishing returns (which was discussed in another flow thread some time ago). It will be interesting to see in the coming years how software developers continue to take advantage of parallel processing.

                                               

                                              For reference, below is the performance improvement I did for 2009.

                                              Flow_2009_MultiCore_Performance.JPG

                                               

                                              Cheers,

                                               

                                              Ian