13 Replies Latest reply on Sep 11, 2014 7:37 PM by Jared Conway

    Data Center Airflow Analysis

    Renee Matthews

      Is anyone else using SolidWorks Flow Simulation to analyze airflow in data centers?  I'm currently in the process of verifying and validating a model I built of an existing data center.  I took a lot of environmental readings (i.e. air temp, air speed, and humidity) for the individual racks/servers, In Row Coolers (IRC), and Computer Room Air Conditioning (CRAC) units to help with fine-tuning my model as well as some readings throughout the room at various monitor points to use as a "basis of comparison" for the simulation results.

       

      Trying to find a way to simplify the model while still being able to duplicate the real-world results in the simulation has proven very difficult.  For instance, this data center uses a lot of different equipment in its server racks, but the most populous one is the HP C7000 blade enclosures.  HP developed their own fan for these enclosures - the Active Cool 200 fans. (I was able to get the specs of the fans from HP and added them to the Engineering Database.) Each C7000 has ten of these fans. In trying to figure out how best to build these enclosures in SolidWorks, I tested them in a "wind tunnel" in two ways: 1) with all the fans modeled separately and 2) as merged fans (fan curve CFM x 10); (I also made sure that the total fan area was the same for both). I found that while the model with separate fans did mimic the real-world readings, the merged fans did not accurately replicate the real-world environmental readings that I took for these enclosures, specifically for air speed readings and temp!  I tried multiplying the airspeed by ten, but that didn't work.  I then used Model #1 and attached a fan curve (vice an axial) for one fan to each modeled fan, and the results were not the same as for the when I attached an axial fan to each separate fan.  If the only difference between these models is to change the fan property from axial to simple a fan curve, shouldn't the resulting air speed and temp be the same?

       

      Also, I've had to make the servers and perforated floor tiles "porous media" in order to take into account the air resistance of the servers perforated tiles, which has greatly increased the number of mesh cells; using Mesh Level 1 overall with no porous media and the number of cells = 1,356,466, and then with porous media = 2,828,189 cells?

       

      Any suggestion in how to simplify this model while still getting accurate results?

       

      My company recently procured a more robust computer for SolidWorks (see below), but the CPU gets maxed out when running the model and is projected to take hundreds and hundreds of hours to finish.

      Dell Precision T5610 with

      Intel Xeon Processor E5-2620 v2 (Six Core HT, 2.1GHz Turbo, 15 MB)

      16GB (4x4GB) 1866MHz DDR3 ECC RDIMM

      3 GB NVIDIA Quadro K4000 (2DP and 1DVI-I) (2DP-DVI and 1DVI-VGA adapter)

      2TB 3.5inch Serial ATA (7,200 Rpm) Hard Drive

      8x Slimline DVD+/-RW Drive

       

       

        • Re: Data Center Airflow Analysis
          Tu Ba

          Running 1 mode  and take hundreds of hours,that's awful 

          • Re: Data Center Airflow Analysis
            Jared Conway

            so it sounds like you have 3 questions

             

            1. simplifying the blades, I don't think I understand what problem you're running into or how you moved from one solution to the other or why you thought they would be equivalent. axial fans and fan curves are completely different.

             

            2. porous medium increases number of cells

             

            3. predicted solve time is 100hrs on decent hardware, how to minimize the size of the problem to reduce solve time

             

            my first thought is that you gave us no impression of the size of the data center you're working with, how well you've simplified the assembly or what the size of the element is. I've seen many situations where someone has setup a large problem and chosen something that is very small, this will make the element size very small even if you have chosen lvl1 mesh. that is where manual meshing comes into play.

             

            after that, you need to decide how much accuracy you want and what your plans are once you complete your benchmark. that will help you determine how simple of a model you can buld..etc. based on the way you described things, it sounds like you have an interest in getting a high degree of accuracy but at the same time you want a fast solve, those things unfortunately don't go hand in hand.

              • Re: Data Center Airflow Analysis
                Renee Matthews

                Thank you for your response!

                 

                1. As far as simplifying the blade server enclosures - each enclosure has ten fans and I was told by our SolidWorks reseller that I could merge the ten fans into one big one by changing the attributes of the Active Cool 200 fan in the engineering database from axial to fan curve and multiplying the CFM of the fan curve by ten and attaching that to one end of the enclosure instead of modeling each individual fan.  And I found that that didn't replicate the real-world readings that I took in the data center.  Now I know why - "axial fans and fan curves are completely different."

                 

                2. I understand that porous media increases the number of cells, but does the solid body and the physical attributes in the engineering database (spedifically length) need to match? Can you make the dummy body of the porous media, say, 1 in. long even though the actual length is 30 in.  As I have it now, if a server is 30 in. long, that's how I have it built in the model with the engineering database also reflecting the actual length.  Would this reduce the number of cells for the porous media?

                 

                3. I've run simulations of the individual server racks and the HAC. Is there a way of incorporating those results into the larger room instead of the model?

                 

                Here's an overhead view of the room I'm working on:

                Room_1121C_overhead.png

                And one with dimensions:

                Room_1121C_overall_dimensions.png

                  • Re: Data Center Airflow Analysis
                    Renee Matthews

                    Continuing from my previous post... I have put local meshing on the fans and their exhaust airflow and around the monitor points that I've placed around the room (they're not noted in the overhead views, sorry).

                     

                    After validating and verifying that the model (specifically the individual parts) is built correctly and results mimic the real world, I will be analyzing new server equipment that will be going in to this room - potential layout of the equipment and perforated plates as well as power density of the racks (i.e. how much can be put into each rack and still be able to privide sufficient cooling) and whether or not there will be any "hot spots".  Accurate airflow, temp, and humidity will be important.

                     

                    Any suggestions would be greatly appreciated!

                    • Re: Data Center Airflow Analysis
                      Jared Conway

                      1. the theory sounds reasonable but it needs to be tested. i would double check that your tests are consistent.

                      2. take a look at the tech reference and help or discussions about porous medium in this forum. you enter the known values, flow simulation will scale relative to your solidworks geometry.

                      3. only the way you've described with 1. basically creating "bulk" properties, there is no direct approach. only the opposite where you start coarse and then add refinement with efd zooming.

                  • Re: Data Center Airflow Analysis
                    jerry towne

                    Renee,

                     

                    With fans in series the head is additive.  With fans in parallel the volumetric flow is additive.

                     

                    Jerry