13 Replies Latest reply on Mar 12, 2014 4:54 PM by Josh Killalea

    reducing file size on your server.

    Josh Killalea

      Just wondering what everyone does to keep the file size to a minimum on thier servers?

       

      we work on some massive project files (one we are working on right now takes up 8.5gb of storage when you copy the project folder out of the vault.) and we only got one terrabite of storage on the server thinking that would be heaps. but even with only this project one other "small" (for us) project on the server it is already almost full. we know about the cold storage thing and we have been reading up on that, but to me that looks like you are just filling up another drive in lieu of the one you have all your active data on just in case you MIGHT want to roll back to a different version oneday. in our line of work we would 99.99% of the time never want to roll back more than 1 or 2 versions and even then we would probably never actually do that.

       

      so my questions are;

       

      what do you do to maintain traceability and version control and keep your file size to a minimum?

       

      also

       

      if you have a whole heap of information on your extra drive with the cold storage on it, can you delete reduntant information from that drive? from what we read it doesn't look like you can.

       

      thanks in advance for any help.

        • Re: reducing file size on your server.
          Charley Saint

          Josh,

           

          EPDM already does a bit of redundant file cleanup if every bit is the same, but even changing a custom property would be enough for it to keep that version. Some people use cold storage to move files to a backup tape, so they're still there if you absolutely need them but they are on cheap massive storage you can keep in a firesafe somewhere. You can look into disk de-duping too if you're on MS Server 2012, but you take a small performance hit. Best bet is to use cold storage.

          • Re: reducing file size on your server.
            Feroz Mahomed

            Josh,

             

            Suppressing all parts in an assembly file and then saving file reduces the file size considerably.

            • Re: reducing file size on your server.
              Josh Killalea

              Thanks for the replies guys. Much Appreciated.

               

              Charley, we had a feeling from everything we were reading that the only option was cold storage. but unless when EPDM sends versions into cold storage is MASSIVELY reduces the file size of the cold stored file as well as the active file it isn't really a viable option. so i guess that is another question. does it work like that or not? i can't recall reading if that was the idea or not.

               

              the only thoughts i has were;

               

              1. we have to regularly "clean up" EPDM by taking out old projects. so we have to go in every 3-6 months and remove old projects that are no longer being worked on and stor that information somewhere else on the server.

               

              2. on massive projects, like the current one, every couple of months we would need to copy the project out of EPDM and delete the EPDM version. then copy the project back into EPDM. this would effectively force EPDM to see all these files as "new" files and they would all be a version 1 again, thus reducing the space the files take up on the disk.

               

              can anyone see any issues with these ideas? obviously, with the first idea, there is the issue of having to get information back into EPDM if we ever need to use it again. but i don't think it would happen often here with the way things work.and i think the second idea would work, but i am not sure what it would do to the SQL data base. and i am still not 100% sure what it is that is actually chewing up the server space SO fast. we keep mimimal information on it other than the EPDM stuff (in total all the folders on the root drive only add up to 415Gb of on disk space when you look at their properties.) so i figured it had to be the files chewing it up with 20 versions of 5Mb files. and it takes massive chunks of space when we push a drawing with it's model through our workflow.

               

              Feroz, i have noticed that in my travels, but it is not a practicle solution for our issue here. we have thousands of assemblies in our master model in a single project. i also fear that it would be counter productive as it would only serve to creat yet another version of the file within the Vault and hence adding to the issue in stead of easing it. have you tried this within a Vault? did it make any difference to your "on disk" file sizes? help free up space?

               

               

              If anyone has ANY other ideas that would be greatly appreciated.

                • Re: reducing file size on your server.
                  Feroz Mahomed

                  Josh,

                   

                  I have done a test by checking out a project and renamed all the parts.All the parts were then suppresed in the assembly and checked back into the vault as a new project. We then got the It guy to compare the size of the 2 projects on the server and found that the project with the suppressed parts was up to 60% smaller.

                   

                  We have not implemented this system yet as our server currently has enough capacity. Our assemblies are up to 4 parts only. Some good people helped me with a macro that would suppress all parts for you with a click of a button.

                    • Re: reducing file size on your server.
                      Tim Read

                      If you are thinking of suppressing files in the assembly to reduce filesize then you'd be better off by just reducing "Image Quality" or not saving the tessalation data in the files. Reducing Image Quality will also increase the speed that graphics perform.

                       

                      I quickly saved one of my smaller assemblies (less than 20 parts) with four different settings of Image Quality to show the effect this has. It will depend on the particular assembly how big it would be on disk but you'll get the idea.

                       

                      Low - 1,124KB -> ~1MB

                      1124KB.JPG

                      Medium - 2,566KB -> ~2.5MB

                      2566KB.JPG

                      High - 3177KB -> ~3MB

                      3177KB.JPG

                      Insane - 42501KB -> ~40MB !!!!

                      42501KB.JPG

                    • Re: reducing file size on your server.
                      Brian McEwen

                      I don't work with such massive projects, but sounds like you need to get more storage for your server.  Hard drives are cheap, time is not - though I imagine all the server setup and stuff I don't know a ton about is more of a pain. 

                       

                      Then - instead of cold storage to save a copy - you could use cold storage to simply delete older versions (that removes the step of going back to clean up the cold storage drive). You can pick keep 5 versions or whatever works for you. I have ours on keep 75 versions (then delete), and always keep Revisions, so quite a differt situation.   Do you even use Revisions with that kind of big project?

                       

                      [Edit] It would be nice if cold storage had more options.  A two tiered system (eg older than 10 versions move to compressed storage, older than 30 versions delete it), cold storage based on how old a file is, etc. But a VAR might be able to custom create some more choices for you...  Also I can't imagine that it duplicates the whole file if you just change custom properties on a file - yeah it creates a version, but it must take less space?  The rule of thumb I heard (probably very approximate) was that versioning would lead to maybe 5 times the file size of your unversioned files - How does that match other's experience?  We have not fully launched yet. 

                        • Re: reducing file size on your server.
                          Josh Killalea

                          thanks again for the tips guys. suppressing files wouldn't really be an option for us. we have WAY to may assemblies and it doesn't look like it makes a big enough difference for what we are after anyway. (thanks for taking the time to do that Tim Read.)

                           

                          Brian, you mentioned that you can use the Cold storage to delete the older versions? so if we set it to say 5 versions and keep the revisions, that would be possible? that to me sounds like what would be the result we are after. we haven't used versions at all yet. because of the nature of our work we have a work flow that means we don't require versions as such. but would still like to keep a few just to be safe. could you point us in the direction to look for that? we should be trying to set this up in the next week or so.

                           

                          thanks again for the help.

                        • Re: reducing file size on your server.
                          Wayne Matus

                          You can also have EPDM compress old versions of files. Search the Administration Guide for Compress.  I will take longer to get previous versions though.

                           

                          Also have someone peodrically destroy deleted files. Deleted files are still stored on the archive server. Destroyed are removed from the archive server.

                        • Re: reducing file size on your server.
                          Josh Killalea

                          well, the other guy here who does our admin stuff as well had a much closer look at what was chewing space on the server yesterday and he discovered that when corperate IT had set up the server for us before delivery, they had set the windows back up to every night. we had left this as is, but had been deleting all bar the last 3 copies. what we didn't know is that when it does a back up, it is creating a "shadow copy" fo the back ups that wasn't evident or obvious when we were looking at the disk space being used. to work this out you have to go to the Windows Backup and then have a look in there and it tells you how much space is being used in the destination folders. it was using 700Gb for these "shadow copies". so we have sorted that now and redirected that to an external drive and will loo ate deleting the old back up stuff as soon as we are confident that the new back up settings are working 100%.

                           

                          thanks again everyone for the help. i hope this thread will help someone else in the future, as there are a lot of great ideas in here.