10 Replies Latest reply on Mar 27, 2017 3:16 PM by Jason Koonce

    PDF's in PDM

    Jason Koonce

      We are getting ready to batch dump about 100,000+ "Legacy" PDF's into our new PDM system.  Currently I have a "PDF Released" folder which is where all the pdf's will be housed.  Now my thoughts are that this is going to be a lot of files in one folder (Doesn't windows have a limitation of 256,000+- files in one folder).  All the PDF's are named a 9 digit part number, so I could create multiple folders under "PDF Released" with 10,000 or so part numbers in each folder.  Is this the best way to handle the situation?  Also, going forward is there a way to tell PDM to only allow a certain number of files (say 10,000) in PDM then create a new folder based on the next 10,000 range or is this something that would have to be manually handled?  Any thoughts or suggestions are greatly appreciated.  Thanks! 

        • Re: PDF's in PDM
          Wojciech Paterski

          you definitelly don't want that many files in one folder - very slow down to load whole content to look through it.

          we have parts/assemblies based on series numbers, so 0426- , 0400-, s018-, etc, and they have they own folders (those are standard parts that can be used on multple projects) then separately we have poject main folder with project sub folders (they start with letters) and generally each part pdf/dwg/sldprt/asm/drw are in they respective folders.

          which some folders haveing been already on few thousands are slow;/

          • Re: PDF's in PDM
            Amos Avery

            We had an issue with a user complaining of slow access to PDM folders full of PDF's.  Searching the forums I found advice to keep the file count below 3000. 

             

            I manage multiple Vaults at multiple sites and found that, while this is a safe rule of thumb, I've experienced much larger folder sizes without issue.  The database doesn't care how many files you put in a folder.  When you open that folder, however, Windows must draw up a pretty picture of every file in the folder so you can see what's there (lots of data loaded into cache).  That takes time for large folders.  It also seems to take more time for large files.  We found that if a folder had many small PDF's, it would still load quickly in comparison to a folder with the same number of large PDF's.  So I've expanded my rule of thumb to say keep your folders smaller than 3Gb.  For us that means some folders have 15000 (CAD generated) PDF's and some have 3000 (scanned paper drawings) PDF's. 

             

            The best advice I can offer is to train people to use the Search tool.  The users were opening the folders and complaining about how long it took just to open a PDF.  With Search, the actions were many times faster (because Windows doesn't need to load all of that folder data into cache).

            • Re: PDF's in PDM
              Jason Koonce

              I spoke to my VAR about this issue and they suggested no more than 5,000 files in a single folder.  Just an FYI is someone else has this question.

              • Re: PDF's in PDM
                Dan Hon

                Have you considered creating an index file of all these PDF files?  It's a bit out of the box and would require some extra work, but index files are very quick.

                 

                For example, by coping this code to a text file with .html extension

                <html>

                <body>

                <h1>   PDF Folder</h1>

                <a href="conisio://VAULT_NAME/open?projectid=xxxx&documentid=xxxxxxxx&objecttype=1">112553.pdf</a></br>

                <a href="conisio://VAULT_NAME/open?projectid=xxxx&documentid=xxxxxxxx&objecttype=1">112554.pdf</a></br>

                <a href="conisio://VAULT_NAME/open?projectid=xxxx&documentid=xxxxxxxx&objecttype=1">112555.pdf</a></br>

                </body>

                </html>

                 

                A searchable index file is created. You would need to replace:

                          VAULT_NAME: with your vault name

                          projectid=xxxx:  Replace xxxx with PDF folders ID **

                          documentid=xxxxxxxx: Replace xxx with the PDFs ID **

                 

                ** The folder ID and document ID can be extracted by adding the file ID tag to the Search Columns.  Then, by utilizing the PDM Search Tool, you can export a  CSV file (spreadsheet) of all these file values.

                 

                Populate the index file with the ID values for all 100,000 PDF's.  Of course you would need to create a scripting function to keep populating this index file with future additions.

                  • Re: PDF's in PDM
                    Jason Koonce

                    Thanks for the info Dan!  All of our file names will be 9 digits long (123456789.pdf).  I have created a dispatch that will get the first 5 numbers (12345) and the 6th number (6) depending if it is 0-4 or 5-9 will create a folder (either 123450000 or 123455000).  All PDF's will be sorted by this technique.  This should allow for no more than 5,000 PDF's in one single folder.  Since I am using PDM 2015 SP5, I have created another dispatch to copy the files to their respective folder and delete the original (I don't like this one bit).  I have heard in PDM 2017 you can move a file with dispatch.  I am looking at upgrading before we "Go-Live" to have this added functionality.  If for some reason, this does not work.  I will be looking at your indexing option.  Thanks again!