21 Replies Latest reply on Jan 3, 2014 4:39 PM by Pete Yodis

Number of files in any given folder

Pete Yodis Jan 2, 2014 9:47 AM

I've seen the question for how many files per folder come up a few times here in this forum. The 2,000 number seems to get thrown out often. We desire to have all of our drawings converted to PDFs and located under one folder for easy access by the rest of the company. We will have around 23,000 PDF files once we migrate to EPDM. We are running EPDM 2013 (we will be at 2014 when we go live) on a Windows Server 2008 R2 (We will be upgrade to Windows Server 2012 when we go live) machine with 32GB of RAM, an 8 core CPU running at 2.2 GHz, storeage for files is on a RAID array with 15,000 RPM SCSI drives with plenty of storage at the moment (540 GB).

We are using the convert task to create PDFs for our SolidWorks drawings. The convert task will be setup to save PDFs to just one directory. Does anyone see this as an issue? Will 23,000 plus PDF files in one folder be a killer? Seems like the convert task would have to get a bit more complicated to place files in subfolders based on file name.

Re: Number of files in any given folder

Pete Yodis Jan 2, 2014 9:50 AM (in response to Pete Yodis)

I should also mention, that shop users will probably use IE to browse/search the vault for the PDF drawing they are looking for. We are migrating from PDMWorks WorkGroup and have used the webportal for shop users to find drawings. The webportal with WorkGroup has a setting to create virtual folders for a folders with large numbers of documents in them. This was to speed up browsing in those large folders. Does the web access in EPDM have something similar?
Like Show 0 Likes(0)

Actions
Re: Number of files in any given folder

Lawrence Kiefer Jan 2, 2014 10:01 AM (in response to Pete Yodis)

Pete,

The standard reply of 2000 files per folder is what my reseller told me as well. We originally had the same question when we implemented EPDM, and we also use convert task to save PDF versions outside the vault for all of the departments outside of engineering to have acces to released drawings.

We have folders outside of the EPDM database with PDF files containing upwards of 5000 files currently, and growing.....and we do not have any issues.

I believe that the 2000 number applies to files stored in the database itself. My reseller basically told me we could store more files in one folder in the EPDM database, however it would start to slow down searching for file.....
I know back in the old days of using the fat and fat 32 file system in windows there were limits, but with the NTFS format windows claims no limits on files per folder.

Hope this helps...
Like Show 0 Likes(0)

Actions
- Re: Number of files in any given folder
  
  Pete Yodis Jan 2, 2014 10:11 AM (in response to Lawrence Kiefer)
  
  Thanks Lawrence,
  
  We plan on storing the PDF files in the vault, so that we can keep the revision history of the PDF files. I just checked and there are about 15,000 plus PDF files...so my 23,000 number was high. I am tempted to just run with 15,000 PDF files in one folder named "Released PDF Drawings' stored in the vault, unless someone else here tells me this is a very bad idea. From an operations point of view, this is really easy. I would hate to have to sub-divide that folder into subfolders and get fancy with the PDF tasks...yuck.
  
  Like Show 0 Likes(0)
  
  Actions
  - Re: Number of files in any given folder
    
    Lawrence Kiefer Jan 2, 2014 10:27 AM (in response to Pete Yodis)
    
    Pete,
    
    I don't think you will have any issues. I would do this, and simply keep adding files until you notice issues. Chances are, you wont have any. As far as keeping them in the vault for revision tracking, with the convert to PDF task you can set it up to add a suffix to the filename with the latest revision, so even if you are storing them outside of the vault the revision can be tracked. Of course there is no rollback or history, but I don't see that as a problem as the solidworks file itself in EPDM has that...My reasoning for storing the released PDF drawings outside of the vault was because the company I set EPDM up for is fairly large and has a large number of people that need access to released drawings outside or engineering. If I kept these files in the vault, the company would have to purchase enough seats of EPDM viewers to accommodate and as you know, that is not cheap.....
    
    Food for thought. Good luck.
    
    Like Show 0 Likes(0)
    
    Actions
    - Re: Number of files in any given folder
      
      Pete Yodis Jan 2, 2014 11:17 AM (in response to Lawrence Kiefer)
      
      Lawrence,
      
      I can see your reasoning to limit the number of EDPM seats. That is why I mentioned the web viewer being used here. With WorkGroup, the web portal did not consume a license of the client software. I am not sure its the same with EPDM. I am curious to see how browsing a large folder with IE is going to behave.
      
      Like Show 0 Likes(0)
      
      Actions
Re: Number of files in any given folder

Jeff Sweeney Jan 2, 2014 10:23 AM (in response to Pete Yodis)

Just playing around one day I put 44,500+ files in an EPDM directory. https://forum.solidworks.com/message/50412

Within reason it mostly boils down to browsing preformance.

That 2000 number is very outdated...back in the old XP days. Most people's threashold of pain on a good Windows 7 seems to be around 8000 files. Clients with slower machines will start to complain first.

If users use the search tool rather than browsing, the size isn't going to be much of an issue.
Like Show 1 Likes(1)

Actions
- Re: Number of files in any given folder
  
  Pete Yodis Jan 2, 2014 11:19 AM (in response to Jeff Sweeney)
  
  Thanks Jeff. We'll start off with around 15,000 files. Hmmm. Sounds like it may be a little slow. Any thoughts on the web viewer and it's performance?
  
  Like Show 0 Likes(0)
  
  Actions
  - Re: Number of files in any given folder
    
    Jeff Sweeney Jan 2, 2014 11:58 AM (in response to Pete Yodis)
    
    Honestly never tried large folders with it. I bet it will be either pretty snappy because it only displays 30 files at a time or very slow.
    
    How's that for covering my bases?!
    
    Like Show 0 Likes(0)
    
    Actions
    - Re: Number of files in any given folder
      
      Pete Yodis Jan 2, 2014 12:02 PM (in response to Jeff Sweeney)
      
      Nice... way to cover the range. Looks like I will have to try it. Does a user surfing the vault with IE, consume a license?
      
      Like Show 0 Likes(0)
      
      Actions
      - Re: Number of files in any given folder
        
        Jeff Sweeney Jan 2, 2014 12:04 PM (in response to Pete Yodis)
        
        Yup. A Contributor or if none are available an Editor.
        
        Like Show 0 Likes(0)
        
        Actions
        
        Re: Number of files in any given folder
        
        Pete Yodis Jan 2, 2014 12:36 PM (in response to Jeff Sweeney)
        
        Figures...
        
        Like Show 0 Likes(0)
        
        Actions
- Re: Number of files in any given folder
  
  Pete Yodis Jan 3, 2014 1:49 PM (in response to Jeff Sweeney)
  
  Jeff,
  
  Any thoughts on how to automatically push the files into subfolders as they are getting created? This doesn't look easy...
  
  Like Show 0 Likes(0)
  
  Actions
  - Re: Number of files in any given folder
    
    Charley Saint Jan 3, 2014 2:15 PM (in response to Pete Yodis)
    
    Pete,
    
    You could use the same method you used here, just specify a sub-folder as part of the path:
    https://forum.solidworks.com/thread/76132?tstart=0
    
    Like Show 0 Likes(0)
    
    Actions
    - Re: Number of files in any given folder
      
      Pete Yodis Jan 3, 2014 2:20 PM (in response to Charley Saint)
      
      Yes, but that would require an input on 15,000 files. The other scenario only comes up infrequently. This scenario happens every single time... I was looking for something automatic, or I'll see how 15,000 files in one folder behaves. I don't have the cozy feeling about either at the moment.
      
      Like Show 0 Likes(0)
      
      Actions
      - Re: Number of files in any given folder
        
        Pete Yodis Jan 3, 2014 3:08 PM (in response to Pete Yodis)
        
        I created copies of pdf files until I had just shy of 20,000 files in a folder. It takes 15 seconds to open that folder on my computer. Search is really fast as noted...
        
        Having a custom column set increased the folder open time to about 19-20 seconds.
        
        Like Show 0 Likes(0)
        
        Actions
        
        Re: Number of files in any given folder
        
        Pete Yodis Jan 3, 2014 4:39 PM (in response to Pete Yodis)
        
        I should mention, that a large number of the PDF files are old scanned hand drawings that are still used. In that case, there is no convert task to run to place those files in a proper subfolder. That's what you get in a 76 year old company. A macro wouldn't conver this scenario.
        
        Like Show 0 Likes(0)
        
        Actions
      - Re: Number of files in any given folder
        
        Jeff Sweeney Jan 3, 2014 4:16 PM (in response to Pete Yodis)
        
        What type of input? You can make those tasks run completely silent if they have all the information they need.
        
        If they don't have all the information they need, I'd investigating writing a little macro or task to do it. The ROI on something like that would be pretty quick if moving that many files around.
        
        Like Show 0 Likes(0)
        
        Actions
  - Re: Number of files in any given folder
    
    Adrian Velazquez Jan 3, 2014 3:30 PM (in response to Pete Yodis)
    
    Somewhat off-topic, Have you tried using Detached Drawings instead of generating PDF?
    
    Like Show 0 Likes(0)
    
    Actions
    - Re: Number of files in any given folder
      
      Pete Yodis Jan 3, 2014 3:35 PM (in response to Adrian Velazquez)
      
      purchasing wants to be able to send the PDF files out for quote, so detached drawings won't suffice. Also, we have some large assembly drawings that the shop views. Edrawings gets absolutely crushed on these large drawing files. PDF is so much more lean for shop floor computers.
      
      Like Show 0 Likes(0)
      
      Actions
      - Re: Number of files in any given folder
        
        Adrian Velazquez Jan 3, 2014 3:39 PM (in response to Pete Yodis)
        
        I see, we have similar scenarios where PDF are required, we just create them on demand and do not manage them. The Detached drawing is the the Release Drawing (PDF would be created from the Detached if necessary).
        
        Like Show 0 Likes(0)
        
        Actions
        
        Re: Number of files in any given folder
        
        Pete Yodis Jan 3, 2014 3:48 PM (in response to Adrian Velazquez)
        
        For the shop floor, we never know which drawing they may want to view and when they want to view it, so we have all released drawings converted to PDF at the time of release. We want to manage these files in the vault, as they are the drawing of record, now.
        
        Like Show 0 Likes(0)
        
        Actions

Go to original post