AnsweredAssumed Answered

Number of files in any given folder

Question asked by Pete Yodis on Jan 2, 2014
Latest reply on Jun 1, 2020 by LIN SHAODUN

Like • Show 0 Likes0
Comment • 22

I've seen the question for how many files per folder come up a few times here in this forum. The 2,000 number seems to get thrown out often. We desire to have all of our drawings converted to PDFs and located under one folder for easy access by the rest of the company. We will have around 23,000 PDF files once we migrate to EPDM. We are running EPDM 2013 (we will be at 2014 when we go live) on a Windows Server 2008 R2 (We will be upgrade to Windows Server 2012 when we go live) machine with 32GB of RAM, an 8 core CPU running at 2.2 GHz, storeage for files is on a RAID array with 15,000 RPM SCSI drives with plenty of storage at the moment (540 GB).

We are using the convert task to create PDFs for our SolidWorks drawings. The convert task will be setup to save PDFs to just one directory. Does anyone see this as an issue? Will 23,000 plus PDF files in one folder be a killer? Seems like the convert task would have to get a bit more complicated to place files in subfolders based on file name.

Jeff Sweeney Jan 2, 2014 10:23 AM

Correct Answer

Just playing around one day I put 44,500+ files in an EPDM directory. https://forum.solidworks.com/message/50412

Within reason it mostly boils down to browsing preformance.

That 2000 number is very outdated...back in the old XP days. Most people's threashold of pain on a good Windows 7 seems to be around 8000 files. Clients with slower machines will start to complain first.

If users use the search tool rather than browsing, the size isn't going to be much of an issue.

See the reply in context

No one else had this question

Outcomes

22 Replies

Pete Yodis Jan 2, 2014 9:50 AM
Mark Correct
Correct Answer
I should also mention, that shop users will probably use IE to browse/search the vault for the PDF drawing they are looking for. We are migrating from PDMWorks WorkGroup and have used the webportal for shop users to find drawings. The webportal with WorkGroup has a setting to create virtual folders for a folders with large numbers of documents in them. This was to speed up browsing in those large folders. Does the web access in EPDM have something similar?
Like • Show 0 Likes0
Actions
Lawrence Kiefer Jan 2, 2014 10:01 AM
Mark Correct
Correct Answer
Pete,

The standard reply of 2000 files per folder is what my reseller told me as well. We originally had the same question when we implemented EPDM, and we also use convert task to save PDF versions outside the vault for all of the departments outside of engineering to have acces to released drawings.

We have folders outside of the EPDM database with PDF files containing upwards of 5000 files currently, and growing.....and we do not have any issues.

I believe that the 2000 number applies to files stored in the database itself. My reseller basically told me we could store more files in one folder in the EPDM database, however it would start to slow down searching for file.....
I know back in the old days of using the fat and fat 32 file system in windows there were limits, but with the NTFS format windows claims no limits on files per folder.

Hope this helps...
Like • Show 0 Likes0
Actions
- Pete Yodis @ Lawrence Kiefer on Jan 2, 2014 10:11 AM
  Mark Correct
  Correct Answer
  Thanks Lawrence,
  
  We plan on storing the PDF files in the vault, so that we can keep the revision history of the PDF files. I just checked and there are about 15,000 plus PDF files...so my 23,000 number was high. I am tempted to just run with 15,000 PDF files in one folder named "Released PDF Drawings' stored in the vault, unless someone else here tells me this is a very bad idea. From an operations point of view, this is really easy. I would hate to have to sub-divide that folder into subfolders and get fancy with the PDF tasks...yuck.
  Like • Show 0 Likes0
  Actions
  - Lawrence Kiefer @ Pete Yodis on Jan 2, 2014 10:27 AM
    Mark Correct
    Correct Answer
    Pete,
    
    I don't think you will have any issues. I would do this, and simply keep adding files until you notice issues. Chances are, you wont have any. As far as keeping them in the vault for revision tracking, with the convert to PDF task you can set it up to add a suffix to the filename with the latest revision, so even if you are storing them outside of the vault the revision can be tracked. Of course there is no rollback or history, but I don't see that as a problem as the solidworks file itself in EPDM has that...My reasoning for storing the released PDF drawings outside of the vault was because the company I set EPDM up for is fairly large and has a large number of people that need access to released drawings outside or engineering. If I kept these files in the vault, the company would have to purchase enough seats of EPDM viewers to accommodate and as you know, that is not cheap.....
    
    Food for thought. Good luck.
    Like • Show 0 Likes0
    Actions
    - Pete Yodis @ Lawrence Kiefer on Jan 2, 2014 11:17 AM
      Mark Correct
      Correct Answer
      Lawrence,
      
      I can see your reasoning to limit the number of EDPM seats. That is why I mentioned the web viewer being used here. With WorkGroup, the web portal did not consume a license of the client software. I am not sure its the same with EPDM. I am curious to see how browsing a large folder with IE is going to behave.
      Like • Show 0 Likes0
      Actions
Jeff Sweeney Jan 2, 2014 10:23 AM
Unmark Correct
Correct Answer
Just playing around one day I put 44,500+ files in an EPDM directory. https://forum.solidworks.com/message/50412

Within reason it mostly boils down to browsing preformance.

That 2000 number is very outdated...back in the old XP days. Most people's threashold of pain on a good Windows 7 seems to be around 8000 files. Clients with slower machines will start to complain first.

If users use the search tool rather than browsing, the size isn't going to be much of an issue.
Like • Show 1 Like1
Actions
- Pete Yodis @ Jeff Sweeney on Jan 2, 2014 11:19 AM
  Mark Correct
  Correct Answer
  Thanks Jeff. We'll start off with around 15,000 files. Hmmm. Sounds like it may be a little slow. Any thoughts on the web viewer and it's performance?
  Like • Show 0 Likes0
  Actions
  - Jeff Sweeney @ Pete Yodis on Jan 2, 2014 11:58 AM
    Mark Correct
    Correct Answer
    Honestly never tried large folders with it. I bet it will be either pretty snappy because it only displays 30 files at a time or very slow.
    
    How's that for covering my bases?!
    Like • Show 0 Likes0
    Actions
    - Pete Yodis @ Jeff Sweeney on Jan 2, 2014 12:02 PM
      Mark Correct
      Correct Answer
      Nice... way to cover the range. Looks like I will have to try it. Does a user surfing the vault with IE, consume a license?
      Like • Show 0 Likes0
      Actions
      - Jeff Sweeney @ Pete Yodis on Jan 2, 2014 12:04 PM
        Mark Correct
        Correct Answer
        Yup. A Contributor or if none are available an Editor.
        Like • Show 0 Likes0
        Actions
        Pete Yodis @ Jeff Sweeney on Jan 2, 2014 12:36 PM
        Mark Correct
        Correct Answer
        Figures...
        Like • Show 0 Likes0
        Actions
- Pete Yodis @ Jeff Sweeney on Jan 3, 2014 1:49 PM
  Mark Correct
  Correct Answer
  Jeff,
  
  Any thoughts on how to automatically push the files into subfolders as they are getting created? This doesn't look easy...
  Like • Show 0 Likes0
  Actions
  - Charley Saint @ Pete Yodis on Jan 3, 2014 2:15 PM
    Mark Correct
    Correct Answer
    Pete,
    
    You could use the same method you used here, just specify a sub-folder as part of the path:
    https://forum.solidworks.com/thread/76132?tstart=0
    Like • Show 0 Likes0
    Actions
    - Pete Yodis @ Charley Saint on Jan 3, 2014 2:20 PM
      Mark Correct
      Correct Answer
      Yes, but that would require an input on 15,000 files. The other scenario only comes up infrequently. This scenario happens every single time... I was looking for something automatic, or I'll see how 15,000 files in one folder behaves. I don't have the cozy feeling about either at the moment.
      Like • Show 0 Likes0
      Actions
      - Pete Yodis @ Pete Yodis on Jan 3, 2014 3:08 PM
        Mark Correct
        Correct Answer
        I created copies of pdf files until I had just shy of 20,000 files in a folder. It takes 15 seconds to open that folder on my computer. Search is really fast as noted...
        
        Having a custom column set increased the folder open time to about 19-20 seconds.
        Like • Show 0 Likes0
        Actions
        Pete Yodis @ Pete Yodis on Jan 3, 2014 4:39 PM
        Mark Correct
        Correct Answer
        I should mention, that a large number of the PDF files are old scanned hand drawings that are still used. In that case, there is no convert task to run to place those files in a proper subfolder. That's what you get in a 76 year old company. A macro wouldn't conver this scenario.
        Like • Show 0 Likes0
        Actions
      - Jeff Sweeney @ Pete Yodis on Jan 3, 2014 4:16 PM
        Mark Correct
        Correct Answer
        What type of input? You can make those tasks run completely silent if they have all the information they need.
        
        If they don't have all the information they need, I'd investigating writing a little macro or task to do it. The ROI on something like that would be pretty quick if moving that many files around.
        Like • Show 0 Likes0
        Actions
  - Adrian Velazquez @ Pete Yodis on Jan 3, 2014 3:30 PM
    Mark Correct
    Correct Answer
    Somewhat off-topic, Have you tried using Detached Drawings instead of generating PDF?
    Like • Show 0 Likes0
    Actions
    - Pete Yodis @ Adrian Velazquez on Jan 3, 2014 3:35 PM
      Mark Correct
      Correct Answer
      purchasing wants to be able to send the PDF files out for quote, so detached drawings won't suffice. Also, we have some large assembly drawings that the shop views. Edrawings gets absolutely crushed on these large drawing files. PDF is so much more lean for shop floor computers.
      Like • Show 0 Likes0
      Actions
      - Adrian Velazquez @ Pete Yodis on Jan 3, 2014 3:39 PM
        Mark Correct
        Correct Answer
        I see, we have similar scenarios where PDF are required, we just create them on demand and do not manage them. The Detached drawing is the the Release Drawing (PDF would be created from the Detached if necessary).
        Like • Show 0 Likes0
        Actions
        Pete Yodis @ Adrian Velazquez on Jan 3, 2014 3:48 PM
        Mark Correct
        Correct Answer
        For the shop floor, we never know which drawing they may want to view and when they want to view it, so we have all released drawings converted to PDF at the time of release. We want to manage these files in the vault, as they are the drawing of record, now.
        Like • Show 0 Likes0
        Actions
LIN SHAODUN Jun 1, 2020 4:19 AM
Mark Correct
Correct Answer
An interesting test result:
Recommendation for number of files per folder
Like • Show 0 Likes0
Actions

Number of files in any given folder

Outcomes

Related Content

Recommended Content