AnsweredAssumed Answered

Recommendation for number of files per folder

Question asked by Lucas Dexter on May 12, 2010
Latest reply on May 18, 2010 by 1-I7AP5K

Like • Show 0 Likes0
Comment • 8

Hello,

Does anyone out there have a recommendation on the maximum number of files you can have per folder in the vault before seeing a performance hit? I heard somewhere the rule of thumb is 2,000 but I want to confirm. I am implementing EPDM at a new site and the file to folder ratio has come up several times with the implementation team.

Also, what is the recommendation for the number of sub folders per folder? Engineers go crazy with folders (I do not allow them to create them in the vault) so I want to make sure we are not creating too many folders at the new site from the start.

Thanks,

Lucas

Jason Capriotti May 12, 2010 10:15 AM

Correct Answer

For our PDF files, we a series of 9 folders with sub folders for each drawing number.....about 80,000 total. I recorded the time to open each folder containing the sub folders.

Folder Name	Number of sub folders	Time in seconds to open
Release 1	24892	55
Release 2	3755	6
Release 3	1712	3
Release 4	3077	4
Release 5	1664	3
Release 6	2883	4
Release 7	3789	5
Release 8	1268	3
Release 9	376	3

This probably depends on how many people are accessing these folders as well. Since these are PDFs that are automatically placed in the folder by a program, the users never browse there, they just search on the files and open them to view.

The average is about 550 files per second on our system running 2009 sp4.

See the reply in context

No one else had this question

Outcomes

Einar Gulbrandsen May 12, 2010 11:01 AM

XP64 BIT, pdm 2010 sp2.1:

We have a couple folders with more than 2000 files in it. On folder has 3500 files and takes about 40 sec. to open. I think that a folder max should be open in 10-15 sec so would prefere not more than 2000 files in a folder.

WIN7 64 BIT, PDM 2010SP 2.1

Same folder ,about 30 sec

EDIT: update with OS

Actions

Jeff Sweeney May 12, 2010 9:14 AM

It seems to depend on the operating system. Windows 7 seems to be quite a bit more efficient than XP and Vista.

As a test, I've put 44,500 files in a directory... https://forum.solidworks.com/message/50412 It was a bit painful.

So depending on your client machines 2-3,000 max is what I say.

Actions

Jason Capriotti May 12, 2010 10:15 AM

For our PDF files, we a series of 9 folders with sub folders for each drawing number.....about 80,000 total. I recorded the time to open each folder containing the sub folders.

Folder Name	Number of sub folders	Time in seconds to open
Release 1	24892	55
Release 2	3755	6
Release 3	1712	3
Release 4	3077	4
Release 5	1664	3
Release 6	2883	4
Release 7	3789	5
Release 8	1268	3
Release 9	376	3

The average is about 550 files per second on our system running 2009 sp4.

Actions

Jason Capriotti @ Jason Capriotti on May 12, 2010 10:28 AM

I ran this on our test machine and it was about 1.5x slower but I don't know if that's because its running Enterprise 2010 sp3 or because its an older server with less memory.

Test Machine:

2x 3ghz Xeon DP 5050

4 gb ram (DDR2)

Windows Server 2003 sp2 (32bit)

Production Machine:

1x 2.53ghz Xeon E5540 (4 cores, shows 8 probably from the hyper-threading option)

12 gb ram (DDR3)

Windows Server 2003 sp2 (64bit)

Actions

1-I7AP5K @ Jason Capriotti on May 18, 2010 12:35 PM

Are you guys talking about the time to get a directory listing? Do you have thumbnails turned on?

I am running SW2007 x64. My primary directory has 6,268 files and I get a directory listing across the network as fast as the display can refresh (1 second or less). Of course, I have thumbnails turned off, which makes a huge difference.

Actions

Ben Kalinowski May 12, 2010 3:52 PM

We've got one folder in particular with more than 8,000 files in and it takes about 10 sec to open and our users work in this foler all day. I have another folder with 17,000 files in it and it takes a little longer.

For us, this is how it needs to be. What I train my users to do is use the search tool. I created a small pdmsearch.bat file to put on their desktops (got the idea and / flag for it from another post on this forum). Any file in the vault can found in less than 3 seconds if you know something about it found on the data card. Using the search tool makes the browsing time irrelevant and it really leverages the power of pdm since it is a database. Plus in PDM you don't want alot of folders anyway, so fewer bigger folder become the norm.

So, IMHO, don't get too hung up on access times - use your search tool.

Good luck,

Ben

Actions

Lucas Dexter May 13, 2010 1:52 PM

Thanks for all the great replies everyone; access times is great information to have. Does anyone know if the larger folder sizes has any ill effects when performing upgrades?

Lucas

Actions

Jeff Sweeney @ Lucas Dexter on May 18, 2010 11:29 AM

I can't imagine any way local cache folder size will effect upgrade preformance.

Actions

8 Replies

Einar Gulbrandsen May 12, 2010 11:01 AM
Mark Correct
Correct Answer
XP64 BIT, pdm 2010 sp2.1:
We have a couple folders with more than 2000 files in it. On folder has 3500 files and takes about 40 sec. to open. I think that a folder max should be open in 10-15 sec so would prefere not more than 2000 files in a folder.

WIN7 64 BIT, PDM 2010SP 2.1

Same folder ,about 30 sec

EDIT: update with OS
Like • Show 0 Likes0
Actions
Jeff Sweeney May 12, 2010 9:14 AM
Mark Correct
Correct Answer
It seems to depend on the operating system. Windows 7 seems to be quite a bit more efficient than XP and Vista.

As a test, I've put 44,500 files in a directory... https://forum.solidworks.com/message/50412 It was a bit painful.

So depending on your client machines 2-3,000 max is what I say.
Like • Show 0 Likes0
Actions
Jason Capriotti May 12, 2010 10:15 AM
Unmark Correct
Correct Answer
For our PDF files, we a series of 9 folders with sub folders for each drawing number.....about 80,000 total. I recorded the time to open each folder containing the sub folders.
Folder Name Number of sub folders Time in seconds to open
Release 1 24892 55
Release 2
3755 6
Release 3 1712 3
Release 4 3077 4
Release 5 1664 3
Release 6 2883 4
Release 7 3789 5
Release 8 1268 3
Release 9 376 3

This probably depends on how many people are accessing these folders as well. Since these are PDFs that are automatically placed in the folder by a program, the users never browse there, they just search on the files and open them to view.

The average is about 550 files per second on our system running 2009 sp4.
Like • Show 0 Likes0
Actions
- Jason Capriotti @ Jason Capriotti on May 12, 2010 10:28 AM
  Mark Correct
  Correct Answer
  I ran this on our test machine and it was about 1.5x slower but I don't know if that's because its running Enterprise 2010 sp3 or because its an older server with less memory.
  
  Test Machine:
  2x 3ghz Xeon DP 5050
  4 gb ram (DDR2)
  Windows Server 2003 sp2 (32bit)
  
  Production Machine:
  1x 2.53ghz Xeon E5540 (4 cores, shows 8 probably from the hyper-threading option)
  12 gb ram (DDR3)
  Windows Server 2003 sp2 (64bit)
  Like • Show 0 Likes0
  Actions
- 1-I7AP5K @ Jason Capriotti on May 18, 2010 12:35 PM
  Mark Correct
  Correct Answer
  Are you guys talking about the time to get a directory listing? Do you have thumbnails turned on?
  
  I am running SW2007 x64. My primary directory has 6,268 files and I get a directory listing across the network as fast as the display can refresh (1 second or less). Of course, I have thumbnails turned off, which makes a huge difference.
  Like • Show 0 Likes0
  Actions
Ben Kalinowski May 12, 2010 3:52 PM
Mark Correct
Correct Answer
Hi

We've got one folder in particular with more than 8,000 files in and it takes about 10 sec to open and our users work in this foler all day. I have another folder with 17,000 files in it and it takes a little longer.

For us, this is how it needs to be. What I train my users to do is use the search tool. I created a small pdmsearch.bat file to put on their desktops (got the idea and / flag for it from another post on this forum). Any file in the vault can found in less than 3 seconds if you know something about it found on the data card. Using the search tool makes the browsing time irrelevant and it really leverages the power of pdm since it is a database. Plus in PDM you don't want alot of folders anyway, so fewer bigger folder become the norm.

So, IMHO, don't get too hung up on access times - use your search tool.

Good luck,

Ben
Like • Show 0 Likes0
Actions
Lucas Dexter May 13, 2010 1:52 PM
Mark Correct
Correct Answer
Thanks for all the great replies everyone; access times is great information to have. Does anyone know if the larger folder sizes has any ill effects when performing upgrades?

Lucas
Like • Show 0 Likes0
Actions
- Jeff Sweeney @ Lucas Dexter on May 18, 2010 11:29 AM
  Mark Correct
  Correct Answer
  I can't imagine any way local cache folder size will effect upgrade preformance.
  Like • Show 0 Likes0
  Actions

Recommendation for number of files per folder

Outcomes

Related Content

Recommended Content

Incoming Links