Speaking as a former user (not as a SOLIDWORKS employee) with companies with which I have worked that didn't have a PDM in place, we created a tree structure based on the file naming conventions. In both places, the files were named according to their part numbers so they could be logically sorted. Using SOLIDWORKS Explorer, you can use the Move command to move the files from their current locations into their respective folders and update the Where used at the same time. This will be a laborious task depending on how many files you need to move but it should update all of the references and create a structure at the same time. You should probably find all of the duplicate files, where they are used, edit the references for each drawing or assembly and point it to a single version of the part, and then move (not delete, yet) the duplicates to a location that is not in your search paths. You should then open each assembly and drawing that you re-referenced and verify that they are pointing to the desired files and that all references are updated, fixing those that have broken.
If the files aren't named in a logical manner, things become much more difficult, especially if you have multiple files with the same name e.g. Plate.sldprt. you will have to do a piece-wise comparison to see which files you want to keep and determine some sort of tree structure for sorting them, maybe by project or machine.
If your company is able to upgrade to SOLIDWORKS Professional, you get WPDM (it's pretty easy to setup and use and is meant for smaller groups of users) included and that will help a lot going forward.
Hope this helps you some,
Thank you so much for the information. It is very useful and I was thinking I was going to have to use Solidworks explorer to move the files. Wish me luck I have a lot to do.