Like someone else said, folders marked private or confidential are going to draw the attention of someone who is naturally nosy.
If you computers are networked is there a way you can set up different drives on the network and limit access to those drive? We've done that here. There is one drive that only myself, the Controller, the Exec VP and President can access. This was done after we realized that if our superintendents could dial in and access the common drive on the network, others could too.
I'd definately mention this to your boss. No telling what else this person has seen and if they are leaving you don't know who else she will tell. Look into limited access via additional network drives and suggest that to your boss. Not every department needs to access the same information so a drive could be set up for confidential info with limited access and another set up for common use.
Michelle
