I'm running a large mail server. Should I set split_spool_directory to improve performance?

Splitting the spool directory has most benefit if there are times when there are a large number of messages on the queue. If all mail is delivered very quickly, and the queue is always less than, say, a few hundred messages, there isn't any need to do this. With larger queues, there is a definite performance benefit to splitting the spool. It shows up earlier on some types of file system, compared with others. Exim was not designed for handling large queues. If you are in an enviroment where lots of messages remain on the queue for long periods of time, consider implementing a back up host to which you pass these messages, so that the main host's queue remains short. You can use fallback_hosts to do this, or a router that is conditional on $message_age.