I have a dial-up machine, and I use the queue_smtp_domains option so that remote mail only goes out when I do a queue run. However, any email I send with an address [anything@aol.com](mailto:anything@aol.com) is returned within about 15 minutes saying retry time exceeded, and all addresses are affected.

You should be using queue_domains rather than queue_smtp_domains. With the latter, Exim is trying to route the addresses, which involves a DNS lookup. This is presumably timing out, causing a retry time to be set for the domain, and somehow a valid lookup never happened before the maximum retry time (default of 4 days) passed. Hence the bounce. The fact that it is aol.com is probably not relevant. You should probably also be using -qq to do your queue run rather than -q.