Can you elaborate on the need? What possible reason, short of a very contrived one, is there for having to keep a large number of machines' clocks in-sync?
And for that matter, why would anyone build any process/system/software that requires a distributed system's machines to all have their clocks in-sync. I am baffled.
Well, it's a reasonable solution to some distributed problems. I imagine that many distributed algorithms can be simplified a lot of you have a reliable and accurate time source. If you can build a reliable clock at less cost (development and/or overhead) than a time-insensitive algorithm would cost, then why not do it?
Many security protocols use credentials that are only valid for a brief time. Here is a simple example of a problem caused by my server's time being more than 10 minutes off from the time on S3:
Yep, security is a big one. Authentication & encryption can behave very unpredictably if your times are not in sync. So, you might not be able to login to a server, or connect via HTTPS etc
People have mentioned distributed transactions and security, another area is synchronizing modeling with (hard or soft) real time inputs from separate hardware. There are a bunch of ways to get yourself tied up in knots once at least 2 physical bits of hardware are involved.
And for that matter, why would anyone build any process/system/software that requires a distributed system's machines to all have their clocks in-sync. I am baffled.