What is the benefit of having this be a standard? Can't an agent follow a guide just as easily in document with similar content in a different structure?
Primarily this being a predictable location for agents. AI not having to fetch the sitemap or llms.txt and then a bunch of subsequent queries saves a lot of time and tokens. There's an advantages section[1] within the proposal docs.
