https://aicrawltest.com 2026-05-29T14:57:34.468Z weekly 1 https://aicrawltest.com/blog 2026-05-29T14:57:34.468Z weekly 0.8 https://aicrawltest.com/about 2026-05-29T14:57:34.468Z monthly 0.5 https://aicrawltest.com/faq 2026-05-29T14:57:34.468Z monthly 0.6 https://aicrawltest.com/contact 2026-05-29T14:57:34.468Z monthly 0.4 https://aicrawltest.com/privacy 2026-05-29T14:57:34.468Z yearly 0.3 https://aicrawltest.com/blog/what-is-llms-txt 2025-04-01T00:00:00.000Z monthly 0.7 https://aicrawltest.com/blog/optimize-for-ai-search 2025-04-08T00:00:00.000Z monthly 0.7 https://aicrawltest.com/blog/robots-txt-ai-crawlers 2025-04-15T00:00:00.000Z monthly 0.7 https://aicrawltest.com/blog/json-ld-structured-data 2025-04-22T00:00:00.000Z monthly 0.7 https://aicrawltest.com/blog/ai-crawlability-guide-2025 2025-04-29T00:00:00.000Z monthly 0.7 https://aicrawltest.com/blog/per-page-markdown-files 2025-05-06T00:00:00.000Z monthly 0.7 https://aicrawltest.com/blog/link-headers-agent-discovery 2026-05-14T00:00:00.000Z monthly 0.7 https://aicrawltest.com/blog/mcp-server-card 2026-05-14T00:00:00.000Z monthly 0.7 https://aicrawltest.com/blog/api-oauth-agent-discovery 2026-05-14T00:00:00.000Z monthly 0.7