# robots.txt for Smilepayz User-agent: * Allow: / # Disallow admin panel Disallow: /admin/ # Disallow API endpoints Disallow: /api/ # Sitemap location Sitemap: https://www.smilepayz.com/sitemap.xml # 文档入口由响应头和页面 meta 标记 noindex;允许抓取才能让搜索引擎看到该指令。 # 分享机器人仍不能抓取文档资源。 User-agent: Twitterbot User-agent: facebookexternalhit User-agent: Facebot User-agent: meta-externalfetcher User-agent: TelegramBot User-agent: WhatsApp User-agent: LinkedInBot User-agent: Slackbot-LinkExpanding User-agent: Discordbot Disallow: /doc/content/ Disallow: /admin/ Disallow: /api/ Allow: /