# Default rules for all crawlers User-agent: * Allow: / # Facebook crawler User-agent: facebookexternalhit Allow: / # Twitter/X crawler User-agent: Twitterbot Allow: /