# ======================================================= # MfuguProject Robots.txt # Optimized for: Security, SEO, AIO # ======================================================= # --- 1. Aggressive Bot Blocking --- User-agent: MJ12bot User-agent: AhrefsBot User-agent: SemrushBot User-agent: DotBot User-agent: PetalBot Disallow: / # --- 2. AI & LLM Crawlers --- User-agent: GPTBot User-agent: Google-Extended User-agent: CCBot User-agent: anthropic-ai Allow: / # --- 3. General Search Engines --- User-agent: * Allow: / # Note: Sensitive paths are protected via Basic Auth & Meta Tags, not listed here. # --- Sitemap Location --- Sitemap: https://mfugu.com/sitemap.xml