Linus OlofssonandClaude Opus 5.5
f452d6d237
- astro.config: site, trailingSlash "never", build.format "file". - /sitemap-index.xml + /sitemap-0.xml as prerendered endpoints (no new dependency); pages are derived from src/pages/*.astro, 404 excluded. - robots.txt: all bots allowed, AI crawlers/training bots explicitly allowed (Linus's decision), /download/ disallowed, sitemap line. - /llms.txt generated from business.ts, incl. usage-with-attribution note. - 404.astro in Swedish with links back, noindex. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WvE3C43Uwtv2vtnoFBLDw7
42 lines
1.2 KiB
Plaintext
42 lines
1.2 KiB
Plaintext
# robots.txt – https://imagesbyolofsson.se
|
||
#
|
||
# Policy (beslut av Linus 2026-10-27): ALLA botar är välkomna, även AI-crawlers
|
||
# och träningsbotar. Innehållet får användas av AI-assistenter med hänvisning
|
||
# till Images by Olofsson (se /llms.txt). Cloudflare-sidan (AI Crawl Control /
|
||
# "Block AI bots" / hanterad robots.txt) ställs in separat så att den inte
|
||
# blockerar eller skriver över detta.
|
||
|
||
User-agent: *
|
||
Allow: /
|
||
# Lösenordsskyddad nedladdning av originalfiler – inget att indexera.
|
||
Disallow: /download/
|
||
|
||
# AI-crawlers och träningsbotar: uttryckligen tillåtna.
|
||
User-agent: GPTBot
|
||
User-agent: OAI-SearchBot
|
||
User-agent: ChatGPT-User
|
||
User-agent: ClaudeBot
|
||
User-agent: Claude-User
|
||
User-agent: Claude-SearchBot
|
||
User-agent: anthropic-ai
|
||
User-agent: CCBot
|
||
User-agent: Google-Extended
|
||
User-agent: Applebot
|
||
User-agent: Applebot-Extended
|
||
User-agent: Bytespider
|
||
User-agent: Amazonbot
|
||
User-agent: meta-externalagent
|
||
User-agent: meta-externalfetcher
|
||
User-agent: FacebookBot
|
||
User-agent: PerplexityBot
|
||
User-agent: Perplexity-User
|
||
User-agent: cohere-ai
|
||
User-agent: Diffbot
|
||
User-agent: YouBot
|
||
User-agent: DuckAssistBot
|
||
User-agent: MistralAI-User
|
||
Allow: /
|
||
Disallow: /download/
|
||
|
||
Sitemap: https://imagesbyolofsson.se/sitemap-index.xml
|