User-agent: SemrushBot
Disallow: /
User-agent: *
Disallow: /dblink/
Disallow: /tw/api/
Disallow: /tw/auto/
Disallow: /tw/include/