User-agent: *

User-Agent: Googlebot
Disallow: /wp-admin
Disallow: /redirect.php
Disallow: /users/register


User-agent: SemrushBot 
Disallow: / 

User-agent: MJ12bot 
Disallow: / 

User-agent: AhrefsBot 
Disallow: /