[Topic has been merged into this relevant Topic]
Hi all, I need help with whitelisting crawler for outbrain.
Outbrain.com is a content syndication service. They recommend your content on other websites. Before they do, they have their crawlers go through their website.
That’s where the problems start for me. I can’t progress with outbrain without having the crawlers look at my site.
I emailed them asking, and they just pointed me to their FAQ page, step 3, whitelist their crawlers.
http://help.outbrain.com/customer/portal/articles/1461551?b_id=1524
I have a feeling is it BP security that is blocking the crawlers. So, how do I do whitelist them? I understand it’s something to do with editing the secure htaccess file. But which one, and how?
I’ve never had a problem with crawlers from search engines like Google or even Facebook crawlers. So I’m pretty surprised that I have a problem with this.
Thanks for any and all help!
- This reply was modified 5 hours, 16 minutes ago by AITpro Admin.
- This reply was modified 5 hours, 15 minutes ago by AITpro Admin.