Close

Copyright © RebelMouse 2019

Follow Us:

Update Robots.txt in Layout & Design Tool

You can now easily allow or disallow user agents in Global Settings

Robots.txt can be updated from our Layout & Design tool. You can now easily allow or disallow user agents within Layout & Design tool's Global Settings as well.

What Is a Robots.txt File?

A robots.txt file tells search engine crawlers which pages or files the crawler can or can't request from your site. This is mainly used to avoid overloading your site with requests. It's not a mechanism for keeping a web page off of Google. To keep a web page off of Google, you should use noindex directives or protect your page with a password.

What Is Robots.txt Used For?

Robots.txt is primarily used to manage crawler traffic to your site, and usually to keep a page off of Google, depending on the file type.

If you have any questions about this feature, email support@rebelmouse.com.

RebelMouse CEO: Reimagining Diversity “Starts in the Office”

Why women, minorities, and the LGBTQ+ community will restructure workplace norms

Entering a new decade means a new opportunity to change how we approach diversity. In a joint interview alongside Leanne Pittsford of Lesbians Who Tech, RebelMouse founder and CEO Andrea Breanna said that the only way to fight patriarchal dominance is to encourage and nurture more women, minorities, and members of the LGBTQ+ community in the workplace, particularly in tech.

Over half of RebelMouse's employees are women or identify as LGBTQ. Read our full diversity and inclusion report here.

Check out the full video below.

Related Articles

Publishers No Longer Have to Submit Their Site to Google News

Google's Publisher Center creates new opportunities for audience growth

Just before the start of the next decade, Google announced an important change to its Google News offering with the launch of Google Publisher Center. The new interface merges Google News Producer and Google News Publisher into one to streamline the partnership process for publishers.

Overall, the change should make it easier for publishers to manage their Google News settings, including updating themes, directing URLs to section pages, and configuring user permissions. Read the full list of features here.

Keep reading...

How to Find Work-Life Balance as a Remote Employee

Tips from our CEO on making the most of an office-free lifestyle

Working from home is becoming increasingly popular, with an estimated 66% of companies now allowing remote work and 16% operating completely office-free. RebelMouse is one of those fully remote companies, and over the years we've mastered how to stay close to each other despite being spread across more than a dozen countries. We believe working remotely is good for both our personal lives and our productivity. Read more about this here.

Still, working free from the shackles of an office environment doesn't mean every day is a dance party in your pajamas from 9 to 5. Working from home comes with its own set of challenges just like any other job.

Keep reading...
Subscribe to Our Newsletter