We use cookies to ensure that we give you the best experience on our website. If you continue to use this site we will assume that you agree to our cookie-policy

SEO ENQUIRY

First, we need your contact details…

SEO ENQUIRY

Which SEO services are you interested in?

SEO ENQUIRY

Anything you’d like to share with us at this stage?
A brief, technical scope, supporting documents or anything else?

SEO ENQUIRY

How soon are you looking to kick off this project?

SEO ENQUIRY

What budget guidance are you working on this project?

SEO ENQUIRY

What budget guidance are you working on this project?

THANK YOU

Thank you for getting in touch with Yellowball!

We’ve received everything and one of our team will be in touch very soon with the next steps. If your enquiry is urgent, you can call us on 020 8064 1832 – we’re open 9am – 6pm Monday to Friday.

BACK TO HOMEPAGE
YELLOWBALL LITE

Unfortunately, you don’t qualify for a bespoke project, but there is good news!

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

VISIT YELLOWBALL LITE

Step 1

of 6

WEB DESIGN ENQUIRY

First, we need your contact details…

WEB DESIGN ENQUIRY

Which web design services are you interested in?

WEB DESIGN ENQUIRY

Anything you’d like to share with us at this stage?
A brief, technical scope, supporting documents or anything else?

SEO ENQUIRY

How soon are you looking to kick off this project?

WEB DESIGN ENQUIRY

What budget guidance are you working on this project?

THANK YOU

Thank you for getting in touch with Yellowball!

We’ve received everything and one of our team will be in touch very soon with the next steps. If your enquiry is urgent, you can call us on 020 8064 1832 – we’re open 9am – 6pm Monday to Friday.

BACK TO HOMEPAGE
YELLOWBALL LITE

The end-to-end delivery of bespoke web project akin to the style and quality of our portfolio typically start from £25,000.

Based on your budget guidance, we’d recommend exploring Yellowball Lite – our flexible WordPress theme that is perfect for small businesses, startups, consultancies and charities. Learn more about Yellowball Lite and speak to one of our specialists on this dedicated site. It’s all the best from Yellowball, now fast, affordable and available anyone, anywhere.

VISIT YELLOWBALL LITE

Step 1

of 5

GRAPHIC DESIGN ENQUIRY

First, we need your contact details…

GRAPHIC DESIGN ENQUIRY

Which design services are you interested in?

GRAPHIC DESIGN ENQUIRY

Anything you’d like to share with us at this stage?
A brief, technical scope, supporting documents or anything else?

GRAPHIC DESIGN ENQUIRY

How soon are you looking to kick off this project?

GRAPHIC DESIGN ENQUIRY

What budget guidance are you working to on this project?

THANK YOU

Thank you for getting in touch with Yellowball!

We’ve received everything and one of our team will be in touch very soon with the next steps. If your enquiry is urgent, you can call us on 020 8064 1832 – we’re open 9am – 6pm Monday to Friday.

BACK TO HOMEPAGE
GRAPHIC DESIGN ENQUIRY

Most design projects at Yellowball are typically more than £500 and our agency-wide hourly rate is £145 plus VAT.

We’re probably not a good strategic fit for this project but you could try the freelancer networks at Fivver.com or upwork.com.

Step 1

of 5

PPC ENQUIRY

First, we need your contact details…

PPC ENQUIRY

Which PPC services are you interested in?

PPC ENQUIRY

Anything you’d like to share with us at this stage?
A brief, technical scope, supporting documents or anything else?

PPC ENQUIRY

How soon are you looking to kick off this project?

PPC ENQUIRY

What budget guidance are you working on this project?

PPC ENQUIRY

What budget guidance are you working on this project?

THANK YOU

Thank you for getting in touch with Yellowball!

We’ve received everything and one of our team will be in touch very soon with the next steps. If your enquiry is urgent, you can call us on 020 8064 1832 – we’re open 9am – 6pm Monday to Friday.

BACK TO HOMEPAGE
YELLOWBALL LITE

Unfortunately, you don’t qualify for a bespoke project, but there is good news!

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

VISIT YELLOWBALL LITE

Step 1

of 6

GOOGLE ANALYTICS ENQUIRY

First, we need your contact details...

GOOGLE ANALYTICS ENQUIRY

Which Google Analytics services are you interested in?

GOOGLE ANALYTICS ENQUIRY

Anything you'd like to share with us at this stage?

GOOGLE ANALYTICS ENQUIRY

How soon are you looking to kick off this project?

GOOGLE ANALYTICS ENQUIRY

What budget guidance are you working to on this project?

THANK YOU

Thank you for getting in touch with Yellowball!

We’ve received everything and one of our team will be in touch very soon with the next steps. If your enquiry is urgent, you can call us on 020 8064 1832 – we’re open 9am – 6pm Monday to Friday.

BACK TO HOMEPAGE

Step 1

of 5

Mobile Menu
Mobile Menu

Award-winning work, focussed on results

Yellowball
Web Design

Air X

Yellowball
Web Design

City & Guilds

Yellowball
Web Design

Ballet with Isabella

Yellowball
Web Design

Tomatin

SEE All PROJECTS

Bot.

If you’re interested in SEO, you’ve likely come across the term ‘bots’ – otherwise known as spiders or crawlers. But what are they? Although there are a variety of bots out there, the most important ones for us SEO types are search engine bots. These bots travel the internet’s huge network of links, to analyse and index the many billions of pages of web content in a process called crawling. These bots will then work as librarians of sorts, categorising the web pages in massive data centres and delivering the most relevant pages to you as and when they are needed.

As we’ve mentioned, there are more than just search engine bots when it comes to SEO. Black hat SEO tactics often involve the use of bots to scrape content and spam links, but more on that later.

How do bots crawl the web?

So, how do search engine bots actually work? They use algorithms to determine which sites they need to crawl, how frequently they should crawl them, and how many pages on a site need to be crawled – that’s a lot of crawling. Once they’ve crawled the links and arrived at a certain web page, the bots will read the content of the page to try to understand it and work out for which search terms that page should be displayed. They do this by reading any text, tags and alt attributes on a page, however, they are unable to read any rich media, such as video or audio files.

Contrary to what some people believe, Google does not allow websites to pay to have their sites (nor any specific web pages) crawled deeper or more frequently. Websites can request a crawl, otherwise known as a ‘fetch’ via Google’s search console but again this is at the discretion of Google. As a rule of thumb the more popular your website the more regularly it will be crawled by bots.

What you want to ensure is that when these bots do come to crawl your site, however regularly that may be, that it is as easy as possible for these digital spiders to read your site. Make sure you have a clear sitemap and information architecture. Help bots move throughout your site by providing useful internal linking and structure pages so that they are easy to understand. Remember that focussing on relevance and value for user will probably result in all of these things happening anyway.

Robots.txt

SEO is all about relevance, so you want the search engine bots arriving at your site to read and index the most relevant content for your users, sometimes that requires you to give them a little nudge on where, or where not, to look. This is where robots.txt comes in.

robots.txt is a text file inserted into a website’s code that gives instructions to the bots that arrive at your website. Using your robots.txt file incorrectly can have major negative consequences for your website, theoretically it would be possible to use the code to have Google completely de-index you from their searches. But don’t worry, when used correctly, robots.txt can be one of the most important tools in your SEO arsenal.

You can use robots.txt beneficially for:

  • Adding an XML sitemap to your site, making it easier for bots to crawl
  • Preventing duplicate content appearing on SERPs
  • Keeping areas of your website private
  • Keeping internal search result pages from appearing on SERPs
  • Preventing search engines from indexing certain files, such as images, PDFs, etc.
  • Specifying crawl delays to prevent servers becoming overloaded when crawlers load up multiple pieces of content at once

To check if you have robots.txt running on your website, add it to the end of your URL like so:

examplesite.com/robots.txt

If you don’t have a robots.txt running, Google have produced this easy to understand guide to setting it up.

What are malicious bots?

As we’ve mentioned, while conducting SEO on your site, the most common and most important bots you’ll encounter will be search engine bots. However, they’re not the only bots out there. Black hat SEO techniques often make use of more nefarious bots to spam, scrape and hack your site, potentially seriously damaging how well your page ranks in search engines. Contrary to popular belief, there are very few manual cyber-attacks carried out by ‘hackers’ in their basements, in fact around 90% of cyber-threats are bot-based, so it’s important for webmasters to be aware of the threat.

So what do malicious bots do to your site? The most prominent methods used by bots to disrupt your SEO campaigns are spamming, scraping and hacking. Spam bots will attempt to generate as many links as possible across comments on thousands of websites. However, thanks to Google’s anti-spam campaigns, only around 0.5% of web traffic comes from spam bots. Fortunately, the link farming used by spam bots is likely to do considerably more damage to the website they link to rather than any sites that they leave comments on. An easy way to combat comment spam is to make sure that you have a reCaptcha installed in the comments section.

Scrapers, on the other hand, can be a real problem for webmasters. Scraper bots lift content from your own site and duplicate it elsewhere. The main issue with this is that the plagiarised content may lead to Google punishing your site for having duplicated content, even though it was duplicated from you in the first place! There is no direct way to combat this except to make sure that your website is popular enough to be crawled regularly – in this way Google will likely see your content published on your website first, prior to these nasty scrapers. Furthermore, trust that scraper website are usually low quality and therefore less trustworthy than yours!

Finally, hacking bots can inject malware into your website’s code that you likely won’t even notice. However, if Google detects that there is malware on your site, they will warn users that your site may be infected, leading to a slow but sure decrease in ranking. If malware remains on your site, it stands to reason that both the search engines and potential customers will lose confidence.

Reportedly, over a quarter of all web traffic is made up of malicious bots, so knowing how to protect yourself from them is important for any webmaster. This article from Moz is a great guide on how to fight back and protect yourself from the major methods used by malicious bots. However, it boils down to this: Be proactive and keep an eye on your site, utilise the best anti-spam programs, and always take action against malicious bots as soon as possible.

Looking for an SEO agency?

Our team of SEOs run fully managed campaigns for a wide variety of clients, focussed on growth and sustainability.

Learn more about our SEO service