Web Design Preston

Use Robots.txt to disallow spiders

admin — Thu, 12 Nov 2009 15:44:44 +0000

Use Robots.txt to disallow spiders from specific pages or sections
Robots.txt is a file in your server which tells various search engine crawlers not to crawl or index specific parts of your site. It can tell certain search engines to ig‐nore certain pages, or tell all engines to ignore your site altogether. Even for op‐timization, you might want to hide certain parts of your site from search engines. For example, if your site has a “terms and conditions” page which is similar to most such pages on other sites and serves no search purpose, or you don’t want bots to crawl your cgi‐bin directory, or have any other directories or pages with duplicate content, you can use this file to tell search engines to ignore them:
A robots.txt file looks like this:
User-Agent: [Bot or Spider name]
Disallow: [File or Directory name]

A list of the names of various search engine spiders names:

Spider Name, Search Engine, Status

AbachoBOT, Abacho, -
Acoon, Acoon, -
AESOP_com_SpiderMan, Aesop, -
ah-ha.com crawler
Ah-ha, -
appie, Walhello, -
Arachnoidea, Euroseek, active
ArchitextSpider, Excite, inactive
Atomz, Atomz, -
DeepIndex, DeepIndex (www.en.deepindex.com), -
ESISmartSpider, Ttravel Finder, -
EZResult, EZResults, -
FAST-WebCrawler, AlltheWeb, active
Fido, PlanetSearch, -
Fluffy the spider, SearchHippo, active
Googlebot, Google, active
Gigabot, Gigablast, active
Gulliver, Northernlight, inactive
Gulper, Yuntis, active
HenryTheMiragoRobot, Mirago, -
ia_archiver, Alexa, active
KIT-Fireball/2.0, Fireball (German SE at www.fireball.de), -
LNSpiderguy, Lexis-Nexis, -
Lycos_Spider_(T-Rex), Lycos, inactive
MantraAgent, LookSmart, active
MSN, Microsoft Prototype Crawler, -
NationalDirectory-SuperSpider, National Directory, -
Nazilla, Websmostlinked, -
Openbot, Openfind, -
Openfind, piranhaShark Openfind, -
Scooter, AltaVista,active
Scrubby, Scrub The Web, active
Slurp/3.0, Inktomi, active
Tarantula, AltaVista, inactive
Teoma_agent1, Teoma, active
UK Searcher Spider, UKSearcher, -
WebCrawler, WebCrawler, -

Defining the User‐Agent field with an asterisk (*) disallows all crawlers from that part of your site. Leaving the disallow field empty means that spiders are free to access and index your entire site. No matter how you use it, the robots.txt file gives you more control over your site and its optimization.

Optimising your website for spiders Optimizing for Spiders Of course, there is more to SEO...

Use CSS to reduce file size

admin — Thu, 12 Nov 2009 15:36:45 +0000

Cascading Style Sheets (CSS) is a style sheet language which makes your design more flexible and separates your layout code from the content of each page. CSS enables web designers to easily implement site‐wide changes without going into each page or accidentally messing with their content. CSS also reduces the size of your pages and makes it easier for spiders to find your content – the most im‐portant thing on your page – easily and quickly.

This is standard practice in the web industry any experienced Web design company should be using this methology.

No related posts.

Use valid, error‐free source code

admin — Thu, 12 Nov 2009 15:35:00 +0000

Make sure that search engines don’t come up against badly coded pages. The World Wide Web Consortium (W3C) has established global standards for HTML and CSS in web pages. In addition to testing for cross‐browser functionality, test each page of your website for errors by using W3C’s validator.

All sites produced by webdesignpreston.org.uk will be checked and re-checked to ensure they are error free

No related posts.

Optimising your website for spiders

admin — Thu, 12 Nov 2009 15:33:43 +0000

Optimizing for Spiders
Of course, there is more to SEO than user experience. For instance, a common design mistake is the use of ‘splash pages’ – a home page or main entry page with only a large image with a link to an inner page. While users may find it ar‐resting or interesting, a splash page will hurt your site because it keeps you from using keyword‐rich text and links, and often redirects automatically to another page.

So in addition to making a site which is functional and easy to navigate, you should take a number of steps to optimize your site for search rankings:

Use valid, error‐free source code
Use CSS to reduce file size
Create a sitemap
Use Robots.txt to disallow spiders from specific pages or sections
Use 301 redirects for moved or deleted pages
Use either www or non‐www, not both
Use static URLs with keywords in the title

Use Robots.txt to disallow spiders Use Robots.txt to disallow spiders from specific pages or sections...
Optimising Each Page Optimizing Each Page In addition to optimizing your site design,...

Optimising Each Page

admin — Thu, 12 Nov 2009 13:37:45 +0000

Optimizing Each Page

In addition to optimizing your site design, there are a number of standard SEO practices that should be followed on each page of your website. Keep the follow‐ing tips in mind when creating a page:

- Each page should be optimized for 1‐2 keywords only. If you find it hard to narrow the content down to only a couple of keywords, consider dividing it into two or more pages.
- Every page should have at least 250 words of content, and the title should always contain keywords.

- Don’t embed images or content in flash – search engines can’t read it!
- Add ALT and TITLE tags to all images. This will help spiders understand what the image is about, help your pages turn up in image searches, and enable you to use some more keywords.
- Write meta descriptions for each page:
Meta tags, for SEO purposes, refer to the keywords and description tag. These are part of the header of your HTML document. The keywords tag, which is used to include misspellings and variations of a page’s primary key‐words, is not supported by major search engines and has no verifiable effect on rankings. However, the description tag is important for SEO and is dis‐played by some engines in search results.
The description tag contains a few sentences about the page, giving you the opportunity to use some important keywords, but remember this should be readable to users as well as search engines.

Optimising your website for spiders Optimizing for Spiders Of course, there is more to SEO...

Websites That WORK

admin — Wed, 11 Nov 2009 15:44:35 +0000

While the quality of your content is the most important feature of your website, the structure and usability of your website play a significant role in ensuring that search crawlers as well as visitors can access and understand information easily.

Building a user‐friendly website is crucial to building a website that is optimized for search engines. And since the entire point of SEO is to attract more visitors to your website, making a functional, easily useable website should be an obvious part of your traffic‐building efforts.

No related posts.

Usability in Design

admin — Wed, 11 Nov 2009 15:44:25 +0000

The internet is all about convenience. Visitors have short attention spans and deeply entrenched surfing habits, so your website should be designed to make their navigation and reading experience as easy as possible. For instance, most users expect to find the site logo on the top left of a website, and to be able to get to the homepage by clicking on it. Contrasting background colors and large images also make for an effortless browsing experience.

Usability is about functionality, so make sure that all your links are functional and that each page enables users to know where they are in your website and how to reach the parent or home page at all times. Your layout and color schemes should also be consistent; this will not only make navigation easy but also build recognition in your users’ minds. Create a custom error page (a 404 page) in the same layout, and test your entire site on all major browsers (Inter‐net Explorer, Firefox, Safari, Chrome etc).

No related posts.

Using a Content Management System

admin — Sat, 10 Oct 2009 14:09:21 +0000

Content Management or CMS system, allows users to easily update content on there websites and produce very complex looking sites with relative ease.
Over the past few years Web Content Management Systems (WCMS) like Wordpress, Joomla, b2evolution and ExpressionEngine have revolutionized the creation and design of sites by providing ready to use publishing platforms.

If your website has more than a few pages, a CMS can make your website easy to manage and update. Numerous blogs and websites of all sizes use such systems, which have some SEO advantages such as CSS design and make it easy to write meta descriptions and image alt tags.

However, CMSs usually create dynamic URLs and make it difficult to execute re‐directs. So finding the best CMS system for your SEO needs will most likely take some research and customization.

Here at webdesignpreston.org.uk we can create a custom content management suite to suit your every need or customise existing suites and allow you to publish your information quickly and easily in a format best suited for your clients.

No related posts.

Create a sitemap

admin — Thu, 13 Aug 2009 08:36:55 +0000

A sitemap represents the architecture of your website, clarifying the hierarchy of all your pages, starting with the home page. (See Pickaweb’s own site map as an example.) Even if your website is relatively small, creating a sitemap will give you a lot of SEO benefits: notably, it will make your site easy to crawl, help the flow of trust (PageRank) between your pages, and alert search engines to changes (new or removed pages) in your website. Site maps are easy to create and can be made in a number of formats, with XML being the most common and widely ac‐cepted. (You can find a free online site map generator here.)

No related posts.

Use 301 redirects for moved or deleted pages

admin — Sun, 12 Jul 2009 15:46:22 +0000

Use 301 redirects for moved or deleted pages

What happens when you remove or change the location of an indexed page or your entire site? Users may continue to find it as a result in Google’s database, but when they arrive, they won’t find anything. Alternatively, if your old page has accumulated high rankings or trust (PageRank) in search engines, deleting or removing it will mean that you’ve wasted all that hard work.

A 301 direct stands for ‘moved permanently’ and is a search‐friendly way of tell‐ing crawlers the location of your new page or domain. For HTML pages, a redi‐rect uses the .htaccess file on your website’s main folder. For a single page redi‐rect, you can use the following code:

Redirect 301 /oldpage.html http://www.yoursite.com/newpage.html

To move an entire site, use the following:
Redirect 301 /http://www.yournewsite.com/

No related posts.