• (the usual) take a high %  (or even 100%) of unique WITHOUT any referrer data (that also are not a bot)
• Make a “bookmark this” link, but make the bookmarked url something like “http://example.com/?camefrom=bookmark”. Then just do something like

• if ($_GET['camefrom'] == “bookmark”)
  {
  // +1 to your stats thing counting bookmarks (database? etc)
  }

If you want to do a cron job, but don’t want that email giving you the result, such as for example when doing a wget command, do this:

wget –quiet –output-document output_filename http://www.urltoget.com/

It will still email you with errors, so if you want it to never email you, add this:

>/dev/null 2>&1

to the end of it.

What is robots.txt

I’ll let Wikipedia explain:

The robots exclusion standard or robots.txt protocol is a convention to prevent cooperating web spiders and other web robots from accessing all or part of a website. The information specifying the parts that should not be accessed is specified in a file called robots.txt in the top-level directory of the website.

It is normally just found in your root directory, for example http://domain.com/robots.txt. Most robots will read it – some of the spammy robots or nasty ones won’t, and there isn’t much you can do about it (not related to robots.txt, anyway). You could use a htaccess or a script to block nasty bots, but that isn’t for this article.

How to block all robots from all of your pages

This is quite simple, however at this basic level I doubt you would want to use it much

User-agent: *
Disallow: /

User-agent: * includes all user agents (* is a wildcard – standing for anything, any length). There are many pages on the internet which will help you, such as this one with a massive list of user agents.

User agent info:
Some common search engine user agents that you may be interested in are as follows:

• Main web search (some also image search):

• Google: googlebot
• MSN: msnbot
• Yahoo: yahoo-slurp
• Ask: teoma
• Alexa: ia_archiver

• Google: googlebot-image
• MSN Pic search: psbot

So if you wanted to block just Alexa from all directories, you would use:

User-agent: ia_archiver
Disallow: /

How to block a certain directory (or more than one) using robots.txt

This isn’t very different to the previous example. This will block all user agents to /private/ and /cgi-bin/

User-agent: *
Disallow: /cgi-bin/
Disallow: /private/

Comments in robots.txt

Comments in robots.txt are simply written after a “#” sign, for example:

User-agent: * #this is a comment here
# another comment
#
# another one! robots.txt is so much fun…
Disallow: /temp/

Other things to add to robots.txt

There has been a Extended Standard for Robot Exclusion proposed, adding features such as request rate (# of requests per # of seconds) and visit time (time that the bot should visit, such as 0400-0800). Some bots will follow it, but you can’t count on it (at the moment, anyway) being too handy. Here is a basic example of it in action.

User-agent: *
Disallow: /temp/
Request-rate: 1/3
Visit-time: 0400-0800

Further Reading/Related Links

• Robots.txt Generator – a neat robots.txt generator
• Robots.txt blog on Webmaster world. Making very good use of its comments feature…

So you have to pretend it did… ;)

Use:

<a xhref="javascript:window.open('','_parent','');window.close();">Close Window</a>

Can’t select database

We were able to connect to the database server (which means your username and password is okay) but not able to select the xxxx database.

* Are you sure it exists?
* On some systems the name of your database is prefixed with your username, so it would be like username_wordpress. Could that be the problem?

If you don’t know how to setup a database you should contact your host. If all else fails you may find help at the WordPress Support Forums.

Well recently I needed to make sure that if oen of my wordpress installs couldn’t connect or whatever that it showed a different page (well, different text). So if you want to do the same (at your own risk), open up wp-includes/wp-db.php and edit:

• line 46 (connecting to server)
• line 66 (selecting the database)
• you may also want to edit the html from around line 312

Hope that helped you. Now digg me (just kidding – who wants this at the top of digg!) to prove that it works (although – of course my host will cope with it ;). They have before)

Some ideas to put on your messages:

• Tell them that something is wrong!
• Put a link to a Google cache (in the form of http://www.google.com/search?q=cache:http://www.example.com/)
• Try the coralize site – in the form of http://www.example.com.nyud.net:8080/
• Put a retry button/link

” ErrorDocument 404 /404.php”

That is it!

Making all links in your word press blog (esp links in comments) not have a rel=no-follow is quite simple. Just edit a couple of files:
Warning: Editing files can **** up your WordPress install. Only continue if you know what you are doing. Make backups!

• Open wp-includes/functions-formatting.php, search for “rel” and find all “nofollow”‘s and remove them. There are quite a few. Don’t just search and replace though, check them all one by one (About 5/6 in total)
• wp-includes\comment-functions.php – line 366 (for me, anyway). Remove the “nofollow” bit :)

This doesn’t mean you can just now spam away on this blog and get free backlinks… (they’re all moderated anyway).

This also removes rel=no-follow from trackbacks. Be carefull! Rel=no-follow was introduced for a reason…

What is mod_rewrite?
Mod_rewrite is a system for apache that lets you basically rewrite urls. It is quite advanced, while at the same time being quite simple.

Why is it so important?
It lets you change pages from something like viewnews.php?id=329579823948 to something like /viewnews/the-news-story-headline/. Of course, you will need to update your actuall links (from id=… to the new ones). But the reason it is so good is because firstly, who can remember 329579823948? And who can remember the-news-story-headline? Additionally, it is a lot better for seo to have keywords in the url, and most times your page title / etc will have the keywords in it.

Ok, walk me through a simple example using mod_rewrite
Ok. We are going to do a very simple mod_rewrite first, one that doesn’t involve any parameters (no id= or whatever).

For example, say at the moment our page is:
http://example.com/about_us.html

We want it to be
http://example.com/about/

Firstly you will need to be using apache. Make a file called .htaccess, and put it in your public html folder (same folder as index.php/index.html etc etc. In my case with my host, it would be /london-seo.com/.htaccess. Often it is something like public_html). You may have a bit of trouble making it in Windows. If you do, name the file “.htaccess“ (with the quotation marks).

Now you wanna put in .htaccess the following (first two lines are not always needed, depending on your server configuration):

Options +Indexes
Options +FollowSymlinks
RewriteEngine on
RewriteBase /
RewriteRule ^about/$ about_us.html [L]

See, its pretty easy. The ^ just means that represents the start of the string, and $ represents the end of the string. It is just regular expression stuff… You don’t need to worry about it too much for the time being. Note how the domain itself doesn’t matter (nothing about example.com is in there)

Now lets do a more complicated example. We want to change:
http://example.com/news.php?newsid=48672

to:
http://example.com/news/48672/

Well we would then put in the following code:

Options +Indexes
Options +FollowSymlinks
RewriteEngine on
RewriteBase /
RewriteRule ^news/(.*)/$ news.php?newsid=$1 [L]

The $1 indicates that it should replace whatever was in the first occurrence of (.*) (note – there can be more than one (.*).)

Well, that’s it for a basic mod_rewrite tutorial. Ideally you don’t wanna have id’s that relate just to a number, but you want some keywords. A good way to do this is to in your news.php only use the newsid=xx, but add in some extra, so you could have:

http://example.com/news/mod-rewrite-tutorial/89487/

The code for that would be:

Options +Indexes
Options +FollowSymlinks
RewriteEngine on
RewriteBase /
RewriteRule ^news/(.*)/(.*)/$ news.php?newsid=$2 [L]

Notice how there is no “$1“. As you get more advanced you will get used to this kinda stuff. Again, this is only a basic tutorial, there is a lot more to it, but this will get you started.