How to Classify 6800 Links in Minutes with URLProfiler

 URLProfiler has been my secret weapon for the past few months and if you are serious about SEO it will be a part of your arsenal too. 

I was part of the beta testing group and have been in the lucky position to master this tool which has significantly improved several of my day to day processes for many different things. URLProfiler carries out numerous checks on a list of URL’s which really help speed up link classification – it is equally useful whether you are reverse engineering a competitor link building campaign, doing a Panda audit or simply auditing links due to a manual penalty.

In this post I will cover how you can use the tool to help you to audit a link profile and build up a contact list for link removals.

Adding URL’s to URLProfiler

When I conduct a link audit I start by pulling together as much  information as possible from all the different SEO tools I have access to. I combine all of this data into one single file and load into URLProfiler.

Importing your URL’s into URLProfiler is very easy and it accepts a lot of common file formats. You can import either a CSV or TXT file with the data you want to analyse. There is also the option to import files from common SEO tools such as Google Webmaster Tools, Moz, Ahrefs, Majestic SEO and Screaming Frog.

Simply right click on the URL list area and the option to paste a list of URL’s or import a file will show up.

Setting Up URLProfiler for the First Time

On the menu bar there is a settings tab, you want to choose “Speed”

Depending on the speed of your computer and internet connection you will need to work out the optimum number of threads for your machine, as an example I recently ran a check of 4000 links through the tool at 10 threads including whois checks in less than 12 minutes to grab some contact information the other day.

Now add in your API keys for Moz, ahrefs, MajesticSEO – you don’t need these in order for the tool to work. For PageRank and Whois data you will need an account with it is very cheap and works out at about $1 per 100,000 URL’s for PageRank checks and $1.63 per 1,000 whois lookups.

If you use the paid Moz API remember to select that in the drop down menu otherwise you will find your URLprofiler runs quite slow to throttle the number of API calls.


Choose your Level of Analysis

Once you have imported your URL’s you want to select the information you are interested in obtaining. As I am using the tool as a backlink checker I will add my site into the domain to check field at the bottom.


You also want to click on the buttons at the bottom and add in your keyword Brand terms and Money Terms.


Now you want to select the different sources you will need to help with the analysis.

  • MajesticSEO
  • MozScape
  • PageRank
  • Ahrefs
  • Email Address
  • HTTP Status
  • Robots Access

You can also select things such as Copyscape or Readability score if you believe the site you are auditing might have been scraped or have built links via spun content.

The URLprofiler will also return the contact us page of the website, the anchor text of the link to your site, nofollow/dofollow, anchor type and if the link is still live.


For this review I downloaded a sample of 6800 links from Google Webmaster tools for TripleSEO and ran it with the above settings – it returned the following fields:

  • URL
  • Domain
  • TLD
  • IP Address
  • Server Country
  • Domains Sharing
  • HTTP Status
  • Original URL
  • Original HTTP Status
  • Content Type
  • Content Length
  • Title
  • Link Status
  • Hash
  • Target URL
  • Anchor Text
  • Anchor Text Distribution
  • Anchor Type
  • Link Type
  • Link Location
  • NoFollow
  • Rel Author
  • Rel Me
  • Total Domain Links
  • Total Domain Links Percentage
  • Site Type
  • CMS
  • Domain PageRank
  • URL PageRank
  • About URL
  • About URL Emails
  • Contact URL
  • Contact URL Emails
  • Write for Us URL
  • Write for Us URL Emails
  • Domain Emails
  • Registered
  • Whois Error
  • Root Domain
  • Whois Email
  • Whois Expiry Date
  • Whois Created Date
  • Name Server One
  • Name Server Two
  • Whois Registrar
  • Robots File Allowed
  • Robots Meta
  • Robots HTTP Header
  • Canonical HTTP Header
  • Canonical Head

As this data contains contact information and Whois data of hundreds of webmasters I don’t really want to make it available for download but if you are interested in seeing the output contact me and I will be happy to send you an anonymised version.

Auditing your Links

Now that you have all this information it makes it much easier to categorise which links to look into first e.g. directory links with money anchors, followed blog comments with money anchors, sitewide footer/sidebar links with money anchors, non-branded author bio links.

If you do not have much experience with auditing unnatural links I wrote a post for AWR (my favourite rank checking software) earlier this year about classifying them.

As URL profiler returns contact information and whois data it makes it much easier to contact the webmasters concerned. You will often find that some of the sites (especially directories) are owned by the same people so grouping them together and filtering by email address really speeds up your process.

Another feature I like about URL profiler is that the domain field it returns is in the exact format you need for the disavow.txt file a little bit on concatenation and you are on your way.

Right now you can download a 7 day free trial version but with the amount of time you’ll save with this tool each year you may as well just cough up the £99 ;)


Link Removal Canned Responses

After you have spent what seems like an eternity tracking down contact information, sending numerous emails, follow ups and contact form messages to website owners asking them politely to remove links you soon find that the same types of responses keep coming back time and time again.

Of course replying to each email individually will soon become time consuming so below are some canned email responses you are welcome to use in your next campaign(*).

When they offer to sell you more directory links Continue reading

Are you Missing out on SlideShare Links?

You may have recently seen SlideShare’s nice little year end egobait as part of their Zeitgeist where they created pages for people to share their personal highlights of 2013.

Congratulations all you 1%ers


Anyway last year I did some work with a client who did a LOT of presenting at conferences and often uploaded his decks to Slideshare, he even uploaded slide decks of Seminars and Training events too. I think he had at least 100 different Slideshares on his account.

If you use Ahrefs or MajesticSEO for link analysis you will already know about the ability to analyse links of a specific URL structure.


When I threw his SlideShare ID up into Ahrefs he had over 500 links from 130+ different LRD’s either linking to his presentations directly, some were linking to his Slideshare profile and many more had embedded the slide decks on their sites.


I then ran a report to see which sites were already linking to my client and filtered to those who were not.


That’s 109 sites I have to go through who already have a relationship with my client and who actually thinks he’s interesting.

I then segmented the different sites into buckets, the high quality sites I pitched an interview with him or guest post from my client to expand on the presentation and the lower quality sites I simply asked if they would be happy to link to my client’s profile page on his website.

Conversion rates were good especially with the offers to interview him further about a recent presentation he had done.

So sometimes you don’t need to be scraping the ideas barrel trying to come up with some crazy d3.js HTML5 interactive game/quiz/visualization thing and just look at the assets you’ve already got.

How To Build An Online Business That Will Last

It’s quite rare for me to publish a guest post on my blog.

I got chatting to Lewis Ogden a couple of months back after he published an epic post for Matthew Woodward on his methods for finding new money making niches. It was a really solid post and I recommend you take a few minutes to read it some time soon.

So I decided to invite Lewis to share his thoughts on building an online business.

Over to you Lewis…

If you paid any attention to Google and SEO over the last 6 months of 2013, you would have witnessed a tidal wave of angry people, complaining about how Google screwed their business. Whilst sad that these people have lost a lot, and in some cases everything, they really should have been asking themselves these 3 questions;

  1. Does my business/website rely solely on Google?
  2. Do I deserve to rank #1 for this keyword?
  3. Is my website built for the user or search engines?

By answering these 3 questions honestly, you can quickly determine if you have a website that will stand the test of time, or will succumb to an Algorithm update 6 months or even 2 years from now.

Here are ten guidelines to follow to ensure any business you start in 2014 is one for the future.

1) User Experience over SEO

user experienceThere is more and more emphasis being placed on user engagement in the ranking factors.

Bounce rate, time on site, CTR all play a part in your overall SERP position and how Google see’s your site. Gone are the days of micro niche sites, 5 pages of crappy content plastered with affiliate links.

Today is the day for awesome content, engaging video and captivating images. Your site structure should make navigation a breeze for the user, it should not look like a maze of “silo links” designed to control link juice.

Your site speed is even more critical now than it ever was, you have less than a second to capture your readers eye, anymore than that and they are gone, never to return again. If your site loads like in the blink of an eye, you bounce rate will drop…simple.

Your content is the foundation of your business, not how may backlinks you have.

If your content is built for the reader, is helpful and provides what they are looking for, they will tell others.

If however, your content looks like a robot wrote it with, keywords bolded and italicized all over the place, you have no business. You have a game of cat and mouse…and guess who the cat is?

Continue reading