Shamir, Author at Satori Technologies

I dig this image post for the SEO beginners who do not well understand about Robots.txt side. Ok let’s get in to the topic to learn the basics of Robots.txt . It is a simple notepad file containing some manually written code to block from crawling the pages, images, directories and a whole website from Search Engine Bots.

A robots.txt file is used to restrict your entire site, pages, images and directories from search engine bots which crawls your website.

Robots.txt Syntax:

User-agent: This section is used on applying for specific search engine robots

Disallow/Allow: This section is used to specify which URL, Folder, Image Path and Directory to block

Bot Pattern to be used in User-agent section:

User-agent: * (This applies for all bots)

User-agent: Googlebot (This applies to Google Search Engine Bot)

User-agent: Bingbot (This applies to Bing Search Engine Bot)

Robots.txt Example Patterns:

To block the entire website

User-agent: *
Disallow: /

To allow indexing the whole website

User-agent: *
Disallow:

(OR)

User-agent: *
Allow: /

To block the specific folder

User-agent: *
Disallow: /foldername/

Disallow a folder from Googlebot but allow indexing a specific file in that folder

User-agent: Googlebot
Disallow: /foldername1/
Allow: /foldername1/specificfilename.html

To block a directory

User-agent: *
Disallow: /directoryname/

To block a page

User-agent: *
Disallow: /page-name.html

To block a specific image from Google images

User-agent: Googlebot – Image
Disallow: /images/imagename.jpg

To block all images from Google Images

User-agent: Googlebot – Image
Disallow: /

To block specific image file type

User-agent: Googlebot – Image
Disallow: /*.jpg$

To block all URLs which includes a “?” (Question Mark)

User-agent: *
Disallow: /*?

To block URLs that ends with .html

User-agent: *
Disallow: /*.html$

Robots Meta Tag:

To block the page from indexing using meta tag

(Or)

Adding Sitemap in Robots.txt File:

Use the below syntax to add a sitemap URL in Robots.txt file

Sitemap: http://www.domainname.com/sitemap.xml
Sitemap: http://www.domainname.com/image-sitemap.xml

Here are some best plugins & tools to create robots.txt file,

WordPress Plugins:

WP Robots TXT

WordPress SEO by Yoast

Robots.txt Generator Tools:

SEOBOOK Robots.txt File Generator

MCANERIN Robots.txt Code Generation Tool

Do you need to validate the syntax of your robots.txt file? Use the below tools,

Robots.txt Checker Tools:

Google Search Console

TechnicalSEO Robots.txt Checker

Advanced Link Building Strategies & Resources

Building links is the way to gain quality backlinks to your site and also it helps to increase your website traffic and ranking in SERPs. Getting a quality backlink is a challenging task and it will come in minutes if you analyze a link like an expert. I have compiled a quality resources & tips from the experts helping you to promote your website and build high quality links.

Are you looking to build authority links and also to improve website traffic?

These experts strategies will guide you to the right path on getting non-spammy links and also these resources will help you to avoid and recover from Google Penguin Algorithm.

What is Link Building
Content Marketing
Content Marketing Tools
Guest Blogging Guides
How to Get Links Using Roud-up Posts
Images Link Building
Email Outreach Techniques
Broken Link Building
Broken Link Building Tools
Competitor Backlink Analysis Techniques
Competitor Backlink Analysis Tools
Link Building Strategies
Popular Link Building Videos
Link Building Tools
Link Building Search Queries Collection
Link Building Blogs to Follow

What is Link Building?

Are you a newbie to SEO?

The below resources will help you to know about the basics of link building and it also help you to create a link building plan from scratch.

Beginners Guide to Link Building

What is Link Building?

Wikipedia’s Link Building Definition & Types

How to Rank in Search Engines?

Content Marketing

How to get quality link & Skyrocket your website traffic using Content Marketing?

Today content is playing a pivotal role in your SEO strategy. Creating and promoting quality content is a must one to build a natural links to your website. If you wanna build more links and improve traffic to your website check the advanced content marketing resources below.

QuickSprout’s Advanced Guide to Content Marketing

How to Write a Perfect Blog Post?

17 Advanced Method for Promoting Your New Blog Post

76 Ways to Drive Traffic to Your Blog Post

Link Building Case Study – Backlinko

Best Content Marketing Tools Collection

100 Awesome Content Marketing Tools

Guest Blogging Guides

Now Big G is not interested on the links generated using guest blogging but if you do it in a right way with informative content it is one of the best way to improve your backlinks and branding. Check out the fantastic advanced guest blogging resources in this section.

Advanced Guide to Guest Posting in 2020 – QuickSprout

Guest Blogging for SEO: How to Build High Quality Links at Scale

Definitive Guide to Guest Blogging – Backlinko

Advanced Guest Blogging Guide – Moz

How to build quality links using round-up posts?

Round-up posts are a unique and easy way to make your content viral. Richard Marriot (Founder of Clambr.com) is an expert in this technique and he generated more high quality links to his round-up posts. Learn more about this technique from the couple of best resources.

How to Rank on Google Using Expert Roundup Posts

Ultimate Guide on Creating Experts Roundup Posts

How to get links using Images?

Getting links through images is one of the prominent & best way to improve branding and acquiring natural links to your website. There are so many blogs made discussion about image link building (like build links through Infographic) but I have furnished few of the most authoritative posts which will be useful to know more about this topic.

Images Link Building – Kaiserthesage

Creative Ways to Use Images for Link Building

Top 5 Ways to Promote Your Infographic to go Viral

How to Get Backlinks with Guestographics?

Best Email Outreach Techniques to gain quality links

Email outreach is a backbone of link building and it is an old strategy. If you want to gain more subscribers for your blog and quality links, this method is a must have in your link building bag. Learn how to craft a lead converting email from the below outstanding resources.

Complete Email Outreach Guide: 52 Curated Resources

The Definitive Guide to Link Building Email Outreach

The Link Builder’s Guide to Email Outreach

Outreach Letters for Link Building [Real Examples]

How to: Get Links on Resource Pages

How to Get Links Using Broken Link Building?

Broken link building strategy is an effective & white hat strategy on getting some good links by helping webmasters. Again I prefer Richard Marriot, he is the one who got awesome links using this strategy and he described in detailed with video in his blog about how he achieved.

Broken Link Building Guide for Beginners

Deep Broken Link Building: Two Levels Deep

Broken Link Building Tools

Open Site Explorer

Ahrefs Backlink Checker Tool

Check My Links (Google Chrome App)

Screaming Frog SEO Spider Tool

Xenu’s Link Sleuth (To Find Broken Links)

Wayback Machine (Digital Archive of World Wide Web)

Boomerang Gmail

Domain Hunter Plus (Google Chrome Extension to Find Dead Links)

How to build links through competitors backlinks?

Competitor backlink analysis is a gold mine technique for every link builder on getting an awesome link to improve ranking and traffic for your website. Read the resources below to learn how to analyze and where to get that links.

How to steal your Competitors Rankings?

Competitor Backlink Analysis to Improve Ranking & Traffic

How to do Competitor Backlink Audit?