Create new file : robots.txt with this content :
User-agent: * # Indicates that the following rules apply to all robots Disallow: /cgi-bin # Do not reference cgi-bin directory Disallow: /wp-admin # Do not reference wordpress admin folder Disallow: /wp-includes # Do not reference wordpress includes files Disallow: /wp-content/plugins # Do not reference plugin directory Disallow: /wp-content/cache # Do not reference cache directory Disallow: /wp-content/themes # Do not reference themes directory Disallow: /*.php$ # Do not reference php files Disallow: /*.js$ # Do not reference javascript files Disallow: /*.inc$ # Do not reference includes files Disallow: /*.css$ # Do not reference css styles Allow: /wp-content/uploads # Reference images # Google Image User-agent: Googlebot-Image Disallow: Allow: /* # Allow google image bot on entire site # Google AdSense User-agent: Mediapartners-Google* Disallow: Allow: /* # Allow AdSense bot on entire site # BEGIN XML-SITEMAP-PLUGIN Sitemap: http://www.devinfo007.com/sitemap.xml # link to your sitemap # END XML-SITEMAP-PLUGIN
Save and upload it to your webroot
