I have a site that is going to start selling it's products this month. I am frantically building social sites and our website.
I was wondering about site rank and opened an account on Alexa and asked to be certified. I had already removed a lot of worthless bots thru htacess and thru robot.txt.
Alexa is supposed to be a legit site and is owned by Amazon another supposed to be good site??????
At any rate, I first battled to get the bots into the site as they started making 403 errors to the tune of 6 lines every 5 minutes around the clock. for a total of 33K in 7 days.
After the site was certified, I asked for them to go away. As the bot "Alexabot" was their certify bot it's business was done I thought and I needed neither 6 lines every 5 minutes of 302 or 403 errors regardless.
ANd then I noticed something really weird.....
220.127.116.11 - - [26/Feb/2016:17:25:16 -0500] "GET /about-us.html HTTP/1.1" 200 25295 "http://www.google.co...x=&startPage=1""Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727)" 18.104.22.168 - - [26/Feb/2016:17:27:40 -0500] "GET /about-us.html HTTP/1.1" 200 25295 "http://www.google.com/search?q=US+mail+hold+-site%3Ausps.com+-site%3A.gov&tbs=qdr:d&rls=com.microsoft:en-us&ie=UTF-8&oe=UTF-8&startIndex=&startPage=1" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:22.214.171.124) Gecko/20120306 Firefox/3.6.28" 126.96.36.199 - - [26/Feb/2016:17:27:51 -0500] "GET /about-us.html HTTP/1.1" 200 25295 "http://www.google.com/search?q=US+mail+hold+-site%3Ausps.com+-site%3A.gov&tbs=qdr:d&rls=com.microsoft:en-us&ie=UTF-8&oe=UTF-8&startIndex=&startPage=1" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:188.8.131.52) Gecko/20120306 Firefox/3.6.28"
Here are my search results using standard Awstats program:
6 different keyphrases Search Percent us mail hold -site usps.com -site .gov 3 33.3 % pickle packer lids 2 22.2 % pickle pushing no 1 11.1 % probiotic jar 1 11.1 % www.ultimatepicklejar.com 1 11.1 % pickle pushing 1
AS you can see I am not a great place to visit according to Google or any other search engine for that matter.
Here is my question................ For those 3 lines to show in Awstats.... would not a user have to find a link to my site FROM THAT SEARCH PAGE?????
Here is the hits basically for the last 10 days.............. More than my legit traffic.
403 Forbidden 34,063 81.9 % 18.28 MB 302 Moved temporarily (redirect) 4,034 9.7 % 805.04 KB 404 Document Not Found (hits on favicon excluded) 2,345 5.6 % 19.56 MB 301 Moved permanently (redirect) 922 2.2 % 125.49 KB
I have repeatedly asked Alexa to stop and they do not honor the robot.txt either.
ALSO RUNNING A BASIC IP SEARCH FOR THE 3 SITES USING THE GOOGLE SEARCH SHOWED THEM TO BE FROM AMAZON?????
I am not a coder.... My handle is a spoof of Darth Vadar and is a carryover from my gaming days..............
Edited by ParseRaider, 28 February 2016 - 11:51 AM.