This year, 59% of all site visits to Distil Networks’ websites were from bots, compared to 45% last year. Meanwhile, human traffic accounted for 41%, down from 55% last year.
The majority of bot traffic (35%) was from ‘good bots’, bots that comply with ‘robot.txt’ files and bring value to a website, while traffic from ‘bad bots’, those that don’t comply, accounted for 23%, down slightly from 24% last year.
According to Distil CEO Rami Essaid, the reason for the drop in percentage of ‘bad bots’ wasn’t due to volume, but because the proportion of ‘good bots’ rose dramatically.
“The reason that good bots increased so much is that social medial companies have doubled the number of bots they used in the past year,” says Essaid. “Google’s rate of crawling went up 30% in the past two weeks because of the change to its algorithm.”
‘Bad bots’ include those that scrape data from websites, are malicious and looking for website vulnerabilities and those that engage in click fraud and spam. An important development is the increasing sophistication of bot behavior. Distil’s report found that 41% of the ‘bad bots’ they uncovered attempted to mimic human behavior in order to get into a web site’s infrastructure. Worryingly, 23% of these ‘bad bots’ are so good at this that they are almost unstoppable by technologies employed by most web application firewalls in use today.
Source : http://www.bizreport.com/2015/06/bots-now-account-for-more-internet-traffic-than-humans.html