Detect bots/crawlers/spiders via the user agent.
$ npm i isbot
isBot(req.headers['user-agent'])
isBot('Googlebot/2.1 (+http://www.google.com/bot.html)') // true
isBot('Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36') // false
Add rules to user agent match RegExp
isBot('Mozilla/5.0') // false
var myList = [
'istat',
'newspaper',
'httpclient',
'^mozilla/\\d\\.\\d$',
];
isBot.extend(myList);
isBot('Mozilla/5.0') // true
Remove rules to user agent match RegExp (see existing rules in list.json
file)
isBot('Ceramic Tile Installation Guide') // true
var myList = [
'Ceramic Tile Installation Guide',
'NORAD National Defence Network'
];
isBot.exclude(myList);
isBot('Ceramic Tile Installation Guide') // false
Use lookbehind assertion, introduced in V8 version 4.9 to exclude "Cubot" from "bot" rule
isBot.exclude(['bot']);
isBot.extend(['(?<! cu)bot']); // Recognise cubot browser as legit browser
isBot('Mozilla/5.0 (Linux; Android 8.0.0; CUBOT_P20) ...') // false
isBot('Googlebot/2.1 (+http://www.google.com/bot.html)') // true
Return the respective match for bot user agent rule
isBot.find('Googlebot/2.1 (+http://www.google.com/bot.html)') // 'bot'