Web crawler
ElnathBot
ElnathBot is the web crawler operated by Elnath AI. This page explains what it does, what it fetches, how it follows robots.txt, and how to limit or block it.
At a glance
- User-Agent
ElnathBot/1.0 (+https://elnathai.com/bot)- robots.txt token
ElnathBot- IP address
8.210.34.206(machine-readable: /bot/ips.json)- Rate
- One connection per site, at least 2 seconds between requests
- Contact
- bot@elnathai.com
What ElnathBot does
Elnath AI builds a directory of manufacturers from information companies publish about themselves, to help buyers find the right factory. ElnathBot reads public pages on company websites and extracts facts such as:
- company name, location, main business and products;
- certifications, factory size and equipment;
- company news such as product launches, trade fairs and awards.
We use AI models to extract these facts. Listings are written by us from the facts and cite the source URL and the date we fetched it.
What we don't do
- We don't republish or sell page text.
- We don't use crawled content to train generative AI models.
- We don't collect personal data. We never store personal names, personal mobile numbers, personal webmail addresses or messaging IDs. For contact details we keep only a company's published landline, fax, e-mail on the company's own domain and business address, and we only display them partly masked.
What ElnathBot fetches
- Public pages only. No pages behind login or registration. We never submit forms, add to carts or crawl search results.
- Page text only. We don't download images, video, PDFs, archives or installers.
- A small number of pages. About 30 pages per site per visit, mainly the home, about, products, contact, certification and news pages. We check back roughly every 30 days.
- Occasional rendering. A few sites only show content after running scripts. For those we use a headless browser with the same identity and rules. It does not load images, video, fonts or third-party resources.
Identifying ElnathBot
Every request from ElnathBot carries this User-Agent:
ElnathBot/1.0 (+https://elnathai.com/bot)
ElnathBot only connects from 8.210.34.206. A machine-readable list is at https://elnathai.com/bot/ips.json and is updated before any change.
We never pretend to be a browser and never rotate IPs to get around a block. Traffic that claims to be ElnathBot but comes from another IP is not us.
How we follow robots.txt
ElnathBot reads and applies robots.txt according to RFC 9309:
- We follow the
User-agent: ElnathBotgroup (case-insensitive). If there is none, we follow theUser-agent: *group. AllowandDisallowuse longest match, with*and$supported.- We check the rules before every fetch and cache robots.txt for at most 24 hours, so changes take effect within a day.
- A 4xx response for robots.txt (such as 404) means no restrictions. A 5xx response or no connection means the whole site is off limits until we try again later.
Crawl-delayis honoured. We already keep to one connection per site and at least 2 seconds between requests. If you set a longer delay we use yours. If it is over 60 seconds, we pause crawling the site.- We sometimes use public web archives such as Common Crawl to fill in sites we cannot reach. Archived pages are filtered by the same rules your robots.txt sets for ElnathBot. If ElnathBot is blocked from a page, we won't use the archived copy either.
If we hit a CAPTCHA, a bot challenge, a 403 or a 429, we log it and stop visiting the site. We never try to get around these.
Limiting or blocking ElnathBot
Block the whole site
User-agent: ElnathBot
Disallow: /
Block some directories
User-agent: ElnathBot
Disallow: /internal/
Disallow: /downloads/
Slow down (at least 10 seconds between requests)
User-agent: ElnathBot
Crawl-delay: 10
E-mail us
Send your domain to bot@elnathai.com. We will stop crawling your site and delete the company data and saved pages we hold for it.
Corrections
If your company is listed and something is wrong, or you want to add or change information, e-mail bot@elnathai.com with the company name and the change. We will verify it and update the listing.
Contact
Questions about ElnathBot: bot@elnathai.com
Last updated 21 November 2026.