AskVaultBot is the crawler AskVault uses to read websites that its customers connect. It fetches public pages, sitemaps and product feeds from a site only after an AskVault customer (usually the site’s owner) adds that site to their workspace, so the site’s own AI support agent can answer its visitors’ questions.
A site is read when the customer first adds it, then re-checked on a schedule the customer picks from the options their plan includes. AskVaultBot limits how many requests it makes to one site at a time, and backs off when a site answers with errors or rate limits.
Product-feed requests send the user agent AskVaultBot/1.0 (+https://askvault.co/bot).
Page requests send a standard browser user agent. The scheduled check for changed pages sends Mozilla/5.0 AskVault-AutoResync/1.0. Requests are not cryptographically signed today.
When AskVault discovers a site’s pages, it reads robots.txt and skips URLs disallowed for User-agent: AskVault, or for User-agent: * when there is no AskVault group. To stop page discovery, add:
robots.txt is not checked when AskVault re-reads pages it already has, when it checks those pages for changes on a schedule, for a page a customer adds by hand, or for product feeds. To stop those requests too, email [email protected] with your domain.
Content AskVaultBot collects is used only to answer questions in the workspace of the customer who added the site. To write those answers it is processed by our AI providers (OpenAI and Google), as our Privacy Policy describes. AskVault does not use it to train AI models and does not sell it.
To have your site or product feed excluded, or to report a problem with how AskVaultBot behaves on your site, email [email protected] with your domain.